Information processing device, information processing method, imaging device, and image transfer system

Abstract

Provided is an information processing device which serves as a server device that performs a transfer from an imaging device to a client terminal, the information processing device comprising a control unit that executes: an initial information transfer process in which, in response to receiving related information that is related to an image captured by the imaging device and that has a data quantity which is less than that of the captured image, additional information is acquired on the basis of the related information, and the related information and the additional information are transmitted to the client terminal; and an actual image transfer process in which an image transmitted from the imaging device is transmitted to the client terminal in accordance with request information generated by a selection process at the client terminal.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, an imaging device, and an image transfer system, and in particular to a technology suitable for transferring an image captured by an imaging device to another device. [Background technology]

[0002] 2. Description of the Related Art It is common to transmit images (still images or moving images) captured by an imaging device to another device, such as a computer device. The following Patent Document 1 discloses a technique for transmitting related data, which has a smaller amount of data, prior to image data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-150781 A Summary of the Invention [Problem to be solved by the invention]

[0004] For news reporting purposes such as television broadcasting and web distribution, there are cases where still images and videos taken by a cameraman on-site must be delivered immediately after the cameraman has taken the images. In this case, the cameraman connects the imaging device to a network at the shooting location and transmits the captured images over the network for delivery. At the delivery destination device (such as a broadcasting station's device, referred to as a "client terminal"), a separate selection person waiting separately selects the delivered images and uses the selected images in news articles.

[0005] In such cases, sending large images (image data of the quality / volume used for news reporting) from the imaging device to an external device (such as an intermediary cloud server or client terminal) can increase the network load and take time, which can also result in a delay before the image selection process can be carried out. Furthermore, if all captured images are transmitted from the imaging device to an external device, the storage capacity of the transmission destination device must be increased, increasing the burden on the system.

[0006] Therefore, the present disclosure proposes a technique that enables appropriate image transmission without significantly increasing the load on network communications and systems. [Means for solving the problem]

[0007] An information processing device according to the present technology includes a control unit that performs an initial information transfer process of obtaining additional information based on related information and transmitting the related information and the additional information to a client terminal in response to receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, and an original image transfer process of transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information. For example, assume that an information processing device is capable of transferring information to and from an imaging device and also to and from a client terminal. This information processing device first transmits related information and additional information to the client terminal before transferring image data (original image) as a captured image. Then, the original image is transferred from the imaging device to the client terminal in response to request information obtained as a result of a selection process based on the related information and additional information.

[0008] In the information processing device according to the present technology described above, it is considered that the control unit obtains the additional information by performing a database search based on the related information in the initial information transfer process. By performing a database search based on the related information supplied from the imaging device, more detailed information about the image (subject) can be obtained from the related information. This information is referred to as additional information.

[0009] In the information processing device relating to the above-mentioned present technology, the related information includes an image having a smaller amount of data representing an image captured by an imaging device, and the control unit, in the initial information transfer process, can obtain the additional information by image recognition processing of the image included in the related information. When the related information supplied from the imaging device includes an image such as a thumbnail image or a screen nail image, information about the subject can be obtained by analyzing the image. This information is referred to as additional information.

[0010] In the information processing device related to the above-mentioned present technology, the control unit is considered to perform a switching process between a plurality of transmission patterns, including a transmission pattern that adds the additional information and a transmission pattern that does not add the additional information, as the initial information transfer process. The transmission pattern can be changed to a plurality of patterns, such as a pattern in which additional information is obtained based on related information supplied from an imaging device and the related information and the additional information are transmitted to a client terminal, a pattern in which the related information supplied from the imaging device is transferred to the client terminal as is, or a pattern in which the content of the related information is different.

[0011] In the information processing device related to the above-mentioned present technology, the control unit may perform a process of transmitting audio information included in the related information to the client terminal as a transmission of a predetermined transmission pattern, which is one of a plurality of transmission patterns. For example, when audio information associated with a still image is transferred from an imaging device, the audio information is sent to the client terminal as is.

[0012] In the information processing device related to the above-mentioned present technology, it is considered that the control unit performs a process of transmitting text data, which is obtained by converting voice information included in the related information into text data, to the client terminal as a transmission of a predetermined transmission pattern, which is one of a plurality of transmission patterns. For example, when audio information associated with a still image is transferred from an imaging device, the audio information is converted into text data and then transmitted to the client terminal.

[0013] In the information processing device according to the present technology described above, it is considered that the control unit performs the switching process in accordance with a result of determining a communication situation. As the initial information transfer process, various transmission patterns are used depending on the communication situation.

[0014] In the information processing device according to the present technology described above, it is considered that the control unit performs the switching process based on information about a subject to be photographed notified from the imaging device. Depending on the subject of the image capture, for example, the type of event or the type of sport, a first process in which additional information is acquired in the initial information transfer process and a second process in which this is not performed are selectively used.

[0015] In the information processing device according to the present technology described above, it is considered that the control unit performs an estimation process of a subject to be photographed by the imaging device, and performs the switching process based on information of the estimated subject to be photographed. If it is possible to estimate the subject to be photographed by the imaging device, it is possible to selectively use a first process in which additional information is acquired as the initial information transfer process, and a second process in which no additional information is acquired, based on the estimation result.

[0016] In the information processing device according to the present technology described above, it is considered that the control unit performs processing to notify the client terminal or the imaging device of the transmission pattern of the initial information transfer processing set by the switching processing. When the information processing device switches between the first process and the second process, the image capture device and the client terminal are notified of the transmission pattern to be executed as a result.

[0017] In the information processing device according to the present technology described above, it is considered that the related information is meta information associated with an image captured by the imaging device. That is, the information is associated with an image on the imaging device side and transmitted to the information processing device. For example, various types of information are assumed, such as information on the imaging device, information on the cameraman, information on the imaging location and time, analysis information on the image on the imaging device side, various parameters related to imaging operation and processing, sensing information of various sensors in the imaging device at the time of imaging, information on image data / files, thumbnail images, screen nail images, text information and voice memo information associated with the image by the cameraman, etc.

[0018] In the information processing device according to the present technology described above, it is considered that the related information is an image with a smaller amount of data, which represents an image captured by the imaging device. Examples of images with smaller amounts of data include thumbnail images, screennail images, and proxy videos.

[0019] In the information processing device according to the present technology described above, it is considered that the related information is an image representing a grouping image which is a plurality of images continuously captured by the imaging device. When a series of images captured by continuous shooting are grouped together, an image that expresses the entire series as a single image, for example a thumbnail image of the top image, is used as related information.

[0020] In the information processing device according to the present technology described above, it is considered that the related information is audio information recorded together with a moving image captured by the imaging device. For example, in the initial information transfer process, the information processing device receives audio information recorded simultaneously with video recording as related information, and transmits it to the client terminal.

[0021] In the information processing device according to the present technology described above, it is considered that the related information is time-series data at the time of capturing a moving image captured by the imaging device. Time series data is information that expresses the progress of imaging, such as position information for each time when imaging is performed.

[0022] In the information processing device related to the above-mentioned present technology, the control unit may perform, as the initial information transfer process, a process of transmitting first related information to the client terminal together with or without the additional information, a process of transferring request information generated by a selection process in the client terminal based on a transmission including the first related information to the imaging device, and a process of transmitting second related information transmitted from the imaging device to the client terminal in response to the request information. That is, as an initial information transfer process, before the original image is transmitted to the client terminal 3, information is transmitted a number of times.

[0023] In the information processing device according to the present technology described above, it is considered that the first related information is information having a smaller data amount than the second related information. For example, the first related information may be meta information indicating the time and place of photography, and the second related information may be a thumbnail image.

[0024] The imaging device according to the present technology includes a control unit that performs a related information transmission process to transmit related information associated with an image captured and having a smaller amount of data than the image to an external information processing device, and an original image transmission process to transmit an image indicated in request information received from the information processing device to the information processing device in response to the request information. This allows the imaging device to transmit related information prior to transmitting a captured image.

[0025] In the image transfer system according to the present technology, information communication is possible between an imaging device and an information processing device, and information communication is possible between the information processing device and a client terminal. The imaging device has the above-mentioned configuration, and the information processing device also has the above-mentioned configuration. Information can be transferred between the imaging device and the information processing device directly or via other devices or transmission paths, and information can be transferred between the information processing device and the client terminal directly or via other devices or transmission paths. Between such devices, related information, which has a smaller amount of data than the captured image, is first transferred and a selection is made on the client terminal side. After that, the original image is transmitted from the imaging device to the client terminal in accordance with the selection. [Brief description of the drawings]

[0026] [Figure 1] 1 is an explanatory diagram of a configuration example of an image transfer system according to an embodiment of the present technology; [Diagram 2] 1 is a block diagram of an imaging apparatus according to an embodiment; [Diagram 3] FIG. 1 is a block diagram of an information processing apparatus according to an embodiment. [Figure 4] FIG. 4 is an explanatory diagram of a first transmission pattern according to the embodiment. [Diagram 5] 11A to 11C are explanatory diagrams of UI screens etc. in a first transmission pattern according to the embodiment. [Figure 6] 11 is a flowchart of processing during image capture by an imaging device that can be applied to the first, second, third, and fourth transmission patterns of the embodiment. [Figure 7] 13 is a flowchart of a process performed by a server device when receiving meta-information, which can be applied to a first transmission pattern according to the embodiment. [Figure 8] 13 is a flowchart of a process performed by a server device when receiving an image request, which can be applied to first to seventh transmission patterns of the embodiment. [Figure 9] 13 is a flowchart of a process performed by an imaging device upon receiving an image request, which can be applied to first to sixth transmission patterns of the embodiment. [Figure 10]13 is a flowchart of a process performed by a server device when receiving an image, which can be applied to the first, third to seventh transmission patterns of the embodiment. [Figure 11] FIG. 11 is an explanatory diagram of a second transmission pattern according to the embodiment. [Figure 12] 13A to 13C are explanatory diagrams of UI screens etc. in a second transmission pattern according to the embodiment. [Figure 13] 13 is a flowchart of a process performed by a server device when receiving meta-information, which can be applied to second, third and fourth transmission patterns according to the embodiment. [Figure 14] 13 is a flowchart of a process performed by a server device when receiving an image, which can be applied to a second transmission pattern of the embodiment. [Figure 15] FIG. 11 is an explanatory diagram of a third transmission pattern according to the embodiment. [Figure 16] 13A to 13C are explanatory diagrams of UI screens etc. in third and fourth transmission patterns according to an embodiment. [Figure 17] 13 is a flowchart of processing during image capture by an imaging device that can be applied to a fourth transmission pattern of the embodiment. [Figure 18] FIG. 13 is an explanatory diagram of a fifth transmission pattern according to the embodiment. [Figure 19] 13A to 13C are explanatory diagrams of UI screens etc. in a fifth transmission pattern according to an embodiment. [Figure 20] 13 is a flowchart of processing during image capture by an imaging device that can be applied to fifth and sixth transmission patterns of the embodiments. [Figure 21] 13 is a flowchart of a process performed by a server device when receiving meta-information, which can be applied to a fifth transmission pattern according to the embodiment. [Figure 22] FIG. 13 is an explanatory diagram of a sixth transmission pattern according to the embodiment. [Figure 23] 13A to 13C are explanatory diagrams of UI screens etc. in a sixth transmission pattern according to the embodiment. [Figure 24] 13 is a flowchart of a process performed by a server device when receiving meta-information, which can be applied to a sixth transmission pattern according to the embodiment. [Diagram 25] FIG. 13 is an explanatory diagram of a seventh transmission pattern according to the embodiment. [Figure 26]FIG. 13 is an explanatory diagram of a UI screen etc. in a seventh transmission pattern according to the embodiment. [Figure 27] 13 is a flowchart of processing during image capture by an imaging device that can be applied to a seventh transmission pattern of the embodiment. [Figure 28] 13 is a flowchart of a process performed by an imaging device upon receiving an image request, which can be applied to a seventh transmission pattern in the embodiment. [Figure 29] FIG. 11 is an explanatory diagram of a switching process example I according to an embodiment. [Diagram 30] 11 is a flowchart of a server device switching process according to an embodiment. [Diagram 31] FIG. 11 is an explanatory diagram of a UI display related to switching of the embodiment. [Diagram 32] 11 is an explanatory diagram of a UI display for requesting a switching instruction according to an embodiment. FIG. [Diagram 33] 13 is a flowchart of a situation determination process according to an embodiment. [Diagram 34] FIG. 11 is an explanatory diagram of a switching process example II according to an embodiment. [Diagram 35] 11 is a flowchart of a server device switching process according to an embodiment. [Diagram 36] FIG. 11 is an explanatory diagram of a switching process example III according to an embodiment. [Figure 37] 11 is a flowchart of a process performed when project information is received by the server device according to the embodiment. [Figure 38] 11 is a flowchart of a server device switching process according to an embodiment. [Figure 39] FIG. 13 is an explanatory diagram of an eighth transmission pattern according to the embodiment. [Diagram 40] FIG. 13 is an explanatory diagram of a UI screen etc. in an eighth transmission pattern according to the embodiment. [Diagram 41] 13 is a flowchart of a process for recording a moving image by an imaging device that can be applied to an eighth transmission pattern of the embodiment. [Diagram 42] 13 is a flowchart of a process performed by a server device when receiving a proxy video, which can be applied to an eighth transmission pattern according to the embodiment. [Diagram 43]13 is a flowchart of a process performed by an imaging device upon receiving a video clip request, which can be applied to eighth, ninth, and tenth transmission patterns of the embodiment. [Diagram 44] 13 is a flowchart showing a process performed by the server device when receiving a main video clip, which can be applied to the eighth, ninth, and tenth transmission patterns of the embodiment. [Diagram 45] FIG. 13 is an explanatory diagram of a ninth transmission pattern according to the embodiment. [Diagram 46] FIG. 13 is an explanatory diagram of a UI screen etc. in a ninth transmission pattern according to the embodiment. [Figure 47] 13 is a flowchart of a process for recording a moving image by an imaging device that can be applied to a ninth transmission pattern in the embodiment. [Figure 48] 13 is a flowchart of a process performed by a server device when receiving an audio-only file, which can be applied to a ninth transmission pattern according to the embodiment. [Figure 49] FIG. 17 is an explanatory diagram of a tenth transmission pattern according to the embodiment. [Figure 50] FIG. 19 is an explanatory diagram of a UI screen etc. in a tenth transmission pattern according to an embodiment. [Figure 51] 23 is a flowchart of a process for recording a moving image by an imaging device that can be applied to a tenth transmission pattern of the embodiment. [Figure 52] 23 is a flowchart of a process performed by a server device when receiving a time-series file, which can be applied to a tenth transmission pattern according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] The embodiments will be described below in the following order. <1. System configuration> <2. Configuration of the imaging device, server device, and client terminal> <3. First transmission pattern> <4. Second transmission pattern> <5. Third and fourth transmission patterns> <6. Fifth transmission pattern> <7. 6th transmission pattern> <8. 7th transmission pattern> <9. Switching Processing Example I> <10. Switching Processing Example II> <11. Switching Processing Example III> <12.8th transmission pattern> <13. 9th transmission pattern> <14. 10th transmission pattern> <15. Summary and variations>

[0028] First, the definitions of various terms will be provided. The term "image" refers to various types of image data, including both still images and moving images. The term "main image" refers to the target image to be delivered from the imaging device 1 shown in Fig. 1 to the client terminal 3. The main image includes still images and videos. When specifically referring to a video, it is written as "main video". "Related information" refers to information related to an image captured by the imaging device 1, regardless of the content of the information. For example, there is so-called meta information including information about the photographer of the image, date and time, location, and various sensing data of the imaging device, voice memo information, and images with a smaller amount of data than the original image (thumbnail images for still images and videos, screennail images, proxy videos for videos, etc.). The "additional information" refers to information that the server device 2 has searched for, extracted, or generated based on the related information, regardless of the content of the information. The "initial information transfer process" refers to the process of transferring related information and additional information before the transmission of the main image. This process may be performed multiple times. The "original image transfer process" refers to the process of transferring the original image. The "transmission pattern" refers to a pattern that is a variation of the procedure leading up to the transfer of the original image between the imaging device 1, the server device 2, and the client terminal 3 and the information content to be transferred. In the embodiment, the first to tenth transmission patterns are shown, which are different from the information content type and the information content to be transferred depending on whether the original image is a still image / continuously shot images / video, and the like, but various patterns are assumed, such as a pattern of whether or not to perform an initial information transfer process in each transmission pattern, a pattern of transferring information once from the imaging device 1 to the client terminal 3 via the server device 2 as the initial information transfer process, or a pattern of transferring information multiple times.

[0029] <1. System configuration> An example of the configuration of an image transfer system according to an embodiment is shown in Fig. 1. The image transfer system is basically configured with one or more imaging devices 1, a server device 2, and a client terminal 3, which are capable of communicating with each other via a network 6.

[0030] There are various types of imaging devices such as video cameras and still cameras as the imaging device 1. The imaging device 1 shown in the figure is assumed to be a camera used by cameramen and journalists at sporting and event venues, reporting sites, and the like.

[0031] Both the server device 2 and the client terminal 3 are information processing devices formed by computer equipment, but since the server device 2 provides a service for image transfer and the client terminal 3 is an information processing device for a customer who uses the transfer service provided by the server device 2, for the purpose of distinction in this disclosure they are referred to as a "server device" and a "client terminal."

[0032] The server device 2 is assumed to function as a so-called cloud server, although it is not limited to the cloud server mode. In the server device 2, a control unit 2a is shown, and the control unit 2a executes an initial information transfer process and an original image transfer process as described later.

[0033] Furthermore, the server device 2 can obtain various types of information by referring to a database 7 (hereinafter, database will be abbreviated as "DB"), and can obtain text data by converting voice data into text data using a text conversion engine 8. The DB 7 may be a DB configured in a storage device that can be used locally by the server device 2, or a DB configured in a storage device that can search for information via various networks. In other words, the DB 7 may be in any form as long as it is a DB that the server device 2 (control unit 2a) can access and obtain information from.

[0034] The client terminal 3 indicates an information processing device to which a cameraman delivers images, such as a broadcasting station, a newspaper company, a news agency, etc. The broadcasting station is just one example, and the delivery destination is not limited.

[0035] The network 6 may be, for example, the Internet, a home network, a LAN (Local Area Network), a satellite communication network, or any other type of network.

[0036] 1, the imaging device 1 communicates with the server device 2 via a base station 4. The base station 4 is, for example, a base station for 4G communication (fourth generation mobile communication system) and 5G communication (fifth generation mobile communication system). A MEC (Mobile Edge Computing) 5 may be disposed in the base station 4. Communication between the imaging devices 1 may be performed via processing by the MEC 5, and some functions of the server device 2 may be executed by the MEC 5.

[0037] 1, if the user of the imaging device 1 is, for example, a cameraman working for a broadcasting station, newspaper company, etc., images captured by the cameraman at an event venue can be transferred to a client terminal 3 installed at the broadcasting station, etc. via the server device 2. At the client terminal 3, a person in charge selects the images sent and uses them for reporting. In situations where immediate delivery is required after shooting for news reports and the like, it is expected that large volumes of high-resolution images will be transferred, which can increase the load on the network and the processing load on each component. In addition, if a large amount of original images are delivered to the client terminal 3, the burden on memory capacity will increase and the selection work by the person in charge will take time, which may impair the speed required for news reporting, etc. Therefore, in this embodiment, the server device 2 separates the transfer of the original artwork up to the delivery into an initial information transfer process and an original artwork transfer process, thereby reducing the load on network communications and memory capacity, and facilitating the selection process, etc. In addition, the transfer pattern (procedure and information content) can be switched depending on the situation, etc., so that image transfer appropriate for the time is performed.

[0038] <2. Configuration of the imaging device, server device, and client terminal> Below, examples of the configuration of the imaging device 1, the server device 2, and the client terminal 3 will be described. First, an example of the configuration of the imaging device 1 will be described with reference to FIG. The imaging device 1 has, for example, a lens system 11, an imaging element unit 12, a camera signal processing unit 13, a recording control unit 14, a display unit 15, a communication unit 16, an operation unit 17, a camera control unit 18, a memory unit 19, a driver unit 22, a sensor unit 23, an audio input unit 25, and an audio processing unit 26.

[0039] The lens system 11 includes lenses such as a zoom lens and a focus lens, an aperture mechanism, etc. The lens system 11 guides light (incident light) from a subject and collects the light on the imaging element unit 12.

[0040] The imaging element unit 12 includes an image sensor 12a (imaging element) such as a complementary metal oxide semiconductor (CMOS) type or a charge coupled device (CCD) type. The image sensor unit 12 performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc., on the electric signal obtained by photoelectrically converting the light received by the image sensor 12a, and further performs A / D (Analog / Digital) conversion processing. Then, the image signal as digital data is output to the downstream camera signal processor 13 and camera controller 18.

[0041] The camera signal processing unit 13 is configured as an image processor, for example, using a DSP (Digital Signal Processor) etc. The camera signal processing unit 13 performs various types of signal processing on the digital signal (captured image signal) from the imaging element unit 12. For example, as camera processes, the camera signal processing unit 13 performs pre-processing, synchronization processing, YC generation processing, resolution conversion processing, file formation processing, etc.

[0042] In the pre-processing, the captured image signal from the imaging element unit 12 is subjected to a clamping process for clamping the R, G, and B black levels to a predetermined level, a correction process between the R, G, and B color channels, and the like. In the synchronization process, a color separation process is performed so that the image data for each pixel has all the color components R, G, and B. For example, in the case of an image sensor that uses a Bayer array color filter, a demosaic process is performed as the color separation process. In the YC generation process, a luminance (Y) signal and a color (C) signal are generated (separated) from R, G, B image data. In the resolution conversion process, the image data that has been subjected to various types of signal processing is subjected to the resolution conversion process.

[0043] In the file creation process, for example, image data that has been subjected to the various processes described above is subjected to compression encoding for recording or communication, formatting, generation and addition of meta information, etc., to generate a file for recording or communication. For example, still image files may be generated in formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), GIF (Graphics Interchange Format), etc. It may also be possible to generate image files in the MP4 format used for recording video and audio conforming to MPEG-4. It is also possible to generate an image file as raw image data.

[0044] The camera signal processing unit 13 generates meta information that includes information on processing parameters within the camera signal processing unit 13, various control parameters obtained from the camera control unit 18, information indicating the operating status of the lens system 11 and the image sensor unit 12, mode setting information, imaging environment information (date, time, location, etc.), identification information of the imaging device itself, information on the attached lens, pre-registered cameraman information (name, identification information), IPTC (International Press Telecommunications Council) metadata, etc. IPTC metadata is meta-information in a format developed by a media company group, and is capable of describing a variety of information, such as "description / caption," "description writer," "headline," and "keywords."

[0045] The recording control unit 14 performs recording and playback on a recording medium, such as a non-volatile memory, for example. The recording control unit 14 performs processing for recording image files such as moving image data and still image data, thumbnail images, screen nail images, and the like on the recording medium, for example. The actual form of the recording control unit 14 may be various. For example, the recording control unit 14 may be configured as a flash memory built into the imaging device 1 and a write / read circuit for the memory. The recording control unit 14 may also be in the form of a card recording / playback unit that performs recording / playback access to a recording medium that can be detached from the imaging device 1, such as a memory card (such as a portable flash memory). The recording control unit 14 may also be realized as an HDD (Hard Disk Drive) built into the imaging device 1.

[0046] The display unit 15 is a display unit that displays various information to the person taking the image, and is, for example, a display panel or viewfinder using a display device such as a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display arranged on the housing of the imaging device 1. The display unit 15 executes various displays on the display screen based on instructions from the camera control unit 18 . For example, the display unit 15 displays a reproduced image of image data read from a recording medium by the recording control unit 14. Further, image data of the captured image that has been resolution-converted for display by the camera signal processing unit 13 is supplied to the display unit 15, and the display unit 15 may perform display based on the image data of the captured image in response to an instruction from the camera control unit 18. This causes a so-called through image (monitoring image of the subject), which is an image captured while checking the composition or recording a video, to be displayed. Further, based on instructions from the camera control unit 18, the display unit 15 executes display of various operation menus, icons, messages, etc., that is, GUI (Graphical User Interface) on the screen.

[0047] The communication unit 16 performs data communication and network communication with external devices in a wired or wireless manner. For example, the captured image data (still image files and video files) and meta information are transmitted and output to an external display device, recording device, playback device, or the like. Furthermore, the communication unit 16 serves as a network communication unit and performs communication via various networks 6, such as the Internet, a home network, and a LAN (Local Area Network), and can transmit and receive various data between a server, a terminal, and the like on the network. For example, in the case of this embodiment, the communication unit 16 performs communication processing to transmit captured image data (such as the above-mentioned image file) and meta information to the server device 2. The imaging device 1 may also be capable of communicating information with a smartphone or tablet terminal owned by a photographer or the like through the communication unit 16, for example, by short-distance wireless communication such as Bluetooth (registered trademark), WI-FI (registered trademark), or NFC (Near Field Communication), or infrared communication. In this case, images can be transferred via a smartphone or the like. Furthermore, the imaging device 1 and other devices may be capable of communicating with each other via wired connection communication.

[0048] The operation unit 17 collectively refers to input devices for a user to input various operations. Specifically, the operation unit 17 refers to various operators (keys, dials, touch panel, touch pad, etc.) provided on the housing of the imaging device 1. An operation performed by the user is detected by the operation unit 17, and a signal corresponding to the input operation is sent to the camera control unit .

[0049] The camera control unit 18 is configured by a microcomputer (arithmetic processing device) equipped with a CPU (Central Processing Unit). The memory unit 19 stores information and the like used for processing by the camera control unit 18. The illustrated memory unit 19 collectively represents, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory, and the like. The memory unit 19 may be a memory area built into a microcomputer chip serving as the camera control unit 18, or may be configured by a separate memory chip. The camera control unit 18 executes a program stored in the ROM, flash memory, or the like of the memory unit 19 to control the entire imaging device 1 . For example, the camera control unit 18 controls the operation of each necessary unit with respect to controlling the shutter speed of the image sensor unit 12, instructing various signal processing in the camera signal processing unit 13, imaging operations and recording operations in response to user operations, playback operations of recorded image files, operations of the lens system 11 such as zoom, focus, and aperture adjustment in the lens barrel, user interface operations, etc.

[0050] The RAM in the memory unit 19 is used as a working area for various data processing by the CPU of the camera control unit 18, for temporarily storing data, programs, and the like. The ROM and flash memory (non-volatile memory) in memory unit 19 are used to store the OS (Operating System) that the CPU uses to control each part, content files such as image files, application programs for various operations, firmware, various setting information, etc. The various types of setting information include communication setting information, exposure settings, shutter speed settings, and mode settings as setting information related to imaging operations, white balance settings, color settings, and settings related to image effects as setting information related to image processing, and custom key settings and display settings as setting information related to operability.

[0051] The driver unit 22 includes, for example, a motor driver for a zoom lens drive motor, a motor driver for a focus lens drive motor, a motor driver for a diaphragm mechanism motor, and the like. These motor drivers apply drive currents to the corresponding drivers in response to instructions from the camera control unit 18, thereby moving the focus lens and zoom lens, opening and closing the aperture blades of the aperture mechanism, and so on.

[0052] The sensor unit 23 collectively represents various sensors mounted on the imaging device. The sensor unit 23 is equipped with, for example, an IMU (inertial measurement unit), and can detect angular velocity using a three-axis angular velocity (gyro) sensor of pitch, yaw, and roll, and can detect acceleration using an acceleration sensor. Furthermore, the sensor unit 23 may be equipped with, for example, a position information sensor, an illuminance sensor, or the like.

[0053] The audio input unit 25 has, for example, a microphone and a microphone amplifier, and outputs an audio signal obtained by collecting surrounding sounds. The audio processing unit 26 performs processing such as converting the audio signal obtained by the audio input unit 25 into a digital audio signal, AGC processing, sound quality processing, noise reduction processing, etc. The audio data that has undergone these processes is output to the camera signal processing unit 13 and the camera control unit 18. For example, when a moving image is captured, the audio data is processed in the camera control unit 18 as audio data accompanying the moving image.

[0054] The audio data can also be converted into an audio file in the camera signal processing unit 13 or the camera control unit 18 as audio memo data as a so-called voice memo during image capture or the like. The audio file can be associated with the image file and recorded on a recording medium by the recording control unit 14, or can be transmitted and output from the communication unit 16 together with the image file.

[0055] Next, a configuration example of an information processing device 70 serving as the server device 2 or the client terminal 3 is shown in FIG. A CPU 71 of an information processing device 70 executes various processes according to programs stored in a ROM 72 or a non-volatile memory unit 74 such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), or programs loaded from a storage unit 79 to a RAM 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes, as appropriate. The CPU 71, ROM 72, RAM 73, and non-volatile memory unit 74 are interconnected via a bus 83. To this bus 83, an input / output interface 75 is also connected.

[0056] The input / output interface 75 is connected to an input unit 76 including an operator and an operating device. For example, the input unit 76 may be various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, or a remote controller. An operation by the user is detected by the input unit 76, and a signal corresponding to the input operation is interpreted by the CPU 71.

[0057] Further, a display unit 77 formed of an LCD or an organic EL panel, etc., and an audio output unit 78 formed of a speaker, etc. are connected to the input / output interface 75 either integrally or separately. The display unit 77 is a display unit that performs various displays, and is configured, for example, by a display device provided in the housing of the information processing device 70, a separate display device connected to the information processing device 70, or the like. The display unit 77 displays images for various types of image processing, moving images to be processed, etc., on the display screen based on instructions from the CPU 71. Furthermore, the display unit 77 displays various operation menus, icons, messages, etc., that is, as a GUI (Graphical User Interface), based on instructions from the CPU 71.

[0058] The input / output interface 75 may be connected to a storage unit 79 configured with a hard disk or solid-state memory, or a communication unit 80 configured with a modem or the like.

[0059] The communication unit 80 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, and the like.

[0060] A drive 81 is also connected to the input / output interface 75 as required, and a removable recording medium 82 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately mounted thereon. The drive 81 allows data files such as image files and various computer programs to be read from the removable recording medium 82. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the sound output unit 78. Furthermore, the computer programs and the like read from the removable recording medium 82 are installed in the storage unit 79 as necessary.

[0061] In this information processing device 70, for example, software for the processing of this embodiment can be installed via network communication by the communication unit 80 or via a removable recording medium 82. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, etc.

[0062] For example, when the information processing device 70 is the server device 2, software for transfer processes including the initial information transfer process and the main image transfer process described below, and for switching the transmission pattern in the transfer process, etc., is installed. In this case, the CPU 71 functions as the control unit 2a in Fig. 1 and performs the necessary processes. In addition, various communications described below are performed via the communication unit 80.

[0063] Furthermore, when the information processing device 70 is the client terminal 3, in the course of the transfer process described below, interface processing using the input unit 76, the display unit 77, etc. is executed under the control of the CPU 71. Furthermore, various communications are performed via the communication unit 80.

[0064] <3. First transmission pattern> Various transmission patterns as the image transfer process performed from the imaging device 1 to the client terminal 3 via the server device 2 will be described.

[0065] FIG. 4 shows the communication procedure and information contents as the first transmission pattern for image transfer. In the diagrams of the first to tenth transmission patterns described below, transmission and reception between the imaging device 1, server device 2, and client terminal 3 are indicated by arrows with the symbols TP** ("**" are numbers), and processing of each device is indicated by the symbol PS**, as in Fig. 4. The vertical direction of the diagram (from top to bottom) corresponds to the progression of time. The first transmission pattern shown in FIG. 4 is performed in the following procedure.

[0066] ·Communication TP1 Meta information is transmitted from the imaging device 1 to the server device 2 as related information about the captured image. At this point, the actual image is not transmitted. The imaging device 1 transmits meta information about one or more images in response to, for example, an operation by a cameraman.

[0067] Processing PS1 The server device 2 searches the DB7 based on the meta information of each image received, and derives additional information. For example, if the location and time of the image are obtained as meta information, the content of the competition and the names of the athletes may be identified by searching the DB7 using the location and time of the image. The names of the competitions and the athletes obtained in this way are examples of additional information. In this way, based on some kind of meta-information transmitted from the imaging device 1, the server device 2 searches the database to obtain various kinds of information as additional information. In addition, in this process PS1, it is assumed that a DB search is performed for each of the meta information regarding the multiple images transmitted from the image capture device 1 to obtain additional information.

[0068] ·Communication TP2 The server device 2 transmits to the client terminal 3 a pair of meta information and additional information for each image.

[0069] Processing PS2 The client terminal 3 performs UI (User Interface) processing using the received meta information and additional information. For example, the client terminal 3 executes a display as shown in Fig. 5A on the display unit 77. In this example, it is assumed that the meta information includes information on the cameraman (photographer), the time of shooting, and the location of shooting, and the name of the sport and the names of the athletes are extracted as additional information based on the time of shooting and the location of shooting. Based on such meta information and additional information, an information content list 30 corresponding to each image is displayed. Fig. 5A shows an example of displaying a list of information corresponding to six images, for example, information on the photographer included in the meta information, and the name of the sport and the names of the athletes obtained as additional information. Also, check boxes 31 are arranged corresponding to each piece of information, so that the person in charge at the client terminal 3 can make a selection. Furthermore, a thumbnail request button 32 is displayed so that the person in charge can operate it.

[0070] Such a UI screen is displayed so that the person in charge of the client terminal 3 can first check only the information content corresponding to the images and roughly select the necessary images. FIG. 5A shows a state in which the person in charge has checked check boxes 31 for the first, second, and sixth information in the information content list 30. Furthermore, if the thumbnail request button 32 is operated in this state, it is deemed that the person in charge has requested thumbnail images of the images corresponding to the first, second, and sixth pieces of information. Such an operation is performed when a person in charge checks meta information and additional information and wishes to confirm, by means of thumbnails, what kind of image content they actually represent. In the process PS2, the CPU 71 of the client terminal 3 controls the display of such a UI screen and detects the corresponding input by the person in charge.

[0071] For example, thumbnails / screennails are known as images that display image content in a low-resolution, small amount of data, but in this disclosure, "thumbnail" broadly refers to images that display the original image (still image or video) as a low-resolution still image, such as images generally called thumbnails / screennails.

[0072] ·Communication TP3 The client terminal 3 transmits a thumbnail image request list to the server device 2. That is, in response to the person in charge checking the checkbox 31 and operating the thumbnail request button 32 in the UI processing of the process PS2, the client terminal 3 generates a thumbnail image request list including list information of images corresponding to the checked information and information requesting the thumbnails of those images, and transmits the thumbnail image request list.

[0073] ·Communication TP4 The server device 2 transfers the thumbnail image request list received from the client terminal 3 to the imaging device 1 .

[0074] ·Communication TP5 In response to receiving the thumbnail image request list, the imaging device 1 transmits to the server device 2 thumbnail images of one or more images listed in the thumbnail image request list.

[0075] ·Communication TP6 The server device 2 transfers the thumbnail image received from the imaging device 1 to the client terminal 3.

[0076] Processing PS3 The client terminal 3 performs UI processing using the received thumbnail image. For example, the client terminal 3 displays the thumbnail list 34 as shown in Fig. 5B on the display unit 77. That is, the received thumbnail images 38 are displayed in a list. As shown in the example of the figure, all or part of the meta information and additional information for each thumbnail image 38 may be displayed as related / additional information 39. Also, check boxes 31 are arranged corresponding to the thumbnail images 38 so that the person in charge at the client terminal 3 can make a selection operation. Furthermore, a main image request button 33 is displayed so that the person in charge can operate it.

[0077] Such a UI screen is displayed so that the person in charge of the client terminal 3 can finally select the necessary images from the images roughly selected on the screen of FIG. 5A while checking the image contents using thumbnail images 38. FIG. 5B shows a state in which the person in charge has checked the check box 31 for the thumbnail image 38 in the center. In this state, when the original image request button 33 is operated, it is deemed that the person in charge has requested the original image corresponding to the thumbnail image. In the process PS3, the CPU 71 of the client terminal 3 controls the display of such a UI screen and detects inputs by the person in charge accordingly.

[0078] ·Communication TP7 The client terminal 3 transmits an original image request list to the server device 2. That is, in response to the person in charge checking the check box 31 and operating the original image request button 33 in the UI processing of the processing PS3, the client terminal 3 generates an original image request list including list information of images corresponding to the checked thumbnails and information requesting the original images, and transmits the list.

[0079] ·Communication TP8 The server device 2 transfers the original image request list received from the client terminal 3 to the imaging device 1.

[0080] ·Communication TP9 In response to receiving the original image request list, the imaging device 1 transmits to the server device 2 the original images of one or more images listed in the original image request list.

[0081] ·Communication TP10 The server device 2 transfers the original image received from the imaging device 1 to the client terminal 3. As a result, the original image 35 in FIG.

[0082] As described above, instead of transferring the original image 35 from the beginning, rough image selection based on meta information and additional information is performed in the first round trip, and then image selection using thumbnail images 38 is performed in the second round trip. The server device 2 relays the processes up to this point as an initial information transfer process. Finally, the server device 2 causes the original images listed in the original image request list by the selection in the initial information transfer process to be transferred from the imaging device 1 to the client terminal 3 by the original image transfer process.

[0083] In this way, instead of transferring a large number of original images captured by the imaging device 1 to the client terminal 3, the person in charge at the client terminal 3 can select and transfer the original images he or she needs, making the transfer process extremely efficient.

[0084] An example of processing by the imaging device 1 and the server device 2 for realizing each communication TP and process PS shown in FIG. 4 will now be described. 6 shows the process for communication TP1 during shooting by the imaging device 1. The process of the imaging device 1 mainly refers to the control process of the camera control unit 18.

[0085] The camera control unit 18 of the imaging device 1 controls the imaging process in step S101 in Fig. 6. That is, the camera control unit 18 controls the necessary units in Fig. 2 so that an image is captured / recorded in response to an operation by a cameraman. For example, the camera control unit 18 causes the imaging element unit 12 and the camera signal processing unit 13 to perform image signal processing of a still image, generate related meta information, and the like.

[0086] In step S102, the camera control unit 18 causes the camera signal processing unit 13 to execute encoding processing of the thumbnail image and the original image. In step S103, the camera control unit 18 causes the recording control unit 14 to record the thumbnail image and the original image (including meta information) on the recording medium.

[0087] In step S104, the camera control unit 18 causes the communication unit 16 to execute a process of transmitting meta information of the captured image. The process of step S104 is executed, for example, in response to a sending operation by the photographer. For example, the camera control unit 18 executes the process in response to the photographer selecting individual images or image folders to be sent (delivered to the client terminal 3) from among the images taken and recorded on a recording medium, and performing a sending operation. The camera control unit 18 causes the communication unit 16 to send meta information corresponding to one or more images selected by the photographer to be sent. Through the above processing, communication TP1 in FIG. 4 is performed.

[0088] The process of the server device 2 for the process PS1 and the communication TP2 in Fig. 4 is shown in Fig. 7. Note that the process of the server device 2 shows the control process of the control unit 2a. The control unit 2a confirms receipt of meta information from the imaging device 1 in step S201. Then, when meta information is received, the control unit 2a executes the loop process LP1.

[0089] In the loop process LP1, steps S202 and S203 are performed for each piece of meta information of an image transmitted from the imaging device 1. For example, when meta information for 10 images is received, these processes are performed for each image.

[0090] In step S202, the control unit 2a performs DB search processing based on the meta information to obtain additional information. For example, when information on the shooting position and shooting time for one image is sent as meta information as described above, the control unit 2a performs DB search based on the information on the shooting position and shooting time to obtain additional information such as the name of the sport and the name of the player. In step S203, the control unit 2a performs a process of transmitting meta-information and additional information about the image to the client terminal 3.

[0091] The control unit 2a performs the processes of steps S202 and S203 for each image in the loop process LP1, so that the meta information transmitted by the imaging device 1 for each image and the corresponding additional information are transmitted to the client terminal 3.

[0092] In response to this, the client terminal 3 performs the UI process as the process PS2 as described above, and transmits a thumbnail image request list as communication TP3. FIG. 8 shows the process of the server device 2 for communication TP4 in response to the thumbnail image request list.

[0093] In step S220, the control unit 2a of the server device 2 confirms receipt of the image request list. The image request list here refers to, for example, a thumbnail image request list or an original image request list. In the transmission patterns described below, the image request list may be information such as a request for a grouping image, a request for a partial clip of the original video, or a request for a partial clip of a proxy video.

[0094] In the case of the first transmission pattern in FIG. 4, when a thumbnail image request list or an original image request list is received, the control unit 2a executes the process of step S221. In step S221, the control unit 2a performs processing to transfer the received image request list to the image capture device 1. As a result, the thumbnail image request list is transferred in communication TP4.

[0095] FIG. 9 shows the process of the imaging device 1 for communication TP5. The camera control unit 18 confirms receipt of the image request list from the server device 2 in step S120. When the image request list is received, the camera control unit 18 executes a loop process LP20.

[0096] In the loop process LP20, the camera control unit 18 repeats the processes from step S121 to step S125 the number of times corresponding to the number of images related to the requests listed in the image request list. First, in step S121, the camera control unit 18 determines whether the image request list is a thumbnail image request list or an original image request list, and branches the process.

[0097] If the image request list is a thumbnail image request list, the camera control unit 18 instructs the recording control unit 14 to read out a thumbnail image for one image included in the request from the recording medium in step S122. Then, in step S123, the camera control unit 18 performs a process of transmitting the thumbnail image read from the recording medium to the server device 2 via the communication unit 16.

[0098] If the image request list is an original image request list, the camera control unit 18 instructs the recording control unit 14 to read out an original image for one image included in the request from the recording medium in step S124. Then, in step S125, the camera control unit 18 performs a process of transmitting the original image read from the recording medium to the server device 2 via the communication unit 16.

[0099] The above process is performed for each image related to the requests listed in the image request list, whereby the thumbnail image or the original image related to the image request list is transmitted. As for communication TP5, thumbnail images are transmitted in response to the thumbnail image request list.

[0100] FIG. 10 shows the process of the server device 2 for the communication TP6. In step S230, the control unit 2a of the server device 2 confirms reception of the image from the imaging device 1. The image here is, for example, a thumbnail image or an original image. When an image is received, the control unit 2a executes a loop process LP10.

[0101] In the loop process LP10, the control unit 2a repeats the process of step S231 the number of times corresponding to the number of received images. In step S231, the control unit 2a performs a process of transmitting the received image to the client terminal 3.

[0102] Therefore, in the communication TP6, when the server device 2 receives one or more thumbnail images from the imaging device 1, it transfers them to the client terminal 3.

[0103] In response to this, the client terminal 3 performs the UI process as the process PS3 as described above, and transmits the main image request list as communication TP7. The process of the server device 2 for the communication TP8 in response to the main image request list is as shown in FIG. Moreover, the process of the imaging device 1 that has received the main image request list through communication TP8 is as shown in Fig. 9. That is, steps S124 and S125 in Fig. 9 are performed for each image related to the requests listed in the main image request list, thereby transmitting the main images related to the main image request list as communication TP9. The process of the server device 2 for communication TP10 is as shown in Fig. 10. When the server device 2 receives one or more original images from the imaging device 1, it transfers them to the client terminal 3 in the process of step S231.

[0104] Note that the image request list transmitted to the imaging device 1 in communication TP5 or TP7 may include information on transmission priority. For example, when requesting multiple images, the person in charge can set the priority order. Then, the order is set in the image request list, and the imaging device 1 transmits the images in the order according to the priority order. This makes it possible to deliver images that are most needed as quickly as possible. This can also be applied to each of the following transmission patterns.

[0105] <4. Second transmission pattern> The second communication pattern will be explained. Note that in the following, for each communication pattern, differences from the communication patterns already described will be mainly explained, and detailed explanations of similar parts will be avoided.

[0106] FIG. 11 shows the second transmission pattern. 4, and meta-information is transmitted from the imaging device 1 to the server device 2. However, in the second transmission pattern, the server device 2 does not derive additional information as the above-mentioned process PS1 at this point, and transmits the received meta-information to the client terminal 3 as is in communication TP2.

[0107] The client terminal 3 performs UI processing as process PS2, but in this case, since no additional information has been received, the display contents are based only on the meta information as shown in FIG. 12A. For example, when the photographer, the location, and the time of shooting are transmitted as meta information, the information of the cameraman (photographer), the location, and the time of shooting corresponding to each image are displayed as an information content list 30. Also, check boxes 31 and thumbnail request buttons 32 corresponding to each piece of information are displayed so that the person in charge can operate them.

[0108] In communication TP3, a thumbnail image request list is transmitted from the client terminal 3 in response to an operation on the screen of FIG. 12A. In communication TP 4 , the server device 2 transfers the thumbnail image request list to the image capture device 1 . In communication TP5, the imaging device 1 transmits the thumbnail images requested in the thumbnail image request list.

[0109] In the second transmission pattern, the server device 2 receives the thumbnail image as related information of the captured main image through communication TP5, and performs processing PS4 to obtain additional information. In this case, image analysis and object recognition processing are performed on the thumbnail image, and additional information is generated based on the results. For example, the player's name can be obtained by performing face recognition, personal identification, and uniform number identification. It is also possible to determine the type of sport by object recognition of the subject, and generate the name of the sport as additional information. In these examples, some information is obtained by recognition processing using the thumbnail image, and the information is used as additional information.

[0110] In communication TP6, the server device 2 transmits to the client terminal 3 the thumbnail image and the additional information determined from the thumbnail image. The client terminal 3 performs UI processing using the received thumbnail image as the process PS3. In this case, since the additional information is also received, the client terminal 3 can perform display based on the additional information together with the thumbnail image. For example, the client terminal 3 displays the received thumbnail images 38 on the display unit 77 as a thumbnail list 34 as shown in Fig. 12B. Also, related / additional information 39 is displayed as all or part of the meta information and additional information for each thumbnail image 38. In this example, the name of the sport and the names of the athletes obtained as additional information are displayed together with the photographer information obtained as meta information. Also, check boxes 31 and original image request buttons 33 corresponding to the thumbnail images 38 are displayed so that the person in charge can operate them.

[0111] In communication TP7, the main image request list is transmitted from the client terminal 3 in response to the operation on the screen of FIG. 12B. In communication TP8, the server device 2 transfers the original image request list to the image capture device 1. In communication TP9, the imaging device 1 transmits the original images requested in the original image request list. In communication TP10, the server device 2 transfers the original image as shown in FIG.

[0112] As described above, the second transmission pattern in FIG. 11 differs from the first transmission pattern in FIG. 4 in that the process for the server device 2 to obtain additional information is based on a thumbnail image as the process PS4. The processing of the imaging device 1 and the server device 2 for the second transmission pattern is as follows.

[0113] The process of the imaging device 1 for communication TP1 is as shown in FIG.

[0114] The process of the server device 2 for the communication TP2 is as shown in FIG. The control unit 2a confirms receipt of meta information from the imaging device 1 in step S201. Then, when meta information is received, the control unit 2a executes the loop process LP2.

[0115] As the loop process LP2, the control unit 2a performs a process of transmitting the meta information transmitted from the imaging device 1 to the client terminal 3 in step S205. The control unit 2a performs the process of step S205 for each image in the loop process LP2, whereby the meta-information transmitted by the imaging device 1 for each image is transmitted to the client terminal 3.

[0116] In response to this, the client terminal 3 performs the UI process as the process PS2 as described above, and transmits a thumbnail image request list as communication TP3. The process of the server device 2 for communication TP4 in response to the thumbnail image request list is as shown in FIG. The process of the imaging device 1 for communication TP5 is as shown in FIG.

[0117] When the thumbnail image 38 is received in communication TP5, the process of the server device 2 for communication TP6 is as shown in FIG. In step S230, the control unit 2a of the server device 2 confirms reception of the image from the imaging device 1. The image here is, for example, a thumbnail image or an original image. In step S240, the control unit 2a branches the process depending on whether the received image is a thumbnail image or an original image. Then, the control section 2a executes a loop process LP11 when a thumbnail image is received, and executes a loop process LP11 when a main image is received.

[0118] In the loop process LP11, the control unit 2a repeats the processes of steps S241, S242, and S243 the same number of times as the number of received thumbnail images. In step S241, the control unit 2a performs image recognition processing, and in step S242, generates additional information based on the result of the image recognition processing. In this case, the additional information may be generated by performing a DB search based on the image recognition result. Furthermore, both the additional information based on the image recognition result and the additional information obtained as a DB search result based on meta information may be derived. In step S243, the control unit 2a performs a process of transmitting the received image to the client terminal 3.

[0119] Therefore, in the communication TP6, when the server device 2 receives one or more thumbnail images from the imaging device 1, the server device 2 transfers the thumbnail images and the additional information to the client terminal 3 together with the thumbnail images.

[0120] In response to this, the client terminal 3 performs the UI process as the process PS3 as described above, and transmits the main image request list as communication TP7. The process of the server device 2 for the communication TP8 in response to the main image request list is as shown in FIG. Moreover, the process of the imaging device 1 that has received the main image request list through communication TP8 is as shown in Fig. 9. That is, steps S124 and S125 in Fig. 9 are performed for each image related to the requests listed in the main image request list, thereby executing the transmission of the main images related to the image request list as communication TP9.

[0121] Moreover, the process of the server device 2 for the communication TP10 is as shown in Fig. 14. When the server device 2 receives one or a plurality of original images from the imaging device 1, the process proceeds from step S240 in Fig. 14 to loop process LP12. In the loop process LP12, the control unit 2a repeats the process of step S244 the number of times corresponding to the number of received main images. In step S244, the control unit 2a performs a process of transmitting the received original image to the client terminal 3. Therefore, in the communication TP10, when the server device 2 receives one or more original images from the imaging device 1, it transfers them to the client terminal 3.

[0122] <5. Third and fourth transmission patterns> As the third and fourth transmission patterns, examples in which acquisition / transfer of additional information is not performed will be described. FIG. 15 shows the third transmission pattern.

[0123] Meta information is transmitted from the imaging device 1 to the server device 2 in communication TP1. In communication TP2, the server device 2 transmits the received meta-information to the client terminal 3 as is. The client terminal 3 performs UI processing as process PS2, but in this case, since no additional information has been received, the display content is based only on the meta information as shown in Fig. 16A. For example, the content will be the same as that shown in Fig. 12A.

[0124] In communication TP3, a thumbnail image request list is transmitted from the client terminal 3 in response to an operation on the screen of FIG. 16A. In communication TP 4 , the server device 2 transfers the thumbnail image request list to the image capture device 1 . In communication TP5, the imaging device 1 transmits the thumbnail images requested in the thumbnail image request list. In communication TP6, the server device 2 transmits the thumbnail image to the client terminal 3.

[0125] The client terminal 3 performs UI processing using the received thumbnail image as process PS3. For example, the client terminal 3 displays the received thumbnail image 38 on the display unit 77 as a thumbnail list 34 as shown in Fig. 16B. In this case, since no additional information is transmitted, only information based on the meta information, such as the shooting location and shooting time, is displayed as related / additional information 39. Also, check boxes 31 and original image request buttons 33 corresponding to the thumbnail images 38 are displayed so that the person in charge can operate them.

[0126] In communication TP7, the main image request list is transmitted from the client terminal 3 in response to the operation on the screen of FIG. 16B. In communication TP8, the server device 2 transfers the original image request list to the image capture device 1. In communication TP9, the imaging device 1 transmits the original images requested in the original image request list. In communication TP10, the server device 2 transfers the original image as shown in FIG.

[0127] As described above, the third transmission pattern in FIG. 15 differs from the first and second transmission patterns described above in that the server device 2 does not perform a process for obtaining additional information.

[0128] The processing of the imaging device 1 and the server device 2 for the third transmission pattern is as follows. The process of the imaging device 1 when photographing is as shown in FIG. 6, and the process when an image request is received is as shown in FIG. The process of the server device 2 when receiving meta information is as shown in FIG. 13, the process when receiving an image request is as shown in FIG. 8, and the process when receiving an image is as shown in FIG.

[0129] Next, the fourth transmission pattern will be described. The overall procedure is similar to that of the third transmission pattern in FIG. 15, but this is an example in which the imaging device 1 has a personal face registration function.

[0130] If the imaging device 1 has registered, for example, the faces, teams, and names of each of the athletes in a competition, the athletes who are the subjects of the images can be identified by image recognition, and this information can be included in the meta information. Therefore, the imaging device 1 performs the process shown in FIG. 17, for example. As the processing by the camera control unit 18, the processing of steps S101, S102, S103, and S104 in FIG. 17 is similar to that in FIG. The camera control unit 18 performs individual face recognition processing as step S180 in Fig. 17. For example, the individual face recognition processing is performed on each image recorded in step S103, the player who is the subject is identified, and the information is added to the meta information. As a result, when meta information is transmitted in step S104, the meta information will also include player information.

[0131] Therefore, when meta information is transmitted to the client terminal 3 through the communication TP1 and the communication TP2 in Fig. 15, the client terminal 3 can display an information content list 30 as shown in Fig. 16D based on the meta information. In other words, the information of the players can be displayed. This can be extremely useful, for example, if a staff member wants to select an image of a specific player.

[0132] <6. Fifth transmission pattern> The fifth communication pattern is explained with reference to FIG. In communication TP1, meta information including voice memo data is transmitted from the imaging device 1 to the server device 2. The voice memo data is data that is stored as a piece of meta information by uttering words as a memo when an image is captured (for example, immediately after a release operation) or after an image is recorded on a recording medium, and the voice is associated with the image as a memo. This fifth communication pattern is applicable when the imaging device 1 has a voice memo function.

[0133] When the server device 2 receives the meta information including the voice memo data, it transmits it to the client terminal 3 as communication TP2.

[0134] The client terminal 3 performs UI processing as process PS2, but in this case, the display content is an information content list 30 based only on meta information, as shown in Fig. 19A. However, since meta information including voice memo data has been received, an audio playback icon 40 is displayed for information content including voice memo data. In the example shown in the figure, it is assumed that voice memo data exists for all images, and an audio playback icon 40 is displayed corresponding to each information content. By operating the audio playback icon 40, the audio memo is played back by, for example, the audio output unit 78 in the client terminal 3. If the photographer appropriately records the audio memo, the person in charge of the client terminal 3 can select images by referring to the audio memo.

[0135] The subsequent communications TP3 to TP10 and process PS3 are similar to those in the third communication pattern, and therefore redundant explanations will be avoided.

[0136] The process of photographing with an imaging device capable of recording voice memo data is as shown in FIG. In FIG. 20, steps S101, S102, S103, and S104 are the same as those in FIG.

[0137] The cameraman can optionally add voice memos to the captured images. If the cameraman does not perform an operation to record a voice memo for the captured image, the process proceeds from step S110 to step S104, similar to that in FIG. 20 proceeds from step S110 to step S111, and camera control unit 18 performs a recording process. That is, in response to a predetermined operation, the process records the voice input from voice input unit 25 for a period of, for example, 10 to 20 seconds. In step S112, the camera control unit 18 causes the camera signal processing unit 13 to execute encoding processing of the input voice memo data. In step S113, the camera control unit 18 causes the recording control unit 14 to execute a process of recording the encoded voice memo data onto a recording medium. Then, in step S114, the camera control unit 18 causes the communication unit 16 to execute a process of transmitting the image meta information to the server device 2 with the voice memo data added thereto.

[0138] When meta information including voice memo data is received, the process of the server device 2 for communication TP2 is as shown in FIG. The control unit 2a confirms receipt of meta information from the imaging device 1 in step S201. Then, when meta information is received, the control unit 2a executes the loop process LP3.

[0139] In the loop process LP3, in step S206, the control unit 2a determines whether or not the meta information includes voice memo data, and branches the process. If the meta information does not include voice memo data, the control unit 2a performs a process of transmitting the meta information transmitted from the imaging device 1 to the client terminal 3 in step S205. If the meta information includes voice memo data, the control unit 2a performs processing to transmit the meta information including the voice memo data transmitted from the imaging device 1 to the client terminal 3 in step S207.

[0140] The control unit 2a performs the processing of steps S206 and S205, or steps S206 and S207 for each image in the loop processing LP3, so that the meta information transmitted by the imaging device 1 for each image is transmitted to the client terminal 3, and if voice memo data is added, it is also transmitted.

[0141] The other processing for this fifth transmission pattern is as follows. The process of the imaging device 1 when it receives an image request is as shown in FIG. The process of the server device 2 when receiving an image request is as shown in FIG. 8, and the process of the server device 2 when receiving an image is as shown in FIG.

[0142] <7. 6th transmission pattern> The sixth communication pattern is explained with reference to FIG. In communication TP1, meta information including voice memo data is transmitted from the imaging device 1 to the server device 2. When the server device 2 receives the meta information including the voice memo data, the server device 2 performs a transcription process of the voice memo as a process PS5. That is, the server device 2 generates information by transcribing the voice memo data into text data. Then, the server device 2 transmits the meta information and the text information based on the voice memo data to the client terminal 3 as a communication TP2.

[0143] The client terminal 3 performs UI processing as process PS2. In this case, the display content is an information content list 30, as shown in FIG. 23A, which can display voice memo text 41, which is a text version of the voice memo data, together with the contents of the meta information. In addition, the text information may be considered as one of the additional information referred to in this disclosure in the sense that it is generated by the server device 2, but since the content is the same as the voice memo data, it may also be considered as the related information referred to in this disclosure.

[0144] By displaying the voice memo text 41, the person in charge at the client terminal 3 can select images by referring to the contents of the voice memo. Since there is no need to play back the voice memo at this time, the selection operation can be performed more efficiently.

[0145] The subsequent communications TP3 to TP10 and process PS3 are similar to those in the third communication pattern.

[0146] The process during shooting with an imaging device capable of recording voice memo data is as shown in FIG. When meta information including voice memo data is received, the process of the server device 2 for communication TP2 is as shown in FIG. The control unit 2a confirms receipt of meta information from the imaging device 1 in step S201. Then, when meta information is received, the control unit 2a executes a loop process LP4.

[0147] In the loop process LP4, in step S206, the control unit 2a determines whether or not the meta information includes voice memo data, and branches the process. If the meta information does not include voice memo data, the control unit 2a performs a process of transmitting the meta information transmitted from the imaging device 1 to the client terminal 3 in step S205. If the meta information includes voice memo data, the control unit 2a performs a process of converting the voice memo data into text in step S208. Then, the control unit 2a performs a process of transmitting the converted voice memo data and other meta information transmitted from the imaging device 1 to the client terminal 3 in step S209.

[0148] At this time, the amount of data to be transmitted can be reduced by not transmitting the voice memo data itself. However, the original voice memo data may be transmitted to the client terminal 3 together with the converted voice memo data, so that the voice memo data can be reproduced on the client terminal 3 side.

[0149] The control unit 2a performs the processing of steps S206 and S205, or steps S206, S208, and S209 for each image in the loop processing LP4, so that the meta information transmitted by the imaging device 1 for each image is transmitted to the client terminal 3, and if voice memo data is added, it is converted to text and transmitted.

[0150] The other processing for this sixth transmission pattern is as follows. The process of the imaging device 1 when it receives an image request is as shown in FIG. The process of the server device 2 when receiving an image request is as shown in FIG. 8, and the process of the server device 2 when receiving an image is as shown in FIG.

[0151] <8. 7th transmission pattern> In FIG. 25, as a seventh transmission pattern, an example will be described in which continuous shooting is performed by an imaging device and a series of images are managed as grouped images.

[0152] ·Communication TP21 The imaging device 1 transmits meta information including a thumbnail image of the top image of the continuous-shot group of images to the server device 2 as related information about the captured image. In this case, the imaging device 1 transmits a thumbnail image of the first image for each of one or more continuous-shot grouping images; however, it is not limited to the first image; it is sufficient that an image included in the continuous-shot grouping images is selected and a thumbnail image of that image is transmitted.

[0153] ·Communication TP22 The server device 2 transmits meta information including thumbnail images for each of the continuous-shot grouped images to the client terminal 3.

[0154] Processing PS6 The client terminal 3 performs UI processing using thumbnail images of each received continuous-shooting grouped image. For example, the thumbnail list 34 as shown in Fig. 26A is displayed on the client terminal 3. In this case, for the continuous-shooting grouped images, thumbnail images of the continuous-shooting grouped images (continuous-shooting thumbnail images 42) are displayed. Furthermore, if the transmitted content includes a thumbnail image of an image obtained by normal single-shot imaging rather than continuous shooting, that thumbnail image 38 is displayed. Corresponding to the continuous-shot thumbnail images 42 and the thumbnail images 38, information on the cameraman (photographer), the shooting time, the shooting location, and the like are displayed as related / additional information 39 based on meta information. Also, check boxes 31 are arranged corresponding to the continuous thumbnail images 42 and the thumbnail images 38, so that the person in charge at the client terminal 3 can perform a selection operation. Furthermore, a main image request button 33 and a deployment button 36 are displayed so that the person in charge can operate them.

[0155] Such a UI screen is a screen for the person in charge of the client terminal 3 to select images for which the original image is to be requested from among the thumbnail images, but for continuous-shot grouped images, it is also possible to select individual images. For example, by selecting the continuous-shot thumbnail image 42 with the check box 31 and operating the expand button 36, the thumbnail images of each image constituting the continuous shot can be viewed as shown in FIG. 26B.

[0156] ·Communication TP23 The client terminal 3 transmits a request for the entire grouped images to the server device 2. That is, in response to the person in charge checking the check box 31 for a certain continuous-shot thumbnail image 42 in the UI processing of the process PS2 and operating the expand button 36, the client terminal 3 generates information for a request for the entire grouped images, including list information for all checked continuous-shot grouped images and information requesting their thumbnails, and transmits this information.

[0157] ·Communication TP24 The server device 2 transfers the request for the entire grouping of images received from the client terminal 3 to the imaging device 1.

[0158] Communication TP25 In response to receiving the request for the entire group of images, the imaging device 1 transmits to the server device 2 thumbnail images of each of the images included in the requested continuous-shot group of images.

[0159] ·Communication TP26 The server device 2 transfers the thumbnail image received from the imaging device 1 to the client terminal 3.

[0160] Processing PS7 The client terminal 3 performs UI processing using all the thumbnail images of the received grouped images. For example, the client terminal 3 displays each thumbnail image 43 of the continuous-shot grouped images on the display unit 77 as an expanded list 37 as shown in FIG. 26B. It is also advisable to display related / additional information 39 for the entire group of continuous shot images. Also, check boxes 31 are arranged corresponding to the thumbnail images 43, so that the person in charge at the client terminal 3 can make a selection operation. Furthermore, a main image request button 33 is displayed so that the person in charge can operate it.

[0161] Such a UI screen also enables the person in charge of the client terminal 3 to select some of the continuous-shot images.

[0162] ·Communication TP27 The client terminal 3 transmits an original image request list to the server device 2. That is, in response to the person in charge checking the check box 31 and operating the original image request button 33 in the UI processing of process PS6 or process PS7, the client terminal 3 generates an original image request list including list information of images corresponding to the checked thumbnails and information requesting those original images, and transmits the list.

[0163] ·Communication TP28 The server device 2 transfers the original image request list received from the client terminal 3 to the imaging device 1.

[0164] ·Communication TP29 In response to receiving the original image request list, the imaging device 1 transmits to the server device 2 the original images of one or more images listed in the original image request list.

[0165] ·Communication TP30 The server device 2 transfers the original image received from the imaging device 1 to the client terminal 3. As a result, the original image 35 in FIG. 26C is delivered to the client terminal 3.

[0166] As described above, when a group of continuous-shot images is assumed, a thumbnail image representative of the group of continuous-shot images is transmitted, thereby allowing the selection of the group of continuous-shot images, and further allowing the selection of individual images of the group of continuous-shot images. In this way, instead of transferring all of a large number of original images continuously shot by the imaging device 1 to the client terminal 3, the person in charge of the client terminal 3 can selectively transfer the original images for the continuous shot grouping images and individual images that he or she requires, making the series of transfers extremely efficient.

[0167] Note that FIG. 25 shows communications TP21 to TP30 assuming that continuous shooting is being performed by the imaging device 1. However, when continuous shooting is not being performed by the imaging device 1, communications TP1 to TP10 will be performed, for example, by any of the first to sixth transmission patterns described above.

[0168] Even when continuous shooting is performed, for example, communication TP1, processing PS1, communication TP2, processing PS2, communication TP3, and communication TP4 shown in FIG. 4 may be performed prior to communication TP21 in FIG. 25, and the processing after communication TP21 in FIG. 25 may be performed for images requested by the thumbnail image request list in communication TP4.

[0169] As a process for the seventh transmission pattern by the imaging device 1 during shooting, an example shown in FIG. 27 is conceivable. In step S140, the camera control unit 18 branches the process depending on whether or not continuous shooting is performed (whether or not the continuous shooting mode is set). If continuous shooting is not performed, the camera control unit 18 performs the processes from step S101 to step S104, which are the same as those in FIG.

[0170] During continuous shooting, the camera control unit 18 repeats the processes from step S141 to step S144 as loop processing LP30 the number of times that images are captured by continuous shooting. In step S141, the camera control unit 18 controls the imaging process for one of the continuous shots. For example, the camera control unit 18 causes the image sensor unit 12 and the camera signal processing unit 13 to execute image signal processing for a still image.

[0171] In step S142, the camera control unit 18 causes the camera signal processing unit 13 to execute the encoding process of the thumbnail image and the original image. In step S143, the camera control unit 18 causes the recording control unit 14 to record the thumbnail image and the original image on the recording medium. In step S144, the camera control unit 18 updates the group information management file. This is processing for updating meta information associated with a continuous-shot grouping of images for a file that manages the continuous-shot grouping of images.

[0172] After performing the above process for the number of continuous shots, the camera control unit 18 proceeds to step S145 and causes the communication unit 16 to execute a process of transmitting meta information including the first thumbnail image of the continuous-shot grouping of images. Through the above processing, communication TP1 in FIG. 4 is performed.

[0173] The process performed by the imaging device 1 when it receives an image request is as shown in FIG. This processing is performed when a request for thumbnail images of all grouped images is received in communication TP25, when a request list for original images is received in communication TP28, and further when a request list for thumbnail images is received as in communication TP5 in FIG.

[0174] The camera control unit 18 confirms receipt of the image request list from the server device 2 in step S120. When the image request list is received, the camera control unit 18 executes a loop process LP21.

[0175] In the loop process LP21, the camera control unit 18 repeats the following process the number of times corresponding to the number of images related to the requests listed in the image request list. First, in step S131, the camera control unit 18 determines whether or not the image request list is a thumbnail image request list.

[0176] If the image request list is a thumbnail image request list, the camera control unit 18 instructs the recording control unit 14 to read out a thumbnail image for one image included in the request from the recording medium in step S122. Then, in step S123, the camera control unit 18 performs a process of transmitting the thumbnail image read from the recording medium to the server device 2 via the communication unit 16. This is repeated as many times as the number of images requested.

[0177] If the received request information is not a thumbnail image request list, the camera control unit 18 determines in step S132 whether the image request list is an original image request list. If it is an original image request list, the camera control unit 18 causes the recording control unit 14 to read out an original image for one image included in the request from the recording medium in step S124. Then, in step S125, the camera control unit 18 performs a process of transmitting the original image read from the recording medium to the server device 2 via the communication unit 16. This is repeated as many times as the number of images requested.

[0178] If the received request information is neither a thumbnail image request list nor an original image request list, the camera control unit 18 determines in step S133 whether or not it is a request for all grouped images In that case, the camera control unit 18 performs a loop process LP22. In the loop process LP22, steps S126 and S127 are repeated according to the number of images constituting the corresponding continuous-shot image group. To be precise, since the thumbnail image of the first image has already been transmitted, the process is repeated for the second and subsequent images.

[0179] In step S126, the recording control unit 14 executes reading out of the recording medium thumbnail images of the untransmitted images of the continuous-shot grouped images. Then, in step S127, the camera control unit 18 performs a process of transmitting the thumbnail image read from the recording medium to the server device 2 via the communication unit 16.

[0180] By carrying out the above-mentioned processing, the thumbnail image or the original image in the image request list is transmitted. Image transmission is performed as communication TP25, communication TP29 (or communication TP9), and communication TP5.

[0181] The process of the server device 2 when receiving meta information is as shown in FIG. 13, the process when receiving an image request is as shown in FIG. 8, and the process when receiving an image is as shown in FIG.

[0182] <9. Switching Processing Example I> Up to this point, the first transmission pattern to the seventh transmission pattern have been explained assuming still images as the images. However, by having the server device 2 appropriately switch between these transmission patterns, appropriate communication can be realized depending on the situation and the subject of photography.

[0183] The switching described below includes, for example, switching between the first transmission pattern and the fourth transmission pattern, or between the fifth transmission pattern and the sixth transmission pattern depending on the situation, and switching the transmission pattern itself, such as automatically switching from the first transmission pattern to the seventh transmission pattern when continuous shooting is performed with an imaging device, but is not limited to this. For example, in the first transmission pattern, meta information and additional information are sent in the first round trip to allow the client terminal 3 to make a selection, and a thumbnail image is sent in the second round trip to allow the client terminal 3 to make a selection; thus, the initial information transfer process is performed two round trips; however, there are also examples in which the actual image transfer process is performed after only one round trip, or in which the initial information transfer process is not performed and the actual image transfer process is performed from the beginning; in these ways, switching to reduce or add steps in the transmission pattern is also included. In the following description, the focus is mainly on switching between transmission patterns, but it should be understood that the description can also be applied to switching between procedures within a transmission pattern.

[0184] FIG. 29 shows a switching process example I. Processing PS10 The server device 2 judges the situation and determines the transmission pattern to be executed depending on the situation. As described above, it is also possible to determine to change the procedure in a certain transmission pattern (for example, to omit the initial information transfer process or to limit the initial information transfer process to one round trip).

[0185] ·Communication TP40 The server device 2 notifies the imaging device 1 of the determined transmission pattern. Processing PS11 The imaging device 1 performs setting processing for image delivery in accordance with the notified transmission pattern. For example, the camera control unit 18 performs the following settings. For the first to fourth transmission patterns, the setting is made to transmit only meta information when transferring images. For the fifth and sixth transmission patterns, the setting is made to transmit meta information including voice memo data when transferring images. For the seventh transmission pattern, the setting is made to transmit a thumbnail image of the first image of a continuous-shot grouping of images. Alternatively, there may be a case where a setting is made such that the initial information transfer process is omitted in a certain transmission pattern.

[0186] ·Communication TP41 The server device 2 notifies the client terminal 3 of the determined transmission pattern. Processing PS12 The client terminal 3 performs UI processing in accordance with the notified transmission pattern.

[0187] The order of the above-mentioned communication TP40 and communication TP41 may be reversed.Also, it is conceivable that one or both of the communication TP40 and communication TP41 and the corresponding processes PS11 and PS12 are not performed.

[0188] An example of the process performed by the server device 2 to perform such a switching process is shown in FIG. In step S300, the control unit 2a judges the situation, such as the communication situation including the current transfer rate and packet error rate, the number of connected imaging devices 1, and the processing load of the server device 2 itself. In step S301, the control unit 2a branches the process depending on whether or not switching should be performed as a result of the situation determination. If switching is not to be performed, the control unit 2a ends the process of Fig. 30, but if switching is to be performed, the process proceeds to step S302.

[0189] In step S302, the control unit 2a determines a transmission pattern according to the determined situation. For example, the first transmission pattern may be used when the communication situation with the imaging device 1 is good, and the third transmission pattern may be used when the communication situation is poor, or the first or second transmission pattern may be used when the processing load of the server device 2 itself is light, and the third transmission pattern may be used when the processing load is heavy.

[0190] In step S303, the control unit 2a performs a process of notifying the imaging device 1 of the transmission pattern, and in step S304 performs a process of notifying the client terminal 3 of the transmission pattern.

[0191] By switching the transmission pattern through such processing, an appropriate transmission pattern is selected depending on the communication situation, the processing load situation, and the like. Specific examples of switching in response to various situational determinations will be given below.

[0192] First, there is a situational assessment regarding timing. For the first and second transmission patterns, the server device 2 needs to generate additional information. Also, for the sixth transmission pattern, text conversion processing is required. Therefore, if information for generating additional information has not been created, or if DB access is not possible, or if text conversion processing is not possible, the third, fourth, and fifth transmission patterns are set, and if these additional information generation, etc. are possible, the first, second, and sixth transmission patterns are set.

[0193] The state of the processing load of the server device 2 may also be determined. When an image list is requested from a large number of client terminals 3, it is desirable to reduce the amount of data to be transmitted in order to reduce the network load on the server device 2. For example, it is desirable to transmit text data rather than voice data. Therefore, when the network load on the server device 2 is high, it is conceivable to switch to the third transmission pattern, and when the network load on the server device 2 is low, it is conceivable to switch to the fifth transmission pattern. Also, when receiving images from multiple cameras, it is desirable to reduce the amount of data to be transmitted in order to reduce the network load on the server device 2. For example, in the case of continuous shot images, it is desirable to reduce the amount of data by sending the first image. Therefore, when the network load on the server device 2 side is high, it is conceivable to switch to the seventh transmission pattern, and when the network load on the server device 2 side is low, it is conceivable to switch to the third transmission pattern. Furthermore, when a large amount of meta information is received from a large number of imaging devices 1, the data creation processing load increases on the server device 2. Therefore, when the interface processing on the server device 2 side or the processing load of the CPU 71 is high, the processing may be switched to the third or fourth transmission pattern, and when the interface processing on the server device 2 side or the processing load of the CPU 71 is low, the processing may be switched to the first, second or fifth transmission pattern.

[0194] It is also possible to judge the situation from the perspective of use cases. There are differences in the imaging location environment, such as outdoor sports where the network environment may be poor, and indoor / stadium sports where the network environment is good. For example, a possible example would be to switch to the third transmission pattern when the network environment is poor, and to the fifth transmission pattern when the network environment is good. Furthermore, when the network environment is poor, the transmission will be carried out in the order of meta information transmission, thumbnail image transmission, and actual image transmission as described in the first to sixth transmission patterns. However, when the network environment is good, it is possible to reduce the number of transmission steps by transmitting a thumbnail image followed by the actual image transmission (i.e., making the initial information transfer process a single round trip), or by transmitting the actual image from the beginning (without performing the initial information transfer process). There are sports such as track and field / swimming where a DB of time, place, and athletes can be created from the competition program, and there are sports such as marathons and triathlons where this is not the case. If the subject event is a sport where a DB can be created, the first transmission pattern can be used, and if the subject event is not a sport where a DB can be created, the third, fourth, fifth, or sixth transmission pattern can be used. The process of switching the transmission pattern depending on the type of sport being photographed will be described later as switching process examples II and III.

[0195] It is also possible to judge the situation from the perspective of whether it is a 5G or 4G environment. We want to reduce the amount of data sent depending on the network environment. For example, in a 4G environment, we could switch to the third and fourth transmission patterns, and in a 5G environment, we could switch to the first, second, fifth, and sixth transmission patterns.

[0196] 29 and 30, the server device 2 notifies the client terminal 3 of the transmission pattern, and the client terminal 3 can display pattern information 44, for example, as shown in Fig. 31A and Fig. 31B, to show the person in charge what transmission pattern the image transfer will be performed in. Fig. 31A shows a display state before the start of transfer in process PS12, and Fig. 31B shows an example of the display state in process PS2 in Fig. 16, for example. This pattern information 44 may indicate the contents and order of information transmitted to the client terminal 3, as shown in the examples in FIG. 31C.

[0197] In the examples of Figs. 29 and 30, the server device 2 automatically performs the switching by the situation determination process, but the server device 2 may request the person in charge of the client terminal 3 to decide whether or not to switch. For example, as shown in Fig. 32, a confirmation dialog 45 is displayed on the client terminal 3, and the transmission pattern is switched when the person in charge agrees to the switch. In the figure, an example is shown in which the person in charge is asked for their approval when switching is performed in accordance with the network environment. For this purpose, the server device 2 performs, for example, the process shown in FIG. 33 in step S301 of FIG.

[0198] In step S310, the control unit 2a performs a network status determination process. For example, when it is determined that the transfer rate has become lower and the information transfer has decreased, the control unit 2a proceeds from step S311 to step S313 and transmits a switching recommendation notification to the client terminal 3. In response to this, the client terminal 3 is caused to display a confirmation dialog 45 as shown in Fig. 32. Then, in step S314, the control unit 2a waits for a response notification from the client terminal 3 side. When the person in charge performs an operation to approve the switch and a notification to that effect is received from the client terminal 3, the process proceeds from step S314 to steps S301 and S302 in FIG. 30, and the transmission pattern is switched. If the person in charge performs an operation to not approve the switch and a notification to that effect is received from the client terminal 3, the process proceeds from step S314 to step S301 in FIG. 30, where it is decided that the transmission pattern will not be switched and the process in FIG. 30 is terminated.

[0199] When it is determined in the network status determination process in step S310 that, for example, the transfer rate has increased and the state has been restored to one in which information transfer can be performed at high speed, the control unit 2a proceeds from step S312 to step S315 and transmits a switching recommendation notification to the client terminal 3. In response to this, the client terminal 3 displays a confirmation dialog 45 that prompts the user to switch to a transmission pattern under good communication conditions. Then, the control unit 2a waits for a response notification from the client terminal 3 in step S316. When the person in charge performs an operation to approve the switch and a notification to that effect is received from the client terminal 3, the process proceeds from step S316 to steps S301 and S302 in FIG. 30, and the transmission pattern is switched. If the person in charge performs an operation not to approve the switch and a notification to that effect is received from the client terminal 3, the process proceeds from step S316 to step S301 in FIG. 30, where it is decided that the transmission pattern will not be switched and the process in FIG. 30 is terminated.

[0200] In this way, the transmission pattern switching process may be performed based on both the result of the situation determination and the intention of the person in charge of the client terminal 3. This makes it possible to prevent inappropriate switching from being performed depending on circumstances on the client terminal 3 side.

[0201] <10. Switching Processing Example II> The switching process example II will be described with reference to FIG. First, in the process PS20, sports information (information on the type of sport) of the subject to be photographed is inputted, for example, by an operation of a cameraman, and the imaging device 1 is made to include the type of sport as meta information.

[0202] As the communication TP50, the imaging device 1 transmits meta information including sports information of the subject at some timing to the server device 2. The timing of transmission may be immediately after the process PS20 or when transmitting meta information for the first time in the first transmission pattern or the like.

[0203] In the process PS21, the server device 2 determines a transmission pattern based on the sports information included in the received meta information. Thereafter, notification of the transmission pattern set as communication TP40 and communication TP41 is performed in the same manner as in the above-described switching process example I. Process PS11 in the imaging device 1 and process PS12 in the client terminal 3 are also performed in the same manner.

[0204] The process of the server device 2 for such a switching process example II is as shown in FIG. In step S320, the control unit 2a checks whether or not a notification of a subject to be photographed from the imaging device 1, that is, sports information, has been received. When sports information is received, the control unit 2a proceeds to step S321 and judges whether the sport of the photographed subject is a sport for which additional information can be acquired. For example, in track and field events with a set schedule (timetable), it is possible to obtain information on the athletes who are the subjects of the images as additional information by DB search, but in marathons, soccer, etc., it is difficult to identify the athletes in the images from the meta information of the images (for example, time and place) by DB search.

[0205] Therefore, if the sport is one for which additional information can be acquired by a DB search, the control unit 2a proceeds to step S322 and selects the first transmission pattern. On the other hand, if it is difficult to acquire additional information about the sport through a DB search, the control unit 2a proceeds to step S323 and selects the third transmission pattern. The control unit 2a then proceeds from step S322 or step S323 to step S324, where it notifies the imaging device 1 of the transmission pattern, and also notifies the client terminal 3 of the transmission pattern in step S325.

[0206] <11. Switching Processing Example III> Example III of the switching process will be described with reference to FIG. As communication TP50, project information is notified from the client terminal 3. Project information is, for example, information associating a camera ID for identifying the imaging device 1 with what sport the imaging device 1 will shoot and when. Alternatively, it may be information associating cameraman information with what sport the cameraman will shoot and when. The server device 2 performs a process PS30 to register the received project information.

[0207] At some timing, the imaging device 1 transmits meta information including time information, a camera ID, or cameraman information to the server device 2 as communication TP51. The transmission timing may be, for example, when the meta information is first transmitted in the first transmission pattern or the like.

[0208] In process PS31, the server device 2 refers to the registered project information from the time information, camera ID, or cameraman information contained in the received meta information, and estimates the sport type that is the subject of shooting by the imaging device 1. Then, the transmission pattern is determined depending on whether the sport type is a sport to which additional information can be added. Thereafter, notification of the set transmission pattern for communication TP40 and communication TP41 is performed in the same manner as in switching process example I. Process PS11 in the imaging device 1 and process PS12 in the client terminal 3 are also performed in the same manner.

[0209] The process of the server device 2 for such a switching process example III is as shown in FIG. 37 and FIG. 37, the control unit 2a checks whether the project information has been received from the client terminal 3. If the project information has been received, the process proceeds to step S331, where the control unit 2a performs a registration process for the project information. That is, the project information is then stored in a predetermined storage unit (for example, the storage unit 79) so that it can be referenced.

[0210] 38, the control unit 2a checks whether or not meta information has been received from the imaging device 1. In this case, reception of meta information including time information, camera ID, or cameraman information is checked. If the corresponding meta information has been received, the control unit 2a proceeds to step S333 and estimates the sport type of the subject of the image capture, that is, by collating the meta information with the project information.

[0211] In step S321, the control unit 2a determines whether or not the sport of the estimated subject of shooting is a sport for which additional information can be acquired. If the sport is one for which additional information can be acquired by DB search, the control unit 2a proceeds to step S322 and selects the first transmission pattern. On the other hand, if the sport is one for which additional information is difficult to acquire by DB search, the control unit 2a proceeds to step S323 and selects the third transmission pattern. The control unit 2a then proceeds from step S322 or step S323 to step S324 and notifies the imaging device 1 of the transmission pattern, and also notifies the client terminal 3 of the transmission pattern in step S325.

[0212] <12.8th transmission pattern> So far, we have mainly explained the first to seventh transmission patterns and their switching, but when the image to be transferred is a video, there are transmission patterns that are more suitable for that. In addition, the above switching process also assumes switching to a transmission pattern that is more suitable for those videos. In the following, transmission patterns suitable for moving images will be described as the eighth, ninth, and tenth transmission patterns.

[0213] FIG. 39 shows the eighth transmission pattern. ·Communication TP71 The imaging device 1 transmits the proxy video to the server device 2. The proxy video is a video with a reduced amount of data, for example with a lower resolution, compared to the main video to be delivered.

[0214] ·Communication TP72 The server device 2 transmits the received proxy video to the client terminal 3 .

[0215] Processing PS40 The client terminal 3 performs UI processing using the received proxy video. For example, the client terminal 3 executes a display as shown in Fig. 40 on the display unit 77. That is, a proxy video playback image 46 and a video timeline 47 are displayed. On the timeline 47, representative images and audio waveforms at each time point extracted from the proxy video are displayed. Also, on the timeline 47, the person in charge can specify a required section DK. Furthermore, a main video request button 48 is displayed so that the person in charge can operate it.

[0216] Such a UI screen is displayed so that the person in charge of the client terminal 3 can first check the playback image of the proxy video and specify a section required for the main video. The person in charge specifies the section DK and then operates the main video request button 48. In the process PS40, the CPU 71 of the client terminal 3 controls the display of such a UI screen and detects the corresponding input by the person in charge.

[0217] ·Communication TP73 The client terminal 3 transmits a request to clip a portion of the main moving image to the server device 2. That is, the client terminal 3 generates information requesting the moving image of the section DK designated by the person in charge in the UI processing of the process PS40, and transmits the information.

[0218] ·Communication TP74 The server device 2 transfers the request for clipping a portion of the original moving image received from the client terminal 3 to the imaging device 1.

[0219] Processing PS41 In response to receiving the request to clip part of the original moving image, the imaging device 1 performs processing to create a clip by cutting out the specified section from the original moving image. Communication TP75 The imaging device 1 transmits to the server device 2 a partial clip of the original moving image generated in the process PS41.

[0220] ·Communication TP76 The server device 2 transfers a partial clip of the original moving image received from the imaging device 1 to the client terminal 3 . As a result, the original video of the scene required by the person in charge is delivered to the client terminal 3.

[0221] As described above, instead of transferring the entire main video from the beginning, the proxy video is transferred as an initial information transfer process, and the required section is specified on the client terminal 3 side. In response to this, a partial clip of the main video for the required section is transferred as a main video transfer process.

[0222] FIG. 41 shows an example of processing performed by the imaging device 1 for this eighth transmission pattern when recording a moving image. The camera control unit 18 of the imaging device 1 performs a moving image recording process as a loop process LP50 in Fig. 41. That is, in step S401, the control of steps S402 and S403 is performed for each frame until a request to stop recording is detected. In step S402, the camera control unit 18 causes the camera signal processing unit 13 to execute encoding processing of the frame image data and audio data. In step S403, the camera control unit 18 causes the recording control unit 14 to record the frame image data, audio data, and meta information on the recording medium.

[0223] For example, when a request to stop recording is made by an operation by the cameraman, the camera control unit 18 proceeds to step S405 and controls the reading of a proxy video for the recorded video. For example, the recording control unit 14 executes video reading, and the camera signal processing unit 13 performs low-resolution conversion to generate the proxy video. Then, in step S406, the camera control unit 18 controls the communication unit 16 to transmit the proxy video. Through the above processing, communication TP71 in FIG. 39 is performed.

[0224] The process of the server device 2 for the communication TP72 in FIG. 39 is shown in FIG. The control unit 2a of the server device 2 confirms reception of the proxy video from the image capture device 1 in step S501. When the proxy moving image is being received, the control unit 2a executes a process of transmitting the proxy moving image to the client terminal 3 in step S502.

[0225] In response to this, the client terminal 3 performs the UI process as the process PS40 as described above, and transmits a request to clip a portion of the moving image as communication TP73. The process of the server device 2 for communication TP74 in response to a request for a partial clip of the moving image is similar to that shown in FIG.

[0226] An example of processing for the processing PS41 and communication TP75 of the imaging device 1 is shown in FIG. The camera control unit 18 confirms receipt of the video request from the server device 2 in step S430. When the video request list is received, the camera control unit 18 executes a loop process LP60.

[0227] In the loop process LP60, the camera control unit 18 repeats the processes from step S431 to step S442 a necessary number of times (the number of requested videos) according to the contents of the video request. First, in step S431, the camera control unit 18 determines whether the requested moving image is a proxy moving image or a main moving image (a request for a partial clip of the main moving image) and branches the process accordingly. The cases where a proxy video is requested occur in a ninth transmission pattern and a tenth transmission pattern, which will be described later, and the processes from step S432 to step S434 will be described later.

[0228] If the request is for a partial clip of the main moving image, the camera control unit 18 performs the processes of steps S440, S441, and S442. In step S440, the camera control unit 18 causes the recording control unit 14 to read the main moving image from the recording medium. In step S441, the camera control unit 18 causes, for example, the camera signal processing unit 13 to execute a process of cutting out the requested section from the main moving image read from the recording medium. Then, in step S442, the camera control unit 18 performs a process of transmitting the partial clip of the main moving image generated in the cut-out process to the server device 2 via the communication unit 16.

[0229] The process of the server device 2 for the communication TP76 is shown in FIG. The control unit 2a of the server device 2 confirms receipt of the main moving image from the imaging device 1 in step S530. When the main video is received, the control unit 2a performs a process of transmitting the received main video to the client terminal 3 in step S531. As a result, a partial clip of the original video is delivered to the client terminal 3.

[0230] <13. 9th transmission pattern> The ninth transmission pattern is explained with reference to FIG. ·Communication TP81 The imaging device 1 transmits an audio-only file for a moving image to the server device 2. An audio-only file is a file that contains only audio data that was recorded when the moving image was recorded.

[0231] ·Communication TP82 The server device 2 transmits the received sound-only file to the client terminal 3 .

[0232] Processing PS42 The client terminal 3 performs UI processing using the received audio-only file. For example, the client terminal 3 executes a display as shown in Fig. 46A on the display unit 77. In this case, since only an audio file is received, the proxy video playback image 46 cannot be played back and is a blank image. Only the audio waveform is displayed as the timeline 47. Furthermore, a proxy video request button 49 is displayed so that the person in charge can operate it.

[0233] Such a UI screen is used by the person in charge of the client terminal 3 to first roughly specify a required section from the audio waveform timeline 47. From the audio waveform, for example, sections in which the audience is excited can be determined. The person in charge operates the proxy video request button 49 after designating the section DKA. In the process PS42, the CPU 71 of the client terminal 3 controls the display of such a UI screen and detects the corresponding input by the person in charge.

[0234] ·Communication TP83 The client terminal 3 transmits a request to clip a portion of the proxy video to the server device 2. That is, the client terminal 3 generates information requesting the proxy video of the section DKA specified by the person in charge in the UI processing of the process PS42, and transmits the information.

[0235] ·Communication TP84 The server device 2 transfers the request for clipping a portion of the proxy video received from the client terminal 3 to the imaging device 1.

[0236] Processing PS43 In response to receiving the request to clip a portion of the proxy video, the image capture device 1 performs processing to create a clip of the proxy video by cutting out the specified section. Communication TP85 The image capture device 1 transmits to the server device 2 a partial clip of the proxy moving image generated in the process PS43.

[0237] The server device 2 transmits the received proxy video to the client terminal 3 as a communication TP72. The process after this communication TP72 is similar to the process after communication TP72 shown in FIG. In the client terminal 3, a UI display as shown in Fig. 46B is executed as a process PS40. This results in the same display content as in Fig. 40. 39 in that the proxy video in this case is not of the entire video recorded by the image capture device 1, but of the section specified in process PS42. The description of communication TP73 and subsequent steps is omitted since it is a repetition.

[0238] That is, in the ninth transmission pattern, first, the audio-only file is used to roughly specify a section, then the proxy video is transmitted accordingly, and the client terminal 3 checks the proxy video and specifies the required section. In response to this, a partial clip of the main video for the required section is transferred as the main video transfer process.

[0239] FIG. 47 shows an example of processing performed by the imaging device 1 for this ninth transmission pattern when recording a moving image. The camera control unit 18 of the imaging device 1 performs a moving image recording process as a loop process LP50, similar to that in FIG. Thereafter, when a request to stop recording is made, for example, by an operation by the cameraman, the camera control unit 18 proceeds to step S410 and controls the reading of only the audio data from the recorded video. For example, the recording control unit 14 reads only the audio data from the video file, and the camera signal processing unit 13 generates an audio-only file. Then, in step S411, the camera control unit 18 controls the communication unit 16 to transmit the audio-only file. Through the above processing, communication TP81 in FIG. 45 is performed.

[0240] The process of the server device 2 for the communication TP82 is shown in FIG. The control unit 2a of the server device 2 confirms receipt of the audio-only file from the imaging device 1 in step S511. When an audio-only file is being received, the control unit 2a executes a process of transmitting the audio-only file to the client terminal 3 in step S512.

[0241] In response to this, the client terminal 3 performs the UI process as the process PS42 as described above, and transmits a request to clip a portion of the proxy video as the communication TP83. The process of the server device 2 for communication TP84 in response to a partial clip request for the proxy video is similar to that shown in FIG.

[0242] The process PS43 and the process for the communication TP85 of the imaging device 1 are performed according to the above-mentioned FIG. That is, when a request to clip part of a proxy video is received, the process advances from step S431 to step S432 in FIG.

[0243] In step S432, the camera control unit 18 causes the recording control unit 14 to read the proxy moving image from the recording medium. In step S433, the camera control unit 18 causes, for example, the camera signal processing unit 13 to execute a process of cutting out the requested section from the proxy video read from the recording medium. Then, in step S434, the camera control unit 18 performs a process of transmitting the partial clip of the proxy moving image generated in the cut-out process to the server device 2 via the communication unit 16.

[0244] The processes of the server device 2 and the imaging device 1 after communication TP72 in FIG. 45 are similar to those described in the eighth transmission pattern.

[0245] <14. 10th transmission pattern> The tenth transmission pattern is explained with reference to FIG. ·Communication TP91 The imaging device 1 transmits a meta time-series file for a moving image to the server device 2. The meta time-series file is a data file capable of presenting meta information (for example, shooting position information) generated when recording a moving image in chronological order.

[0246] ·Communication TP92 The server device 2 transmits the received meta time-series file to the client terminal 3 .

[0247] Processing PS44 The client terminal 3 performs UI processing using the received meta time-series file. For example, the client terminal 3 executes a display based on a meta time-series file as shown in FIG. 50 on the display unit 77. For example, when a change in shooting location information over time can be determined as the meta time-series file, a map image 50 and time-series information 51 according to the shooting location information are displayed. In the map image 50, each shooting location is indicated by a marker. The time-series information 51 indicates the time and the shooting location information in chronological order. Only the timeline 47 indicating the section length is displayed. It is desirable for the timeline 47 to display the position at each point in time. A proxy video request button 49 is also displayed so that the person in charge can operate it.

[0248] Such a UI screen is used by the person in charge of the client terminal 3 to roughly specify a section in which the desired subject is likely to be captured based on the shooting position. By learning information that an event occurred at a certain time and place, the person in charge can determine the video section in which the event may have been captured. The person in charge operates the proxy video request button 49 after designating the section DKP. In the process PS44, the CPU 71 of the client terminal 3 controls the display of such a UI screen and detects the corresponding input by the person in charge.

[0249] As the communication TP83, the client terminal 3 generates information requesting a proxy video of the section DKP designated by the person in charge in the UI processing of the process PS44, and transmits this information. This communication TP83 and subsequent communication are the same as the ninth transmission pattern shown in FIG. 45, so the description will be omitted.

[0250] That is, in the tenth transmission pattern, first, a rough section is specified by time-series information, then the proxy video is transmitted accordingly, and the client terminal 3 checks the proxy video and specifies the required section. In response to this, a partial clip of the main video for the required section is transferred as the main video transfer process.

[0251] FIG. 51 shows an example of processing performed by the imaging device 1 for this tenth transmission pattern when recording a moving image. The camera control unit 18 of the imaging device 1 performs a moving image recording process as a loop process LP50, similar to that in FIG. Thereafter, when a request to stop recording is made by, for example, an operation by the cameraman, the camera control unit 18 proceeds to step S420 and controls the reading of time-series data of meta information about the recorded video. For example, the recording control unit 14 reads meta information about the video file, and the camera signal processing unit 13 generates a time-series file. Then, in step S421, the camera control unit 18 controls the communication unit 16 to transmit the time-series file. Through the above processing, communication TP91 in FIG. 49 is performed.

[0252] The process of the server device 2 for the communication TP92 is shown in FIG. The control unit 2a of the server device 2 confirms receipt of the time-series file from the imaging device 1 in step S521. When the time-series file is being received, the control unit 2a executes a process of transmitting the time-series file to the client terminal 3 in step S522.

[0253] In response to this, the client terminal 3 performs the UI process as the process PS44 as described above, and transmits a request to clip a portion of the proxy video as the communication TP83. The subsequent processing by the server device 2 and the imaging device 1 is similar to that described in the eighth and ninth transmission patterns.

[0254] <15. Summary and variations> According to the above embodiment, the following effects can be obtained. In response to receiving related information associated with an image captured by the imaging device 1 and having a smaller amount of data than the image, the control unit 2a of the server device 2 in the embodiment performs an initial information transfer process of obtaining additional information based on the related information and transmitting the related information and additional information to the client terminal 3. The control unit 2a also performs an original image transfer process of transmitting an image transmitted from the imaging device 1 to the client terminal 3 in response to request information generated by a selection process in the client terminal 3 based on the transmission of the related information or additional information. By performing such processing, the server device 2 can realize image transfer that does not increase the network load or the system memory capacity load and does not cause transfer delays, for example in a scene where immediate delivery is required after shooting for news reports, while properly transferring high-resolution original images required by the client terminal 3. Furthermore, by being able to present images based on related information at the client terminal 3, it is possible to provide an environment in which the person in charge can easily and appropriately select the necessary images.

[0255] Note that specific examples of the information processing device as the server device 2 or the client terminal 3 include various examples such as a personal computer device, a tablet-type information processing device, a mobile phone device, a game device, an audio device, a video device, a communication device, a television device, a server device, etc. Any device capable of performing calculations as information processing, for example, a device having a built-in microcomputer, can be realized as the information processing device of the present disclosure.

[0256] Although the MEC 5 is shown in FIG. 1, the MEC 5 may have a function substituting for the server device 2 in the embodiment. Furthermore, the MEC 5 may have some of the functions of the imaging device 1 (transmission function).

[0257] In the embodiment, the server device 2 has given an example in which, in the initial information transfer process, additional information is obtained by performing a DB search based on related information (for example, meta information) (see the first transmission pattern). By obtaining more detailed additional information from meta information etc. regarding the original image through a DB search, it becomes possible to provide the person in charge at the client terminal 3 with information that is more easily understood and more suitable for image selection. This optimizes the selection operation, and improves the efficiency of network communication.

[0258] In the embodiment, the related information includes an image with a smaller amount of data (e.g., a thumbnail image) representing an image captured by the imaging device 1, and in the initial information transfer process, the server device 2 obtains the additional information by performing image recognition processing on the image included in the related information (see the second transmission pattern). By obtaining information about the subject through image analysis and using it as additional information, it becomes possible to provide the person in charge at the client terminal 3 with information that is easier to understand and more suitable for image selection, which also contributes to optimizing the selection operation and improving the efficiency of network communication.

[0259] As described above, the server device 2 of the embodiment performs a switching process between multiple transmission patterns as an initial information transfer process, including the first and second transmission patterns that add additional information and transmission patterns (second to tenth transmission patterns) that do not add additional information. Since there are environments, situations, image (subject) contents, etc. that are not suitable for transferring additional information, it is not always appropriate for the server device 2 to perform processing to obtain additional information. Therefore, in some cases, the server device 2 may be prevented from performing processing to obtain additional information, thereby making it possible to perform processing appropriate to the situation or circumstances. This also contributes to reducing the server load and improving network efficiency.

[0260] In the embodiment, an example has been given in which the server device 2 performs a process of transmitting voice information included in related information to the client terminal 3 as a predetermined transmission pattern, which is one of a plurality of transmission patterns (see the fifth transmission pattern). When a voice memo associated with a still image or a video is transmitted from the imaging device, the server device 2 transfers the voice memo to the client terminal 3 during the initial information transfer process, so that the person in charge of the client terminal 3 can play back the voice memo and select an image. This provides an environment where more desirable image selection can be performed.

[0261] In the embodiment, an example has been given in which the server device 2 performs a process of converting voice information included in the related information into text data and transmitting the text data to the client terminal 3 as a transmission of a predetermined transmission pattern, which is one of a plurality of transmission patterns (see the sixth transmission pattern). When a voice memo associated with a still image or a video is transmitted from the imaging device, the server device 2 converts the voice memo into text data during the initial information transfer process and transfers it to the client terminal 3. By converting the voice memo into text data, the amount of data can be reduced compared to voice information. Furthermore, by displaying the voice memo content in text form, the person in charge at the client terminal 3 does not need to play back the voice memo, providing an environment in which images can be selected more easily.

[0262] In the embodiment, an example has been given in which the server device 2 performs the switching process depending on the determination result of the communication situation (see switching process example I). By switching the transmission pattern in response to changes in communication conditions, such as the communication speed of the network transmission path, packet rate, 4G / 5G status, and the communication environment depending on the location of the imaging device, it is possible to transfer images according to the current situation. In particular, when transferring images of sports, the communication environment can easily change outdoors. In addition, at large-scale sporting events, there are many reporters covering the event, and transmission paths can become congested. By selecting an appropriate transmission pattern according to such conditions, it is possible to maintain an appropriate transfer state in real time. It is likely that the network conditions will change, particularly between the imaging device 1 and the server device 2. This is because factors such as the position and movement of the cameraman, and the area, location, facilities, and conditions of the venue for a sporting event or the like are involved. Therefore, with respect to whether or not additional information is to be transferred, the transmission pattern may be switched depending on whether or not related information for obtaining the additional information is to be transferred. For example, when obtaining additional information through image analysis, the amount of data can be adjusted according to the communication conditions by switching between requesting a thumbnail image from the imaging device 1 and not obtaining additional information, thereby eliminating the need to send a thumbnail image. Furthermore, the switching process is not limited to a switching process between a transmission pattern including additional information (e.g., the first or second transmission pattern) and a transmission pattern not including additional information (e.g., the third to seventh transmission patterns), but may be performed within the third to seventh transmission patterns depending on the communication situation. For example, the third transmission pattern and the fourth transmission pattern may be switched, or the fifth transmission pattern and the third transmission pattern may be switched. Similarly, it is also possible to switch between the first transmission pattern and the second transmission pattern. Furthermore, by switching from still image transfer to video transfer, switching may be performed from any of the first to seventh transmission patterns to any of the eighth to tenth transmission patterns, and further, during video transfer, switching may be performed from the eighth to the tenth transmission pattern depending on the communication conditions.

[0263] In the embodiment, an example has been given in which the server device 2 performs the switching process based on information about the subject to be photographed notified from the imaging device 1 (see switching process example II). Depending on the type of sport, additional information may not be obtained properly. For example, if the additional information is the player's name, if the sport has a fixed schedule, the player's name can be obtained by performing a DB search based on the image capture date, time, and location, but there are some sports where this is not possible. For images of sports for which additional information cannot be obtained, performing the process of obtaining additional information may result in a decrease in processing efficiency. Therefore, it is preferable to switch the transmission pattern according to the information on the type of sport, as in switching process example II.

[0264] In the embodiment, an example has been given in which the server device 2 performs the process of estimating the subject to be photographed by the imaging device 1, and performs the switching process based on information on the estimated subject to be photographed (see switching process example III). There are cases where it is appropriate to switch the transmission pattern depending on the type of sport. For example, in switching process example III, project information is registered in advance. This makes it possible to estimate the type of sport of the subject being photographed based on time information, camera ID, cameraman information, and other meta information related to the image. By switching the transmission pattern through this estimation process, it is possible to perform transfer appropriate to the type of sport.

[0265] In the embodiment, an example has been given in which the server device 2 performs processing to notify the client terminal 3 or the imaging device 1 of the transmission pattern of the initial information transfer processing set by the switching processing (see switching processing examples I, II, and III). For example, in the imaging device 1, it is possible to set the information content to be transmitted in response to notification of the transmission pattern. For example, it is possible to set whether meta information or a thumbnail image is to be transmitted in the first transmission, and what kind of meta information is to be transmitted. The switching process not only switches between a transmission pattern that acquires additional information and a transmission pattern that does not, but also switches between, for example, the first transmission pattern to the tenth transmission pattern, and further switches between whether to perform the initial information transfer process in the first transmission pattern to the tenth transmission pattern or not. For example, depending on the communication situation, there may be cases where the original image is transferred from the beginning without performing the initial information transfer process. Therefore, notifying the imaging device 1 of the transmission pattern is useful for cooperative control. Furthermore, by notifying the client terminal 3 of the transmission pattern of the initial information transfer process, it becomes possible to display, for example, the display shown in FIG. 31, so that the person in charge can easily understand what type of transfer is being performed.

[0266] In the embodiment, an example has been given in which the related information is meta information associated with an image captured by the imaging device 1. That is, it is information associated with an image on the imaging device 1 side and transmitted to the server device 2. A variety of information is envisioned, such as information about the imaging device 1, information about the photographer, information about the imaging location and time, image analysis information on the imaging device 1 side, various parameters related to imaging operation and processing, sensing information of various sensors in the imaging device 1 at the time of imaging, information about image data / files, thumbnail images, screen nail images, text information and voice memo information associated with images by the photographer, and the like. This meta information is useful for presenting the image contents and for obtaining additional information about the image.

[0267] In the embodiment, an example has been given in which the related information is an image representing an image captured by the imaging device 1 and having a smaller amount of data. Examples of images with smaller amounts of data include thumbnail images, screennail images, and proxy videos. Thumbnail images and proxy videos, which are images with less data than the original image, are extremely useful information for the person in charge of the client terminal 3 to determine the contents of the original image. They are also useful as information for obtaining additional information by image analysis.

[0268] In the embodiment, an example has been given in which the related information is an image representing grouped images captured by the imaging device 1 through continuous shooting (see the seventh transmission pattern). When a series of images captured by continuous shooting are grouped together, an image that expresses the entire series as a single image, for example a thumbnail image of the top image, is used as related information. Grouped images taken in rapid succession have a large number of images and a large amount of data, but they have almost the same subject. Therefore, they can be expressed as representative thumbnail images, and it is desirable to transmit such thumbnail images in the initial information transfer process in order to improve processing efficiency and reduce network load.

[0269] In the embodiment, an example has been given in which the related information is audio information recorded together with a moving image by capturing the moving image (see the ninth transmission pattern). For example, in the initial information transfer process, the server device 2 receives, as related information, a sound-only file that was recorded simultaneously with the video recording, and transmits the received information to the client terminal 3 . The audio information of the video is useful information for the person in charge at the client terminal 3 to check the contents of the video and identify the necessary scenes by listening to the audio information of the video.

[0270] In the embodiment, an example has been given in which the related information is time-series data on the time of capturing a moving image (see the tenth transmission pattern). The time-series data of the video is useful information for the person in charge of the client terminal 3 to check the video content and identify the necessary scenes.

[0271] In the embodiment, the server device 2 performs, as an initial information transfer process, an example of a process of transmitting first related information to the client terminal 3 together with or without additional information, a process of transferring request information generated by a selection process in the client terminal 3 based on a transmission including the first related information to the imaging device 1, and a process of transmitting second related information transmitted from the imaging device 1 in response to the request information to the client terminal 3 (see the first to seventh, ninth and tenth transmission patterns). That is, as an initial information transfer process, before the original image is transmitted to the client terminal 3, information is transmitted a number of times. As an initial information transfer process, the server device 2 can, for example, first transmit meta-information to the client terminal 3 to allow the client to make a rough selection, and second transmit a thumbnail image to allow the client to make a final selection, thereby allowing the client terminal 3 to make a more appropriate selection that is easy for the person in charge to understand, without excessively increasing the amount of data to be transferred. In the embodiment, an example has been described in which the initial information transfer process mainly involves two round trips, but the initial information transfer process before the transmission of the actual image may involve only one round trip, or it may involve three or more round trips.

[0272] In the embodiment, the first related information when the initial information transfer process is performed multiple times is information having a smaller data amount than the second related information. For example, the first related information may be meta information indicating the time and place of photography, and the second related information may be a thumbnail image. In the first round trip of the initial information transfer process, first related information with a relatively small amount of data, such as the shooting time and shooting location, is sent to the client terminal 3, making it possible to present information representing many images that can be transferred without increasing the transfer burden. In the second round trip, for example, thumbnail images are transferred to present more detailed image content in accordance with the initial selection of the client terminal 3. This allows thumbnail images to be transferred only for images that have been narrowed down to a certain extent, making it possible for the person in charge of the client terminal 3 to more clearly grasp and select the image content, again without increasing the transfer burden.

[0273] The imaging device 1 of the embodiment is equipped with a camera control unit 18 that performs a related information transmission process to transmit related information, which is information associated with a captured image and has a smaller amount of data than the image, to an external server device 2, and an original image transmission process to transmit an image indicated in request information received from the server device 2 to the server device 2 in response to the request information. This makes it possible to realize the imaging device 1 that operates in a manner suitable for the image transfer system of the embodiment.

[0274] The program of the embodiment is a program that causes the above-mentioned processing of the server device 2 to be executed by, for example, a CPU, a DSP, or a device including these. In other words, the program of the embodiment is a program that causes an information processing device to execute an initial information transfer process in which, in response to receiving related information associated with an image captured by the imaging device 1 and having a smaller amount of data than the image, additional information is obtained based on the related information and the related information and additional information are transmitted to the client terminal 3, and an original image transfer process in which, in response to request information generated by a selection process in the client terminal 3 based on the transmission of the related information and additional information, an image transmitted from the imaging device 1 is transmitted to the client terminal 3. By using such a program, the above-mentioned server device 2 can be realized by various types of computer devices.

[0275] The program according to the embodiment is a program that causes the processing of the imaging device 1 to be executed by, for example, a CPU, a DSP, or a device including these. In other words, the program is a program that causes a processor or the like acting as a control unit to execute a related information transmission process that transmits related information associated with a captured image and having a smaller amount of data than the image to an external server device 2, and an original image transmission process that transmits an image indicated by request information to the server device 2 in response to request information received from the server device 2. The imaging device 1 described above can be realized by such a program.

[0276] These programs can be recorded in advance on a HDD as a recording medium built into a device such as a computer device, or on a ROM in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0277] Furthermore, such a program is suitable for a wide range of provision of the server device 2 and the imaging device 1 of the embodiment. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), or the like, the smartphone or the like can function as the server device 2 or the imaging device 1 of the present disclosure.

[0278] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0279] The present technology can also be configured as follows. (1) an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information and the additional information to a client terminal; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit for performing Information processing device. (2) The control unit obtains the additional information by performing a database search based on the related information in the initial information transfer process. The information processing device according to (1) above. (3) the related information includes an image with a smaller amount of data representing an image captured by an imaging device; The control unit obtains the additional information by performing an image recognition process on an image included in the related information in the initial information transfer process. The information processing device according to (1) or (2) above. (4) The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed. An information processing device according to any one of (1) to (3) above. (5) The control unit is A process of transmitting voice information included in the related information to the client terminal as a transmission of a predetermined transmission pattern, which is one of a plurality of transmission patterns, is performed. The information processing device according to (4) above. (6) The control unit is As a transmission of a predetermined transmission pattern, which is one of a plurality of transmission patterns, a process of converting voice information included in the related information into text data and transmitting the text data to the client terminal is performed. The information processing device according to (4) above. (7) The control unit is The switching process is performed according to the result of the communication status determination. An information processing device according to any one of (4) to (6) above. (8) The control unit is The switching process is performed based on information on a subject to be photographed notified from the imaging device. An information processing device according to any one of (4) to (7) above. (9) The control unit is An estimation process of a subject to be photographed by the imaging device is performed, and the switching process is performed based on information of the estimated subject to be photographed. An information processing device according to any one of (4) to (8) above. (10) The control unit is A process of notifying the client terminal or the imaging device of the transmission pattern of the initial information transfer process set by the switching process. An information processing device according to any one of (4) to (9) above. (11) The related information is meta information associated with an image captured by the imaging device. An information processing device according to any one of (1) to (10) above. (12) The related information is an image with a smaller amount of data representing an image captured by the imaging device. An information processing device according to any one of (1) to (11) above. (13) The related information is an image representing a grouping of images that is a plurality of images continuously captured by the imaging device. An information processing device according to any one of (1) to (12) above. (14) The related information is audio information recorded together with the moving image captured by the imaging device. An information processing device according to any one of (1) to (13) above. (15) The related information is time-series data on the video captured by the imaging device. An information processing device according to any one of (1) to (14) above. (16) The control unit, as the initial information transfer process, sending the first related information to the client terminal together with or without the additional information; a process of transferring request information generated by a selection process in the client terminal based on the transmission including the first related information to the imaging device; and transmitting, in response to the request information, second related information transmitted from the imaging device to the client terminal. An information processing device according to any one of (1) to (15) above. (17) The first related information is information having a smaller amount of data than the second related information. The information processing device according to (16) above. (18) an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information and the additional information to a client terminal; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information and the additional information; The information processing method is performed by an information processing device. (19) a related information transmission process for transmitting related information associated with the captured image, the related information having a smaller amount of data than the captured image, to an external information processing device; an original image transmission process for transmitting an image indicated by request information received from the information processing device to the information processing device in response to the request information; A control unit for performing Imaging device. (20) Information communication is enabled between the imaging device and the information processing device, Information communication is enabled between the information processing device and a client terminal; The imaging device includes: a related information transmission process for transmitting related information associated with the captured image, the related information having a smaller amount of data than the captured image, to the information processing device; an original image transmission process that is generated by a selection process in the client terminal that receives the related information via the information processing device, and transmits an image indicated by the request information to the information processing device in response to the request information received via the information processing device; A control unit that performs The information processing device includes: an initial information transfer process of obtaining additional information based on the related information in response to receiving the related information from the imaging device, and transmitting the related information and the additional information to the client terminal; an original image transfer process for transmitting an image transmitted from the imaging device in response to the request information to the client terminal; A control unit for performing Image transfer system. [Explanation of symbols]

[0280] 1. Imaging device 2. Server equipment 2a Control section 3. Client terminal 4 base station 5 MEC 6 Network 7 DB 8. Text Conversion Engine 18 Camera control section 70 Information processing device, 71 CPU

Claims

1. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information and the additional information to a client terminal; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs the related information includes an image with a smaller amount of data representing an image captured by an imaging device; The control unit obtains the additional information by performing an image recognition process on an image included in the related information in the initial information transfer process. Information processing device.

2. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed, and a process of transmitting audio information included in the related information to the client terminal as transmission of a predetermined transmission pattern that is one of the plurality of transmission patterns is performed. Information processing device

3. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed, and a process of transmitting text data obtained by converting voice information included in the related information into text data to the client terminal as transmission of a predetermined transmission pattern that is one of the plurality of transmission patterns is performed. Information processing device.

4. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed according to a result of determining a communication situation. Information processing device.

5. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed based on information on a subject to be photographed notified from the imaging device. Information processing device.

6. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed based on information of a subject estimated by performing an estimation process of a subject to be photographed by the imaging device. Information processing device.

7. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit is As the initial information transfer process, a process of switching between a plurality of transmission patterns including a transmission pattern in which the additional information is added and a transmission pattern in which the additional information is not added is performed, and a process of notifying the client terminal or the imaging device of the transmission pattern of the initial information transfer process set by the switching process is performed. Information processing device.

8. an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information to a client terminal, and for enabling the additional information to be transmitted; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information or the additional information; A control unit that performs The control unit, as the initial information transfer process, sending first related information to the client terminal together with or without the additional information; a process of transferring request information generated by a selection process in the client terminal based on the transmission including the first related information to the imaging device; and transmitting second related information transmitted from the imaging device to the client terminal in response to the request information. Information processing device.

9. The first related information is information having a smaller amount of data than the second related information. The information processing device according to claim 8.

10. The control unit obtains the additional information by performing a database search based on the related information in the initial information transfer process. The information processing device according to claim 1 .

11. The related information is meta information associated with an image captured by the imaging device. The information processing device according to claim 1 .

12. The related information is an image with a smaller amount of data representing an image captured by the imaging device. The information processing device according to claim 1 .

13. The related information is an image representing a grouping of images that is a plurality of images continuously captured by the imaging device. The information processing device according to claim 1 .

14. The related information is audio information recorded together with the moving image captured by the imaging device. The information processing device according to claim 1 .

15. The related information is time-series data on the video captured by the imaging device. The information processing device according to claim 1 .

16. An information processing device, an initial information transfer process for receiving related information associated with an image captured by an imaging device and having a smaller amount of data than the image, obtaining additional information based on the related information, and transmitting the related information and the additional information to a client terminal; an original image transfer process for transmitting an image transmitted from the imaging device to the client terminal in response to request information generated by a selection process in the client terminal based on the transmission of the related information and the additional information; Do the following: the related information includes an image with a smaller amount of data representing an image captured by an imaging device; In the initial information transfer process, the additional information is obtained by performing image recognition processing on the image included in the related information. Information processing methods.

17. Information communication is enabled between the imaging device and the information processing device, Information communication is enabled between the information processing device and a client terminal; The imaging device includes: a related information transmission process for transmitting related information associated with the captured image, the related information having a smaller amount of data than the captured image, to the information processing device; an original image transmission process that is generated by a selection process in the client terminal that receives the related information via the information processing device, and transmits an image indicated by the request information to the information processing device in response to the request information received via the information processing device; A control unit that performs The information processing device includes: an initial information transfer process of obtaining additional information based on the related information in response to receiving the related information from the imaging device, and transmitting the related information and the additional information to the client terminal; an original image transfer process for transmitting an image transmitted from the imaging device in response to the request information to the client terminal; A control unit that performs the related information includes an image with a smaller amount of data representing an image captured by an imaging device; The control unit of the information processing device obtains the additional information by performing an image recognition process on an image included in the related information in the initial information transfer process. Image transfer system.

Images