Information processing equipment, information processing methods, imaging equipment and image forwarding systems

Abstract

An information processing apparatus is provided for use as a server device performing forwarding from an imaging device to a client terminal. The information processing apparatus includes a control unit that performs: initial information forwarding processing, wherein, in response to receiving associated information, additional information is obtained based on the associated information, and the associated information and the additional information are sent to the client terminal, the associated information being associated with an image captured by the imaging device and having a data size smaller than that of the captured image; and actual image forwarding processing, wherein an image sent from the imaging device is sent to the client terminal according to request information generated through selection processing at the client terminal.

Description

Technical Field

[0001] This technology relates to information processing equipment, information processing methods, imaging equipment, and image forwarding systems, and particularly to techniques suitable for forwarding images captured by an imaging device to other devices. Background Technology

[0002] Images captured by imaging devices (still or moving images) are typically sent to other devices, such as computer devices.

[0003] Patent document 1 below discloses a technique for first transmitting related data that has a smaller amount of data relative to the image data.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2007-150781 Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] For purposes such as news reporting via television broadcasting or online distribution, there are situations where camera operators are required to deliver still or moving images immediately after capturing them on-site.

[0009] In this scenario, the camera operator connects the imaging equipment to the network at the imaging site and sends the captured images to the network for delivery. At the delivery destination device (e.g., equipment at a broadcasting station, etc., and referred to as the "client terminal"), a designated person responsible for selection chooses the delivered images and uses them for news reporting.

[0010] In such a situation, due to the increased network load, it may take time to send large images (image data of quality / volume for news reporting) from the imaging device to external devices (e.g., relay cloud servers or client terminals), and correspondingly, the image selection process may also take time.

[0011] Furthermore, when all captured images are sent from the imaging device to an external device, the storage capacity required by the destination device needs to be increased, and the system load increases.

[0012] Therefore, this disclosure proposes a technique for sending appropriate images without significantly increasing network communication and system load.

[0013] Solution to the problem

[0014] The information processing device according to this technology includes: a control unit that performs: initial information forwarding processing, in response to receiving associated information, obtaining additional information based on the associated information, and sending the associated information and the additional information to a client terminal, wherein the associated information is information associated with an image captured by an imaging device and has a smaller data volume than the image; and main image forwarding processing, sending an image sent from the imaging device to the client terminal according to a request information generated by a selection process performed in the client terminal based on the transmission of the associated information or the additional information.

[0015] For example, suppose an information processing device is capable of forwarding information with both an imaging device and a client terminal. Before forwarding image data (the main image) as a captured image, the information processing device first sends associated and supplementary information to the client terminal. Then, based on request information obtained as a result of selection processing based on the associated and supplementary information, the main image is forwarded from the imaging device to the client terminal.

[0016] In the information processing device according to the present technology described above, it is conceivable that the control unit obtains the additional information by performing a database search based on the associated information during the initial information forwarding process.

[0017] By performing a database search based on the correlation information provided by the imaging device, more detailed information about the image (subject) can be obtained from the correlation information. This is called supplementary information.

[0018] In the information processing device according to the present technology described above, it is conceivable that the associated information includes an image representing an image captured by the imaging device and having a smaller data volume, and that the control unit obtains the additional information by performing image recognition processing on the image included in the associated information in the initial information forwarding process.

[0019] When the associated information provided by the imaging device includes images such as thumbnail images or screen nail images, information about the subject can be obtained by analyzing the images. This is called supplementary information.

[0020] In the information processing device according to the present technology described above, it is conceivable that, as the initial information forwarding process, the control unit performs a switching process for multiple transmission modes, including a transmission mode with the additional information added and a transmission mode without the additional information added.

[0021] As a transmission mode, for example, multiple transmission modes can be changed, such as a mode in which additional information is obtained based on the associated information provided by the imaging device and the associated information and additional information are sent to the client terminal, a mode in which the associated information provided by the imaging device is forwarded to the client terminal as is, and a mode in which the content of the associated information is different.

[0022] In the information processing device according to the present technology described above, it is conceivable that the control unit performs the process of sending the audio information included in the associated information to the client terminal as a predetermined sending mode, the predetermined sending mode being one of the plurality of sending modes.

[0023] For example, when forwarding audio information associated with an image as a still image from an imaging device, the audio information is sent to the client terminal as is.

[0024] In the information processing device according to the present technology described above, it is conceivable that the control unit performs the process of sending text data obtained by converting the audio information included in the associated information into text data to the client terminal as a predetermined sending mode, the predetermined sending mode being one of the plurality of sending modes.

[0025] For example, in the case of forwarding audio information associated with an image as a still image from an imaging device, the audio information is converted into text data and sent to the client terminal.

[0026] In the information processing device according to the present technology described above, it is conceivable that the control unit performs the switching process based on the determination result of the communication status.

[0027] As an initial information forwarding process, various sending modes are selectively used depending on the communication status.

[0028] In the information processing device according to the present technology described above, it is conceivable that the control unit performs the switching process based on information about the imaging object that is notified from the imaging device.

[0029] Depending on, for example, the type of activity or competition that is the object of imaging, a first process that acquires additional information and a second process that does not acquire additional information are selectively used in the initial information forwarding process.

[0030] In the information processing device according to the present technology described above, it is conceivable that the control unit performs a process of estimating the imaging object of the imaging device, and performs the switching process based on the information of the estimated imaging object.

[0031] When the imaging object of the imaging device can be estimated, the first process of acquiring additional information and the second process of not acquiring additional information can be selectively used as the initial information forwarding process based on the estimation result.

[0032] In the information processing device according to the present technology described above, it is conceivable that the control unit performs the process of notifying the client terminal or the imaging device of the sending mode of the initial information forwarding process set by the switching process.

[0033] When the information processing device switches between first processing and second processing, it notifies the imaging device or client terminal of the sending mode as the result.

[0034] In the information processing device according to the present technology described above, it is conceivable that the associated information includes metadata associated with the image captured by the imaging device.

[0035] That is, it is information associated with the image on the imaging device side and sent to the information processing device. For example, suppose various types of information, such as information about the imaging device, information about the camera operator, information about the location and time of image capture, image analysis information on the imaging device side, various parameters about imaging operation and processing, sensing information of various sensors in the imaging device at the time of image capture, information about image data / files, thumbnail images, screen pin images, and text and audio memo information associated with the image by the camera operator.

[0036] In the information processing device according to the present technology described above, it is conceivable that the associated information includes images representing images captured by the imaging device and images with a smaller data volume.

[0037] Examples of images with smaller data volumes include thumbnail images, screen pin images, and proxy motion images.

[0038] In the information processing device according to the present technology described above, it is conceivable that the associated information includes images representing grouped images, which are multiple images continuously captured by the imaging device.

[0039] When a series of images captured consecutively are set as grouped images, the image that represents the entire group as a single image (e.g., a thumbnail of the first image, etc.) is set as associated information.

[0040] In the information processing device according to the present technology described above, it is conceivable that the associated information includes audio information recorded together with the motion image by motion image capture performed by the imaging device.

[0041] For example, in the initial information forwarding process, the information processing device receives the audio information recorded simultaneously with the motion picture recording as associated information and sends the information to the client terminal.

[0042] In the information processing device according to the present technology described above, it is conceivable that the associated information includes time-series data at the time of capture of the motion image captured by the imaging device.

[0043] Time series data is information that represents the progress of imaging, such as location information at each time point when the image is captured.

[0044] In the information processing device according to the present technology described above, it is conceivable that, as the initial information forwarding process, the control unit performs the process of sending first associated information to the client terminal with or without the additional information, the process of transmitting request information generated by a selection process performed in the client terminal based on the transmission of the first associated information to the imaging device, and the process of sending second associated information from the imaging device to the client terminal according to the request information.

[0045] That is, as the initial information forwarding process, the information transmission is performed multiple times before the main image is sent to the client terminal 3.

[0046] In the information processing device according to the present technology described above, it is conceivable that the first associated information includes information having a smaller data volume than the second associated information.

[0047] For example, one could conceive of an example where the first associated information is metadata representing the imaging time, imaging location, etc., and the second associated information is a thumbnail image, etc.

[0048] The imaging device according to this technology includes: a control unit that performs: an association information transmission process that sends association information to an external information processing device, the association information being information associated with a captured image and having a smaller data volume than the image; and a main image transmission process that sends an image represented by the request information to the information processing device according to a request information received from the information processing device.

[0049] Therefore, the imaging device sends associated information before sending the captured image.

[0050] The image forwarding system according to this technology enables information communication between the imaging device and the information processing device, and also enables information communication between the information processing device and the client terminal. The imaging device has the above-described configuration, and the information processing device also has the above-described configuration.

[0051] Information can be forwarded directly or via other devices or transmission paths between the imaging device and the information processing device, and information can also be forwarded directly or via other devices or transmission paths between the information processing device and the client terminal. Between such devices, firstly, associated information with a smaller data volume than the captured image is forwarded, and a selection is performed on the client terminal side. Subsequently, based on this selection, the main image is sent from the imaging device to the client terminal. Attached Figure Description

[0052] Figure 1 This is an explanatory diagram illustrating a configuration example of an image forwarding system according to an embodiment of the present technology.

[0053] Figure 2 This is a block diagram of the imaging device in this embodiment.

[0054] Figure 3 This is a block diagram of the information processing device in this embodiment.

[0055] Figure 4 This is an explanatory diagram of the first transmission mode in this embodiment.

[0056] Figure 5 These are explanatory diagrams of the UI screen, etc., in the first sending mode of this embodiment.

[0057] Figure 6 This is a flowchart illustrating the processing of imaging by the imaging device in the first, second, third, and fourth transmission modes applicable to this embodiment.

[0058] Figure 7 This is a flowchart illustrating the processing of metadata reception by a server device in the first sending mode applicable to this embodiment.

[0059] Figure 8 This is a flowchart illustrating the process of receiving image requests by the server device in the first to seventh sending modes applicable to this embodiment.

[0060] Figure 9 This is a flowchart of the image request reception process of the imaging device applicable to the first to sixth transmission modes of this embodiment.

[0061] Figure 10 This is a flowchart of the image reception process of the server device applicable to the first and third to seventh transmission modes of this embodiment.

[0062] Figure 11 This is an explanatory diagram of the second transmission mode in this embodiment.

[0063] Figure 12 This is an explanatory diagram of the UI screen, etc., in the second sending mode of this embodiment.

[0064] Figure 13 This is a flowchart illustrating the processing of metadata reception by the server device in the second, third, and fourth transmission modes applicable to this embodiment.

[0065] Figure 14 This is a flowchart of the image reception process of the server device applicable to the second transmission mode of this embodiment.

[0066] Figure 15 This is an explanatory diagram of the third transmission mode in this embodiment.

[0067] Figure 16 These are explanatory diagrams of the UI screen, etc., in the third and fourth sending modes of this embodiment.

[0068] Figure 17 This is a flowchart of the process when the imaging device performs imaging in the fourth transmission mode applicable to this embodiment.

[0069] Figure 18 This is an explanatory diagram of the fifth transmission mode in this embodiment.

[0070] Figure 19 This is an explanatory diagram of the UI screen, etc., in the fifth sending mode of this embodiment.

[0071] Figure 20 This is a flowchart of the processing performed by the imaging device during the fifth and sixth transmission modes applicable to this embodiment.

[0072] Figure 21 This is a flowchart of the processing of metadata reception by the server device in the fifth transmission mode applicable to this embodiment.

[0073] Figure 22 This is an explanatory diagram of the sixth transmission mode in this embodiment.

[0074] Figure 23 This is an explanatory diagram of the UI screen, etc., in the sixth sending mode of this embodiment.

[0075] Figure 24 This is a flowchart of the processing of metadata reception by the server device in the sixth transmission mode applicable to this embodiment.

[0076] Figure 25 This is an explanatory diagram of the seventh transmission mode in this embodiment.

[0077] Figure 26 This is an explanatory diagram of the UI screen, etc., in the seventh sending mode of this embodiment.

[0078] Figure 27 This is a flowchart of the process when the imaging device performs imaging in the seventh transmission mode applicable to this embodiment.

[0079] Figure 28 This is a flowchart of the image request reception process of the imaging device applicable to the seventh transmission mode of this embodiment.

[0080] Figure 29 This is an explanatory diagram of example I of the switching process in this embodiment.

[0081] Figure 30 This is a flowchart of the server device switching process in this embodiment.

[0082] Figure 31 This is an explanatory diagram of the switching of related UI displays in this embodiment.

[0083] Figure 32 This is an explanatory diagram of the UI display of the request switching command in this embodiment.

[0084] Figure 33 This is a flowchart of the situation determination process in this embodiment.

[0085] Figure 34 This is an explanatory diagram of Example II of the switching process in this embodiment.

[0086] Figure 35 This is a flowchart of the server device switching process in this embodiment.

[0087] Figure 36 This is an explanatory diagram of Example III of the switching process in this embodiment.

[0088] Figure 37 This is a flowchart illustrating the process of receiving project information from the server device in this embodiment.

[0089] Figure 38 This is a flowchart of the server device switching process in this embodiment.

[0090] Figure 39 This is an explanatory diagram of the eighth transmission mode in this embodiment.

[0091] Figure 40 These are explanatory diagrams of the UI screen, etc., in the eighth sending mode of this embodiment.

[0092] Figure 41 This is a flowchart of the process when an imaging device in the eighth transmission mode applicable to this embodiment records motion images.

[0093] Figure 42 This is a flowchart illustrating the process of receiving proxy motion images by the server device in the eighth transmission mode applicable to this embodiment.

[0094] Figure 43This is a flowchart illustrating the process of receiving motion image clipping requests by the imaging device applicable to the eighth, ninth, and tenth transmission modes of this embodiment.

[0095] Figure 44 This is a flowchart of the main motion image clipping reception process of the server device applicable to the eighth, ninth, and tenth transmission modes of this embodiment.

[0096] Figure 45 This is an explanatory diagram of the ninth transmission mode in this embodiment.

[0097] Figure 46 This is an explanatory diagram of the UI screen, etc., in the ninth sending mode of this embodiment.

[0098] Figure 47 This is a flowchart of the process for recording motion images by an imaging device in the ninth transmission mode applicable to this embodiment.

[0099] Figure 48 This is a flowchart illustrating the process of receiving only audio files using the server device in the ninth transmission mode applicable to this embodiment.

[0100] Figure 49 This is an explanatory diagram of the tenth transmission mode in this embodiment.

[0101] Figure 50 These are explanatory diagrams of the UI screen and other elements in the tenth sending mode of this embodiment.

[0102] Figure 51 This is a flowchart of the process for recording motion images by an imaging device in the tenth transmission mode applicable to this embodiment.

[0103] Figure 52 This is a flowchart illustrating the processing of time-series file reception by a server device in the tenth transmission mode applicable to this embodiment. Detailed Implementation

[0104] The embodiments will be described below in the following order.

[0105] <1. System Configuration>

[0106] <2. Configuration of Imaging Equipment, Server Equipment, and Client Terminals>

[0107] <3. First Sending Mode>

[0108] <4. Second Sending Mode>

[0109] <5. Third and Fourth Sending Modes>

[0110] <6. Fifth Sending Mode>

[0111] <7. Sixth Sending Mode>

[0112] <8. Seventh Sending Mode>

[0113] <9. Switching Processing Example I>

[0114] <10. Switching Processing Example II>

[0115] <11. Switching Process Example III>

[0116] <12. Eighth Sending Mode>

[0117] <13. Ninth Sending Mode>

[0118] <14. Tenth Sending Mode>

[0119] <15. Summary and Variation Examples>

[0120] First, the definitions of various terms are described.

[0121] "Image" is a term used to refer to various types of image data, including both still images and moving images.

[0122] "Main image" indicates what to start from Figure 1 The imaging device 1 shown delivers a target image to the client terminal 3. The main image includes still images and moving images. In particular, in the case of representing a moving image, it is described as the "main moving image".

[0123] "Associated information" refers to information associated with the image captured by imaging device 1, but is unrelated to the content of the information. Examples include so-called meta-information, audio memo information, and images with a lower data volume than the main image (thumbnail images associated with still and moving images, screen pin images, proxy moving images associated with moving images, etc.), where the meta-information includes information about the imager, date and time, location, various sensing data in the imaging device, etc.

[0124] "Additional information" refers to information that is searched, extracted, or generated by server device 2 based on related information, and is unrelated to the content of the information.

[0125] "Initial Information Forwarding Processing" refers to the process of forwarding associated and additional information before sending the main image. It can be performed multiple times.

[0126] "Main Image Forwarding Processing" refers to the processing of forwarding the main image.

[0127] "Transmission mode" refers to the process and the changing information content to be forwarded until the main image is forwarded between imaging device 1, server device 2, and client terminal 3. In this embodiment, depending on the type of information content, whether the main image is a still image, a continuous imaging image, or a moving image, the first to tenth transmission modes are represented as different forwarded information contents. For example, as initial information forwarding processing, various modes are assumed, such as whether the initial information forwarding processing is performed in each transmission mode, a mode in which one information forwarding is performed from imaging device 1 to client terminal 3 via server device 2, and a mode in which multiple information forwardings are performed.

[0128] <1. System Configuration>

[0129] Figure 1 An example configuration of the image forwarding system of this embodiment is shown. The image forwarding system basically includes one or more imaging devices 1, a server device 2, and a client terminal 3 that can communicate via network 6.

[0130] As imaging device 1, there are various imaging devices such as video cameras and still cameras. The imaging device 1 shown is assumed to be a camera used by a camera operator or reporter in sports or event venues, news gathering sites, etc.

[0131] Both server device 2 and client terminal 3 are information processing devices that include computer equipment. However, since server device 2 provides image forwarding services and client terminal 3 is an information processing device that is a client using the forwarding services of server device 2, they are referred to as "server device" and "client terminal" in this disclosure for the purpose of distinction.

[0132] Assume server device 2 is used as a so-called cloud server. Of course, it is not limited to being used as a cloud server.

[0133] Although control unit 2a is shown in server device 2, control unit 2a performs the initial information forwarding process and main image forwarding process described later.

[0134] In addition, server device 2 can obtain various information by referring to database 7 (hereinafter referred to as "DB"), and can obtain text data by performing audio data to text data conversion using text conversion engine 8.

[0135] DB 7 can be a DB configured on a storage device that can be used locally by server device 2, or it can be a DB configured on a storage device that can search for information via various networks. In other words, DB 7 can be any form of DB, as long as it is a DB that server device 2 (control unit 2a) can access to obtain information.

[0136] Client terminal 3 represents an information processing device from which a camera operator delivers images, such as a radio station, newspaper, or news agency. Radio stations are examples, and the delivery destination is unrestricted.

[0137] As a network 6, we can assume, for example, the Internet, home networks, local area networks (LANs), satellite communication networks, and various other networks.

[0138] exist Figure 1 In the example, imaging device 1 communicates with server device 2 via base station 4. Base station 4 is, for example, a 4G communication (fourth generation mobile communication system) and 5G communication (fifth generation mobile communication system) base station.

[0139] There is a scenario where Mobile Edge Computing (MEC) 5 is deployed in base station 4. There is a scenario where communication for each imaging device 1 is performed through processing by MEC 5, or there is a scenario where some functions of server device 2 are performed by MEC 5.

[0140] By constructing such Figure 1 The image forwarding system shown, for example, in the case where the user of imaging device 1 is a camera operator performing work at a broadcasting station, newspaper, etc., images captured by the camera operator at an event location can be forwarded via server device 2 to client terminal 3 installed in the broadcasting station, etc. On the client terminal 3 side, the person in charge selects the transmitted image and uses it for news reporting.

[0141] As mentioned above, it is assumed that a large number of high-resolution images need to be forwarded on-site immediately after imaging for use in news reporting, etc., which would increase the processing load and network load of each unit.

[0142] Furthermore, when a large number of main images are delivered to the client terminal 3 side, the storage capacity load also increases, and it takes time for the person in charge to perform the selection work, as well as the possibility that the speed required for news reporting, etc., may be compromised.

[0143] Therefore, in this embodiment, the server device 2 divides the forwarding until the delivery of the main image into initial information forwarding processing and main image forwarding processing, thereby reducing network communication load, storage capacity load, and facilitating selection work.

[0144] In addition, image forwarding can be performed at appropriate times by switching the sending mode (process or information content) according to the situation.

[0145] <2. Configuration of Imaging Equipment, Server Equipment, and Client Terminals>

[0146] The following section will describe a configuration example of imaging device 1, server device 2, and client terminal 3.

[0147] First, refer to Figure 2 A configuration example of imaging device 1 is described.

[0148] Imaging device 1 includes, for example, a lens system 11, an imaging element unit 12, a camera signal processing unit 13, a recording control unit 14, a display unit 15, a communication unit 16, an operation unit 17, a camera control unit 18, a memory unit 19, a driver unit 22, a sensor unit 23, an audio input unit 25, and an audio processing unit 26.

[0149] The lens system 11 includes lenses such as zoom lenses and focusing lenses, as well as an aperture mechanism. Light from the subject (incident light) is guided by the lens system 11 and focused onto the imaging element unit 12.

[0150] The imaging element unit 12 includes an image sensor 12a (imaging element), such as, for example, a complementary metal-oxide-semiconductor (CMOS) or a charge-coupled device (CCD).

[0151] For the electrical signal obtained by performing photoelectric conversion on the light received by the image sensor 12a, the imaging element unit 12 performs, for example, correlated double sampling (CDS) processing, automatic gain control (AGC) processing, and further performs analog-to-digital (A / D) conversion processing. Then, the captured signal, as digital data, is output to the camera signal processing unit 13 or the camera control unit 18 in subsequent stages.

[0152] The camera signal processing unit 13 is configured as an image processing processor by means of, for example, a digital signal processor (DSP). The camera signal processing unit 13 performs various signal processing on the digital signals (captured image signals) from the imaging element unit 12. For example, as camera processing, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, resolution conversion processing, file formation processing, etc.

[0153] In the preprocessing, clamping processing is performed on the captured image signal from the imaging element unit 12 to clamp the black level of R, G and B to a predetermined level, and correction processing is performed between the color channels of R, G and B, etc.

[0154] During synchronization processing, color separation is performed so that each pixel's image data contains all R, G, and B color components. For example, in the case of an imaging element using a Bayer array color filter, demosaicing is performed as a color separation process.

[0155] In the YC generation process, luminance (Y) and color (C) signals are generated (separated) from R, G, and B image data.

[0156] In resolution conversion processing, resolution conversion is performed on image data that has already undergone various signal processing steps.

[0157] In file creation processing, for example, image data that has undergone the various processes described above undergoes compression encoding, formatting, generation or addition of metadata, etc., for recording or communication, to generate a file for recording or communication.

[0158] Image files can be generated in formats such as Joint Photo Experts Group (JPEG), Tagged Image File Format (TIFF), and Graphics Interchange Format (GIF) as still image files. Furthermore, it is conceivable to generate image files in formats such as MP4, used for recording motion pictures and audio conforming to MPEG-4.

[0159] Note that one can also think of generating image files as raw image data.

[0160] The camera signal processing unit 13 generates metadata that includes the following: processing parameter information in the camera signal processing unit 13, various control parameters obtained from the camera control unit 18, information indicating the operating status of the lens system 11 or the imaging element unit 12, mode setting information, imaging environment information (date and time, location, etc.), identification information of the imaging device itself, lens mounting information, information of previously registered camera operators (name and identification information), and metadata of the International Press and Telecommunications Commission (IPTC).

[0161] Note that IPTC metadata is metadata in a format designed by the Association of Media Companies, and can describe various types of information, such as “description / caption”, “description author”, “title”, and “keywords”.

[0162] The recording control unit 14 performs recording and playback on a recording medium using, for example, non-volatile memory. For example, the recording control unit 14 performs processing to record image files, thumbnails, screen pin images, etc., such as motion picture data and still image data, on the recording medium.

[0163] Various practical forms of the recording control unit 14 are conceivable. For example, the recording control unit 14 can be configured as a flash memory and its write / read circuitry built into the imaging device 1. Furthermore, the recording control unit 14 can be in the form of a card recording / reproducing unit that performs recording / reproducing access to a recording medium (e.g., a memory card (portable flash memory, etc.)) that can be connected to and detached from the imaging device 1. Additionally, as a form built into the imaging device 1, the recording control unit 14 can be implemented as a hard disk drive (HDD), etc.

[0164] Display unit 15 is a display unit that performs various displays for image capture, and includes, for example, a display panel or viewfinder composed of a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display arranged in the housing of imaging device 1.

[0165] The display unit 15 performs various displays on the display screen based on instructions from the camera control unit 18.

[0166] For example, display unit 15 displays a reproduced image of image data read from the recording medium in recording control unit 14.

[0167] Furthermore, there exists a situation where image data of the captured image, converted by the camera signal processing unit 13 to a resolution suitable for display, is provided to the display unit 15, and the display unit 15 performs display based on the image data of the captured image according to instructions from the camera control unit 18. Therefore, a so-called pass-through image (subject monitoring image) is displayed, which is a captured image taken during framing confirmation, motion picture recording, etc.

[0168] In addition, the display unit 15 performs various operation menus, icons, messages, etc. on the screen based on the instructions of the camera control unit 18, that is, it serves as a graphical user interface (GUI) display.

[0169] The communication unit 16 performs data communication or network communication with external devices via wired or wireless means.

[0170] For example, it performs the sending and output of captured image data (still image files or moving image files) or metadata to external display devices, recording devices, playback devices, etc.

[0171] Furthermore, the communication unit 16, as a network communication unit, can perform communication via various networks 6 such as the Internet, home networks, and local area networks (LANs), and can perform various data transmission and reception to / from servers, terminals, etc. on the network. For example, in the case of this embodiment, the communication unit 16 performs communication processing to send captured image data (such as the aforementioned image files) and metadata to the server device 2.

[0172] Furthermore, the imaging device 1 can be configured to communicate with devices such as smartphones or tablets owned by a camera operator via a communication unit 16 through short-range wireless communication such as Bluetooth (trademark), Wi-Fi (trademark), or Near Field Communication (NFC) or infrared communication. In this case, images can be forwarded via the smartphone.

[0173] Furthermore, imaging device 1 and other devices can communicate with each other via wired connection.

[0174] The operation unit 17 collectively represents the input device for the user to perform various operation inputs. Specifically, the operation unit 17 represents various operation parts (buttons, dials, touch panels, touchpads, etc.) disposed in the housing of the imaging device 1.

[0175] The operation unit 17 senses the user's operation and sends a signal corresponding to the input operation to the camera control unit 18.

[0176] The camera control unit 18 includes a microcomputer (arithmetic processing device) containing a central processing unit (CPU).

[0177] The memory unit 19 stores information for processing performed by the camera control unit 18. Examples of memory units 19 shown include read-only memory (ROM), random access memory (RAM), and flash memory.

[0178] The memory unit 19 may be a memory region built into the microcomputer chip that serves as the camera control unit 18, or it may be composed of a separate memory chip.

[0179] The camera control unit 18 executes programs stored in the ROM, flash memory, etc. of the memory unit 19 to control the entire imaging device 1.

[0180] For example, the camera control unit 18 controls the shutter speed of the imaging element unit 12, various signal processing instructions in the camera signal processing unit 13, imaging or recording operations corresponding to user operations, reproduction operations of recorded image files, operations of the lens system 11 (e.g., zoom, focus and aperture adjustment in the lens barrel), user interface operations, etc., controlling the operation of each required unit.

[0181] As the working area when the CPU of the camera control unit 18 processes various data, the RAM of the memory unit 19 is used for temporary storage of data, programs, etc.

[0182] In addition to the operating system (OS) that the CPU uses to control each unit and content files such as image files, the ROM and flash memory (non-volatile memory) of memory unit 19 are also used to store applications, firmware, various settings information, etc. for various operations.

[0183] Various settings include communication settings, exposure settings, shutter speed settings, mode settings, white balance and color settings, image effect settings, custom key settings and display settings, etc., which are settings related to imaging operation.

[0184] The driver unit 22 is provided with, for example, a motor driver for a zoom lens drive motor, a motor driver for a focusing lens drive motor, and a motor driver for an aperture mechanism motor.

[0185] These motor drivers apply drive current to the corresponding drivers according to instructions from the camera control unit 18, and perform actions such as moving the focusing lens and zoom lens, opening and closing the aperture blades of the aperture mechanism.

[0186] Sensor unit 23 comprehensively represents the various sensors installed on the imaging equipment.

[0187] For example, as sensor unit 23, an inertial measurement unit (IMU) is installed, and for example, a three-axis angular velocity (gyroscope) sensor for pitch, yaw and roll can detect angular velocity, and an acceleration sensor can detect acceleration.

[0188] In addition, as a sensor unit 23, for example, a location information sensor, an illumination sensor, etc. can be installed.

[0189] The audio input unit 25 includes, for example, a microphone, a microphone amplifier, etc., and outputs an audio signal obtained by collecting ambient audio.

[0190] The audio processing unit 26 performs processes such as converting the audio signal obtained from the audio input unit 25 into a digital audio signal, AGC processing, sound quality processing, and noise reduction processing. The audio data that has undergone these processes is output to the camera signal processing unit 13 and the camera control unit 18.

[0191] For example, when capturing moving images, the audio data is processed into audio data accompanying the moving images in the camera control unit 18.

[0192] Furthermore, audio data, which serves as a so-called voice memo during image capture, can be converted into an audio file in the camera signal processing unit 13 or the camera control unit 18.

[0193] Audio files can be recorded on a recording medium in association with image files in the recording control unit 14, or they can be sent and output from the communication unit 16 together with image files.

[0194] Next, in Figure 3 The diagram shows a configuration example of an information processing device 70 that functions as server device 2 and client terminal 3.

[0195] For example, the CPU 71 of the information processing device 70 performs various processes based on programs stored in non-volatile memory cells 74 such as ROM 72 or electrically erasable programmable read-only memory (EEP-ROM) or programs loaded from storage cells 79 into RAM 73. Data required by the CPU 71 to perform various processes are also appropriately stored in RAM 73.

[0196] CPU 71, ROM 72, RAM 73, and non-volatile memory unit 74 are interconnected via bus 83. Input / output interface 75 is also connected to bus 83.

[0197] The input unit 76, which includes the operation section and operation equipment, is connected to the input / output interface 75.

[0198] For example, as input unit 76, various operating units and operating devices are assumed, such as keyboard, mouse, buttons, dial, touch panel, touchpad and remote control.

[0199] The user's operation is sensed by the input unit 76, and the signal corresponding to the input operation is interpreted by the CPU 71.

[0200] In addition, display units 77, including LCDs, organic EL panels, etc., and audio output units 78, including speakers, etc., are connected integrally or separately to the input / output interface 75.

[0201] The display unit 77 is a display unit that performs various displays, and includes, for example, a display device disposed in the housing of the information processing device 70, a separate display device connected to the information processing device 70, etc.

[0202] The display unit 77 performs various image processing operations, such as displaying images and motion images to be processed, on the display screen based on instructions from the CPU 71. Furthermore, the display unit 77 displays various operation menus, icons, messages, etc., based on instructions from the CPU 71; that is, it displays a graphical user interface (GUI).

[0203] There may be a situation where the input / output interface 75 is connected to a storage unit 79, including a hard disk, a solid-state storage device, etc., and a communication unit 80, including a modem, etc.

[0204] The communication unit 80 performs communication processing via a transmission path such as the Internet, or communicates with various devices via wired / wireless communication, bus communication, etc.

[0205] The drive 81 is also connected to the input / output interface 75 as needed to properly mount a removable recording medium 82, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0206] Data files such as image files and various computer programs can be read from the removable recording medium 82 via the drive 81. The read data files are stored in the storage unit 79, and the images and audio included in the data files are output by the display unit 77 and the audio output unit 78. Furthermore, computer programs and the like read from the removable recording medium 82 can be installed in the storage unit 79 as needed.

[0207] In the information processing device 70, for example, the software used for processing in this embodiment can be installed via network communication through the removable recording medium 82 or the communication unit 80. Alternatively, the software can be pre-stored in the ROM 72, storage unit 79, etc.

[0208] For example, if the information processing device 70 is a server device 2, software is installed for forwarding processing, including initial information forwarding processing and main image forwarding processing, and switching processing of transmission modes during forwarding processing, as described below. In this case, the CPU 71 is used as... Figure 1 The control unit 2a performs the necessary processing. Furthermore, various communications, as described below, are performed via the communication unit 80.

[0209] Furthermore, when the information processing device 70 is the client terminal 3, during the forwarding process described below, interface processing using the input unit 76, display unit 77, etc., is performed under the control of the CPU 71. Additionally, various communications are performed via the communication unit 80.

[0210] <3. First Sending Mode>

[0211] Various transmission modes will be described as image forwarding processes performed from imaging device 1 to client terminal 3 via server device 2.

[0212] Figure 4 The communication process and information content are shown as the first transmission mode for image forwarding.

[0213] Note that, as Figure 4 As shown in the diagrams illustrating the first to tenth transmission modes described below, transmission and reception between imaging device 1, server device 2, and client terminal 3 are indicated by arrows with reference numeral TP** ("**" is a numerical value), and the processing of each device is indicated by reference numeral PS**. The vertical direction (from top to bottom) in the diagram corresponds to the progression of time.

[0214] Execute in the process described below Figure 4 The first transmission mode is shown.

[0215] -Communication TP1

[0216] Imaging device 1 sends metadata as associated information about the captured image to server device 2. At this time, the main image is not sent.

[0217] Imaging device 1, for example, sends metadata about one or more images based on the operation of the camera operator.

[0218] - Handling PS1

[0219] Server device 2 searches DB 7 based on the metadata of each received image and derives additional information. For example, if the imaging location and time are obtained as metadata, there are cases where the competition content and contestant names can be specified by searching DB 7 using the imaging location and time. The competition name, contestant names, etc., obtained in this way are examples of additional information.

[0220] In this way, based on the metadata of certain content sent from imaging device 1, server device 2 sets various information obtained through DB search as additional information.

[0221] Furthermore, in processing PS1, it is assumed that a DB search is performed for each piece of metadata about multiple images sent from imaging device 1 to obtain additional information.

[0222] -Communication TP2

[0223] Server device 2 sends a pair of metadata and additional information about each image to client terminal 3.

[0224] - Handling PS2

[0225] Client terminal 3 uses the received metadata and additional information to perform user interface (UI) processing.

[0226] For example, client terminal 3 performs the following on display unit 77: Figure 5 The display shown in Figure A. In this example, it is assumed that the metadata includes information about the camera operator (imager), imaging time, and imaging location, and that the competition name and contestant name are extracted as additional information based on the imaging time and imaging location. Based on such metadata and additional information, a list of information content 30 corresponding to each image is displayed. Figure 5 A illustrates such an example, where, for example, the imager's information included in the metadata, along with the competition name and contestant's name obtained as additional information, are displayed in a list as information corresponding to six images.

[0227] In addition, checkboxes 31 are arranged for each piece of information, so that the person in charge on the client terminal 3 can perform a selection operation.

[0228] In addition, a thumbnail request button 32 is provided so that the person in charge can operate it.

[0229] To enable the person in charge of client terminal 3 to first check only the information content corresponding to the image and roughly select the required image, such a UI screen is executed.

[0230] Figure 5 A indicates that the person in charge has selected checkboxes 31 for the first, second, and sixth information in the information content list 30.

[0231] Furthermore, when the thumbnail request button 32 is operated in this state, it is assumed that the person in charge requests thumbnail images of the images corresponding to the first, second, and sixth information.

[0232] This operation is performed when the person in charge confirms the metadata and additional information and attempts to use the thumbnails to confirm what type of image content they actually are.

[0233] In the PS2 processing, the CPU 71 of the client terminal 3 controls the display of such a UI screen and senses the input of the person in charge.

[0234] Note that, for example, thumbnails / screen pins are known to represent image content with a small amount of data at low resolution, but the term "thumbnail" in this disclosure broadly refers to an image that represents the main image (still image or moving image) using a still image with low resolution, such as an image commonly referred to as a thumbnail / screen pin.

[0235] -Communication TP3

[0236] The client terminal 3 sends a thumbnail image request list to the server device 2. That is, in response to the user's selection of checkbox 31 and operation of thumbnail request button 32 in the UI processing of PS2, the client terminal 3 generates a thumbnail image request list including a list of images corresponding to the selected information and information requesting thumbnails, and sends the thumbnail image request list.

[0237] -Communication TP4

[0238] Server device 2 will forward the list of thumbnail image requests received from client terminal 3 to imaging device 1.

[0239] -Communication TP5

[0240] In response to receiving a list of thumbnail image requests, imaging device 1 sends thumbnail images of one or more images listed in the list to server device 2.

[0241] -Communication TP6

[0242] Server device 2 will forward the thumbnail image received from imaging device 1 to client terminal 3.

[0243] - Handling PS3

[0244] Client terminal 3 uses the received thumbnail image to perform UI processing.

[0245] For example, client terminal 3 performs the following on display unit 77: Figure 5 The thumbnail list 34 shown in Figure B is displayed. That is, a list of received thumbnail images 38 is displayed.

[0246] Furthermore, as in the example in the figure, it is advantageous to display all or part of the meta-information and additional information about each thumbnail image 38 as associated / additional information 39.

[0247] In addition, checkboxes 31 are arranged for each thumbnail image 38, so that the person in charge on the client terminal 3 can perform a selection operation.

[0248] In addition, a main image request button 33 is displayed so that the person in charge can operate it.

[0249] This UI screen is displayed so that the person in charge of client terminal 3 can confirm the above information via thumbnail image 38. Figure 5 While roughly selecting the image content on screen A, the final image to be selected is also selected.

[0250] Figure 5 B shows the state where the person in charge has checked the checkbox 31 for the thumbnail image 38 in the center.

[0251] When the main image request button 33 is operated in this state, it is assumed that the person in charge requests the main image corresponding to the thumbnail image.

[0252] In the PS3, the CPU 71 of the client terminal 3 controls the display of such a UI screen and senses the input of the corresponding person in charge.

[0253] -Communication TP7

[0254] The client terminal 3 sends a main image request list to the server device 2. That is, in response to the user's selection of checkbox 31 and operation of the main image request button 33 in the UI processing of the PS3, the client terminal 3 generates a main image request list including a list of images corresponding to the selected thumbnails and information requesting the main image, and sends the main image request list.

[0255] -Communication TP8

[0256] Server device 2 will forward the list of main image requests received from client terminal 3 to imaging device 1.

[0257] -Communication TP9

[0258] In response to receiving the main image request list, imaging device 1 sends the main image of one or more images listed in the main image request list to server device 2.

[0259] -Communication TP10

[0260] Server device 2 will forward the main image received from imaging device 1 to client terminal 3.

[0261] therefore, Figure 5 The main image 35 in C is delivered to the client terminal 3.

[0262] As described above, instead of forwarding the main image 35 from the beginning, a coarse image selection based on metadata and additional information is performed first as the first round trip, followed by an image selection using the thumbnail image 38 as the second round trip. The relay processing up to this point is performed as the initial information forwarding processing by server device 2.

[0263] Then, server device 2 finally forwards the main image listed in the main image request list through the selection in the initial information forwarding process from imaging device 1 to client terminal 3 through main image forwarding processing.

[0264] In this way, the main image required by the person in charge of the client terminal 3 can be selected and forwarded, instead of forwarding all the large number of main images captured by the imaging device 1 to the client terminal 3, and the series of forwarding is very efficient.

[0265] The description will be used for implementation. Figure 4 The processing examples shown are for each communication TP and processing PS of imaging device 1 and server device 2.

[0266] As a process used for communication TP1, in Figure 6 The diagram illustrates the processing during imaging by the imaging device 1. The processing of the imaging device 1 mainly represents the control processing of the camera control unit 18.

[0267] exist Figure 6 In step S101, the camera control unit 18 of the imaging device 1 controls the capture process. That is, the camera control unit 18 controls... Figure 2 The camera control unit 18 requires units to perform image capture / recording according to the camera operator's operations. For example, the camera control unit 18 causes the imaging element unit 12 and the camera signal processing unit 13 to perform image signal processing of still images, generation of associated metadata, etc.

[0268] In step S102, the camera control unit 18 causes the camera signal processing unit 13 to perform encoding processing of the thumbnail image and the main image.

[0269] In step S103, the camera control unit 18 causes the recording control unit 14 to record the thumbnail image and the main image (including metadata) on the recording medium.

[0270] In step S104, the camera control unit 18 causes the communication unit 16 to perform processing to transmit the metadata of the image being imaged.

[0271] For example, the processing in step S104 is performed based on the camera operator's sending operation. For example, in response to the camera operator selecting a single image or image folder from the images that have been imaged and recorded on the recording medium for sending (delivering to the client terminal 3) and performing a sending operation, the camera control unit 18 performs processing. The camera control unit 18 causes the communication unit 16 to send the corresponding metadata of one or more images selected by the camera operator for sending.

[0272] Through the above processing, execution Figure 4 Communication TP1.

[0273] Figure 7 The image shows the use of Figure 4 The processing of PS1 and communication TP2 is handled by server device 2. Note that the control processing of control unit 2a is shown as part of the processing of server device 2.

[0274] In step S201, the control unit 2a checks the metadata received from the imaging device 1.

[0275] Then, when the metadata is received, the control unit 2a executes the loop process LP1.

[0276] In the loop processing LP1, steps S202 and S203 are performed for the metadata of each image sent from the imaging device 1. For example, if metadata for ten images is received, processing is performed on each image.

[0277] In step S202, the control unit 2a performs a DB search based on the metadata and obtains additional information. For example, as described above, for an image, when the imaging location and imaging time information are sent as metadata, the control unit 2a performs a DB search based on the imaging location and imaging time information and obtains additional information such as, for example, the competition name and the contestant's name.

[0278] In step S203, the control unit 2a performs the process of sending metadata and additional information about the image to the client terminal 3.

[0279] In the cyclic processing LP1, the control unit 2a performs the processing steps S202 and S203 for each image, so that the metadata and corresponding additional information sent by the imaging device 1 for each image are sent to the client terminal 3.

[0280] In response, the client terminal 3 performs UI processing as described above as processing PS2, and sends a list of thumbnail image requests as communication TP3.

[0281] Figure 8 The processing of server device 2 for communication TP4 based on the thumbnail image request list is shown.

[0282] In step S220, the control unit 2a of the server device 2 checks the receipt of the image request list. The image request list mentioned here is, for example, a thumbnail image request list or a main image request list. In the transmission mode described later, it may be information such as grouped image request information, requests for partial clippings of the main moving image, and requests for partial clippings of proxy moving images.

[0283] exist Figure 4 In the first sending mode, when a thumbnail image request list or a main image request list is received, the control unit 2a executes the processing of step S221.

[0284] In step S221, the control unit 2a performs the process of forwarding the received image request list to the imaging device 1. Therefore, as communication TP4, the thumbnail image request list is forwarded.

[0285] Figure 9 The image shows the processing of the imaging device 1 used for communication TP5.

[0286] In step S120, the camera control unit 18 checks the image request list received from the server device 2.

[0287] Then, when the image request list is received, the camera control unit 18 performs loop processing LP20.

[0288] In the loop processing LP20, the camera control unit 18 repeats the processing of steps S121 to S125 for the number of images related to the requests listed in the image request list.

[0289] First, in step S121, the camera control unit 18 determines whether the image request list is a thumbnail image request list or a main image request list, and branches the processing accordingly.

[0290] If the image request list is a thumbnail image request list, in step S122, the camera control unit 18 causes the recording control unit 14 to read a thumbnail image of one of the requested images from the recording medium.

[0291] Then, in step S123, the camera control unit 18 performs the process of sending the thumbnail image read from the recording medium to the server device 2 via the communication unit 16.

[0292] If the image request list is a main image request list, in step S124, the camera control unit 18 causes the recording control unit 14 to perform the reading of the main image, which includes one of the images in the request, from the recording medium.

[0293] Then, in step S125, the camera control unit 18 performs the process of sending the main image read from the recording medium to the server device 2 via the communication unit 16.

[0294] Perform the above processing for each image related to the requests listed in the image request list, and send the thumbnail image or main image related to the image request list.

[0295] Regarding communication TP5, thumbnail images are sent according to the thumbnail image request list.

[0296] Figure 10 The diagram illustrates the processing of server device 2 for communication TP6.

[0297] In step S230, the control unit 2a of the server device 2 verifies the image received from the imaging device 1. The image referred to here is, for example, a thumbnail image or the main image.

[0298] Then, when an image is received, the control unit 2a executes the loop processing LP10.

[0299] In the loop processing LP10, the control unit 2a repeats the processing of step S231 for the number of received images.

[0300] In step S231, the control unit 2a performs the process of sending the received image to the client terminal 3.

[0301] Therefore, as a communication TP6, when one or more thumbnail images are received from imaging device 1, server device 2 forwards them to client terminal 3.

[0302] In response, the client terminal 3 performs UI processing as described above as processing PS3, and sends the main image request list as communication TP7.

[0303] The processing of server device 2 for communication TP8 based on the main image request list is as described above. Figure 8 As shown in the image.

[0304] Furthermore, the processing of the imaging device 1 that receives the main image request list via communication TP8 is as follows: Figure 9 As shown in the diagram. That is, execution is performed on images related to requests listed in the main image request list. Figure 9 Steps S124 and S125 in the process are performed to send the main image in relation to the main image request list as communication TP9.

[0305] also, Figure 10 The diagram illustrates the processing of server device 2 for communication TP10. Upon receiving one or more main images from imaging device 1, server device 2 forwards them to client terminal 3 in step S231.

[0306] Note that the image request list sent to imaging device 1 in communication TP5 or TP7 can include transmission priority information. For example, when multiple images are requested, the person in charge can set the priority level. The order is then set in the image request list, and imaging device 1 executes image transmission according to the priority order. Therefore, highly needed images can be delivered as quickly as possible. This also applies to each transmission mode described below.

[0307] <4. Second Sending Mode>

[0308] The second communication mode will be described. Note that in the following text, the main differences between each communication mode and the communication modes described above will be described, and details of similar parts will be omitted.

[0309] Figure 11 The second transmission mode is shown.

[0310] Communication TP1 is similar to Figure 4 The communication TP1 in the imaging device 1 is used to send metadata from the imaging device 1 to the server device 2.

[0311] However, in the second transmission mode, the server device 2 transmits the received metadata to the client terminal 3 as is in communication TP2, without exporting additional information as part of the processing PS1 at this time.

[0312] Client terminal 3 performs UI processing as processing PS2, but in this case, since no additional information is received, the displayed content is only based on... Figure 12 The metadata shown in A.

[0313] For example, when the imager, image location, and time are sent as metadata, the information of the camera operator (imager), image location, and image time corresponding to each image is displayed as an information content list 30.

[0314] In addition, checkboxes 31 and thumbnail request buttons 32 corresponding to each piece of information are displayed so that the person in charge can take action.

[0315] In communication TP3, according to the Figure 12 The operation of screen A involves sending a list of thumbnail image requests from client terminal 3.

[0316] In communication TP4, server device 2 forwards the thumbnail image request list to imaging device 1.

[0317] In communication TP5, imaging device 1 sends the thumbnail image requested in the thumbnail image request list.

[0318] In the second transmission mode, server device 2, which has already received the thumbnail image as associated information of the captured main image via communication TP5, performs processing to obtain additional information, as described in processing PS4 at this time. In this case, image analysis and object recognition processing are performed on the thumbnail image, and additional information is generated based on the results. For example, the contestant's name can be obtained by performing facial recognition, personal identification, and uniform number recognition. Furthermore, the competition type can be determined by object recognition of the subject, and the competition name can be generated as additional information. Based on these examples, certain information is obtained through thumbnail image recognition processing, and the obtained information is set as additional information.

[0319] In communication TP6, server device 2 sends a thumbnail image and additional information determined based on the thumbnail image to client terminal 3.

[0320] Client terminal 3 uses the received thumbnail image to perform UI processing as a processing PS3. In this case, since additional information is also received, client terminal 3 can also perform display based on the additional information along with the thumbnail image.

[0321] For example, client terminal 3 displays the received thumbnail image 38 on display unit 77, as... Figure 12 The thumbnail list 34 shown in Figure B is displayed. Additionally, associated / additional information 39 is displayed, as all or part of the meta-information and additional information about each thumbnail image 38. In this example, the competition name and contestant name, obtained as additional information, are displayed together with the imager information obtained as meta-information.

[0322] In addition, checkboxes 31 corresponding to each thumbnail image 38 and a main image request button 33 are displayed so that the person in charge can operate them.

[0323] In the TP7 communication system, according to the... Figure 12 The operation of screen B involves sending the main image request list from client terminal 3.

[0324] In communication TP8, server device 2 forwards the main image request list to imaging device 1.

[0325] In communication TP9, imaging device 1 sends the main image requested in the main image request list.

[0326] In communication TP10, server device 2 will be as follows: Figure 12 The main image shown in C is forwarded to client terminal 3.

[0327] As mentioned above, Figure 11 The second sending mode in Figure 4 The difference in the first sending mode is that, as a process for PS4, the processing of additional information obtained by server device 2 is based on thumbnail images.

[0328] The processing of the imaging device 1 and server device 2 for the second transmission mode is as follows.

[0329] The processing of the imaging device 1 for communication TP1 is as described above. Figure 6 As shown in the image.

[0330] Figure 13 The diagram illustrates the processing of server device 2 for communication TP2.

[0331] In step S201, the control unit 2a checks the metadata received from the imaging device 1.

[0332] Then, when the metadata is received, the control unit 2a executes the loop process LP2.

[0333] In the cyclic processing LP2, in step S205, the control unit 2a performs the process of sending the metadata sent from the imaging device 1 to the client terminal 3.

[0334] In the loop processing LP2, the control unit 2a performs the processing step S205 on each image, so that the metadata sent by the imaging device 1 for each image is sent to the client terminal 3.

[0335] In response, the client terminal 3 performs UI processing as described above as processing PS2, and sends a list of thumbnail image requests as communication TP3.

[0336] The processing of server device 2 for communication TP4 based on the thumbnail image request list is as described above. Figure 8 As shown in the image.

[0337] Furthermore, the processing of the imaging device 1 used for communication TP5 is as described above. Figure 9 As shown in the image.

[0338] When thumbnail image 38 is received in communication TP5, the processing of server device 2 for communication TP6 is as follows: Figure 14 As shown in the image.

[0339] In step S230, the control unit 2a of the server device 2 verifies the image received from the imaging device 1. The image referred to here is, for example, a thumbnail image or the main image.

[0340] In step S240, the control unit 2a branches the processing according to whether the received image is a thumbnail image or the main image.

[0341] Then, when a thumbnail image is received, the control unit 2a executes loop processing LP11, and when the main image is received, loop processing LP11 is executed again.

[0342] In the loop processing LP11, the control unit 2a repeats the processing steps S241, S242 and S243 according to the number of received thumbnail images.

[0343] In step S241, the control unit 2a performs image recognition processing. Then, in step S242, additional information is generated based on the result of the image recognition processing.

[0344] Note that in this case, a DB search can be performed based on the image recognition results to generate additional information. Furthermore, both additional information based on the image recognition results and additional information obtained as DB search results based on metadata can be exported.

[0345] In step S243, the control unit 2a performs the process of sending the received image to the client terminal 3.

[0346] Therefore, as a communication TP6, when one or more thumbnail images are received from imaging device 1, server device 2 forwards them and additional information to client terminal 3.

[0347] In response, the client terminal 3 performs UI processing as described above as processing PS3, and sends the main image request list as communication TP7.

[0348] The processing of server device 2 for communication TP8 based on the main image request list is as described above. Figure 8 As shown in the image.

[0349] Furthermore, the processing of imaging device 1, which has already received the main image request list via communication TP8, is as follows: Figure 9 As shown in the diagram. That is, the process is performed on images related to the requests listed in the main image request table. Figure 9 Steps S124 and S125 in the process are performed to send the main image related to the image request list as communication TP9.

[0350] also, Figure 14 The diagram illustrates the processing of server device 2 for communication TP10. When server device 2 receives one or more main images from imaging device 1, the processing... Figure 14 Step S240 proceeds to the loop processing LP12.

[0351] In the loop processing LP12, the control unit 2a repeats the processing of step S244 for the number of received main images.

[0352] In step S244, the control unit 2a performs the process of sending the received main image to the client terminal 3.

[0353] Therefore, as a communication TP10, when one or more main images are received from imaging device 1, server device 2 forwards them to client terminal 3.

[0354] <5. Third and Fourth Sending Modes>

[0355] The description will be an example of how the third and fourth sending modes do not perform the acquisition / forwarding of additional information.

[0356] Figure 15 The third transmission mode is shown.

[0357] As a communication TP1, metadata is sent from imaging device 1 to server device 2.

[0358] As a communication TP2, server device 2 sends the received metadata to client terminal 3 as is.

[0359] Client terminal 3 performs UI processing as processing PS2, but in this case, since no additional information is received, the displayed content is only based on... Figure 16 The metadata shown in A. For example, the content is similar to... Figure 12 The content of A.

[0360] In communication TP3, according to the Figure 16 The operation of screen A involves sending a list of thumbnail image requests from client terminal 3.

[0361] In communication TP4, server device 2 forwards the thumbnail image request list to imaging device 1.

[0362] In communication TP5, imaging device 1 sends the thumbnail image requested in the thumbnail image request list.

[0363] In communication TP6, server device 2 sends thumbnail images to client terminal 3.

[0364] Client terminal 3 uses the received thumbnail image to perform UI processing as processing PS3. For example, client terminal 3 displays the received thumbnail image 38 on display unit 77 as... Figure 16 The thumbnail list 34 shown in B is displayed. In this case, since no additional information is sent, only information based on meta-information (e.g., imaging location, imaging time, etc.) is displayed as associated / additional information 39.

[0365] In addition, checkboxes 31 corresponding to each thumbnail image 38 and a main image request button 33 are displayed so that the person in charge can operate them.

[0366] In the TP7 communication system, according to the... Figure 16 The operation of screen B involves sending the main image request list from client terminal 3.

[0367] In communication TP8, server device 2 forwards the main image request list to imaging device 1.

[0368] In communication TP9, imaging device 1 sends the main image requested in the main image request list.

[0369] In communication TP10, server device 2 will be as follows: Figure 16 The main image shown in C is forwarded to client terminal 3.

[0370] As mentioned above, Figure 15 The third sending mode differs from the first and second sending modes mentioned above in that the server device 2 does not perform the processing of obtaining additional information.

[0371] The processing of imaging device 1 and server device 2 for the third transmission mode is as follows.

[0372] The processing of imaging device 1 during imaging is as follows: Figure 6 As shown, and the processing when receiving an image request is as follows: Figure 9 As shown in the image.

[0373] The processing of metadata received by server device 2 is as follows: Figure 13 As shown, the processing when receiving an image request is as follows: Figure 8As shown, and the processing when receiving images is as follows: Figure 10 As shown in the image.

[0374] Next, the fourth transmission mode will be described.

[0375] The whole process is similar to Figure 15 The process of the third sending mode is described, but it is an example in the case where the imaging device 1 has a personal facial registration function.

[0376] When imaging device 1 registers, for example, the personal face, team affiliation, name, etc. of a contestant in a competition, it can identify the contestant who has been identified as the subject through image recognition and can include this information in the metadata.

[0377] Therefore, imaging device 1 performs, for example Figure 17 The processing.

[0378] As a processing component of the camera control unit 18, Figure 17 The processing of steps S101, S102, S103, and S104 in the process and Figure 6 The processing is similar to that in [the text].

[0379] As Figure 17 In step S180, the camera control unit 18 performs personal face recognition processing. For example, personal face recognition processing is performed on each image recorded in step S103 to identify the contestant as the subject and to add that information to the metadata.

[0380] Therefore, when sending meta-information in step S104, the contestant information is also included in the meta-information.

[0381] Therefore, in passing Figure 15 When communication TP1 and communication TP2 send metadata to client terminal 3, client terminal 3 can display, based on the metadata, such as... Figure 16 The information content list 30 shown in D indicates that it can display the player's information.

[0382] Therefore, this is very useful, for example, when the person in charge wants to select images of specific players.

[0383] <6. Fifth Sending Mode>

[0384] Reference Figure 18 Describe the fifth communication mode.

[0385] In communication TP1, metadata including audio memo data is sent from imaging device 1 to server device 2.

[0386] Audio memo data is data in which, when an image is captured (e.g., immediately after a release operation) or recorded on a recording medium, a utterance is emitted as a memo to associate the audio with the image, and the audio is saved as metadata. This fifth communication mode can be applied to situations where the imaging device 1 has an audio memo function.

[0387] When receiving metadata including audio memo data, server device 2 sends it to client terminal 3 as communication TP2.

[0388] Client terminal 3 performs UI processing as processing PS2, but in this case, the displayed content is only based on... Figure 19 The metadata content list 30 shown in Figure A. However, since metadata including audio memo data is received, an audio playback icon 40 is displayed for the metadata content including audio memo data. In the example in the figure, as a case where audio memo data exists for all images, an audio playback icon 40 is displayed for each metadata content.

[0389] For example, in client terminal 3, audio output unit 78 reproduces an audio memo by operating audio reproduction icon 40. When the camera operator appropriately records the audio memo, the person in charge of client terminal 3 can refer to the audio memo to perform image selection.

[0390] Since subsequent communication TP3 to communication TP10 and processing PS3 are similar to the third communication mode, redundant descriptions are omitted.

[0391] Processing of imaging devices capable of recording audio memo data during imaging, such as Figure 20 As shown in the image.

[0392] Notice, Figure 20 Steps S101, S102, S103, and S104 in the process are similar to... Figure 6 The steps are similar.

[0393] The camera operator can arbitrarily add audio memos to the captured images.

[0394] If the camera operator does not perform the operation of recording an audio memo on the captured image, the process proceeds from step S110 to step S104, and... Figure 6 similar.

[0395] When performing the operation of recording an audio memo... Figure 20The processing proceeds from step S110 to step S111, and the camera control unit 18 performs recording processing. That is, it is, for example, processing of an audio recording period of approximately 10 to 20 seconds from the audio input unit 25 according to a predetermined operation.

[0396] In step S112, the camera control unit 18 causes the camera signal processing unit 13 to perform encoding processing of the input audio memo data.

[0397] In step S113, the camera control unit 18 causes the recording control unit 14 to perform the process of recording encoded audio memo data onto the recording medium.

[0398] Then, in step S114, the camera control unit 18 causes the communication unit 16 to perform the transmission process to the server device 2 while the audio memo data is added to the image's metadata.

[0399] When metadata including audio memo data is received, the server device 2 used for communication TP2 processes it as follows: Figure 21 As shown in the image.

[0400] In step S201, the control unit 2a checks the metadata received from the imaging device 1.

[0401] Then, when the metadata is received, the control unit 2a executes the loop process LP3.

[0402] As part of the loop process LP3, in step S206, the control unit 2a determines whether the audio memo data is included in the metadata and branches the processing.

[0403] When the metadata does not include audio memo data, in step S205, the control unit 2a performs the process of sending the metadata sent from the imaging device 1 to the client terminal 3.

[0404] When the metadata includes audio memo data, in step S207, the control unit 2a performs the process of sending metadata including audio memo data sent from the imaging device 1 to the client terminal 3.

[0405] In the loop processing LP3, the control unit 2a performs the processing steps S206 and S205 or steps S206 and S207 for each image, so that the metadata sent by the imaging device 1 for each image is sent to the client terminal 3, and when audio memo data is added, the audio memo data is also sent.

[0406] Another process for the fifth transmission mode is described below.

[0407] The processing of the imaging device 1 when receiving an image request is as follows: Figure 9 As shown in the image.

[0408] The processing of server device 2 when receiving image requests is as follows: Figure 8 As shown, and the processing when receiving images is as follows: Figure 10 As shown in the image.

[0409] <7. Sixth Sending Mode>

[0410] Reference Figure 22 Describe the sixth communication mode.

[0411] In communication TP1, metadata including audio memo data is sent from imaging device 1 to server device 2.

[0412] Upon receiving metadata including audio memo data, server device 2 performs audio memo transcription processing as processing PS5. That is, it generates information that transcribes the audio memo data into text data. Then, as communication TP2, it sends the metadata and text information based on the audio memo data to client terminal 3.

[0413] Client terminal 3 performs UI processing as processing PS2, and in this case, as display content, such as Figure 23 As shown in A, the audio memo text 41 obtained by converting audio memo data into text can be displayed as an information content list 30 along with the content of the metadata.

[0414] Note that, in the sense that the text information is generated in server device 2, the text information can be regarded as one of the additional information referred to in this disclosure. However, since the text information is identical in content to the audio memo data, it can be regarded as the related information referred to in this disclosure.

[0415] By displaying the audio memo text 41, the person in charge of client terminal 3 can refer to the content of the audio memo to perform image selection. Since it is not necessary to reproduce the audio memo at this time, the selection operation can be performed more efficiently.

[0416] The subsequent communication TP3 to TP10 and processing PS3 are similar to the third communication mode.

[0417] The processing of imaging devices capable of recording audio memo data during imaging is as described above. Figure 20 As shown in the image.

[0418] When metadata including audio memo data is received, the server device 2 used for communication TP2 processes it as follows: Figure 24 As shown in the image.

[0419] In step S201, the control unit 2a checks the metadata received from the imaging device 1.

[0420] Then, when the metadata is received, the control unit 2a executes the loop process LP4.

[0421] As part of the loop process LP4, in step S206, the control unit 2a determines whether the audio memo data is included in the metadata and branches the processing.

[0422] When the metadata does not include audio memo data, in step S205, the control unit 2a performs the process of sending the metadata sent from the imaging device 1 to the client terminal 3.

[0423] When the metadata includes audio memo data, in step S208, the control unit 2a performs text conversion processing on the audio memo data. Then, in step S209, the control unit 2a performs processing to send the text data and other metadata of the audio memo data sent from the imaging device 1 to the client terminal 3.

[0424] Note that at this point, you can reduce the amount of data sent by not sending the audio memo data itself.

[0425] However, the original audio memo data can be sent together with the text data of the audio memo data to the client terminal 3, so that audio playback can be performed on the client terminal 3 side.

[0426] In the cyclic processing LP4, the control unit 2a performs steps S206 and S205 or steps S206, S208 and S209 on each image, thereby sending the metadata sent by the imaging device 1 for each image to the client terminal 3, and when audio memo data is added, converting the audio memo data into text and sending it.

[0427] Another process for the sixth transmission mode is described below.

[0428] The processing of the imaging device 1 when receiving an image request is as follows: Figure 9 As shown in the image.

[0429] The processing of server device 2 when receiving image requests is as follows: Figure 8 As shown, and the processing when receiving images is as follows: Figure 10 As shown in the image.

[0430] <8. Seventh Sending Mode>

[0431] exist Figure 25 In the seventh transmission mode, an example will be described where the imaging device performs continuous imaging and manages a series of images as grouped images.

[0432] -Communication TP21

[0433] Imaging device 1 sends the metadata of a thumbnail image of the first image, which includes grouped images of continuous imaging, as associated information about the captured image to server device 2.

[0434] In this case, imaging device 1 sends a thumbnail image of the first image for each of one or more consecutively imaged grouped images, but is not limited to the first one, and it is sufficient if a specific image is selected to be included in the consecutively imaged grouped images and a thumbnail image of that image is sent.

[0435] -Communication TP22

[0436] Server device 2 sends metadata to client terminal 3, including thumbnail images of each group of images in a continuous imaging sequence.

[0437] - Handling PS6

[0438] Client terminal 3 uses thumbnail images of each group of images received in succession to perform UI processing.

[0439] For example, client terminal 3 displays as follows Figure 26 The thumbnail list 34 shown in A. In this case, for a group of images of sequential imaging, thumbnail images of the group of images of sequential imaging are displayed (thumbnail images of sequential imaging 42).

[0440] In addition, if the content sent includes thumbnail images of images obtained through normal single-image capture that is not continuous imaging, thumbnail image 38 is displayed.

[0441] In addition, corresponding to the thumbnail image 42 or thumbnail image 38 of continuous imaging, information such as the camera operator (imager), imaging time, imaging location, etc. are displayed as associated / additional information 39 based on meta-information.

[0442] In addition, checkboxes 31 are arranged corresponding to the thumbnail images 42 or 38 of the continuous imaging, so that the person in charge on the client terminal 3 side can perform a selection operation.

[0443] In addition, the main image request button 33 and the expand button 36 are displayed so that the person in charge can operate them.

[0444] This UI is designed so that the person in charge of client terminal 3 can select the requested main image based on the thumbnail image, and can select individual images from a group of continuously imaged images.

[0445] For example, by selecting the thumbnail image 42 of the continuous imaging using checkbox 31 and operating the expand button 36, one can... Figure 26 See Figure B for thumbnail images of the individual images that make up the continuous imaging.

[0446] -Communication TP23

[0447] The client terminal 3 sends a request for the entire group of images to the server device 2. That is, in the UI processing of PS2, the person in charge checks the checkbox 31 for the thumbnail image 42 of a specific continuous imaging, operates the expand button 36, and in response, the client terminal 3 generates a list of all selected images of the group of continuous imaging and information requesting the entire group of images, including the information requesting the thumbnails, and sends the generated information.

[0448] -Communication TP24

[0449] Server device 2 will forward the request for the entire group of images received from client terminal 3 to imaging device 1.

[0450] -Communication TP25

[0451] In response to receiving a request for the entire group of images, imaging device 1 sends thumbnail images of each image included in the requested group of images to server device 2.

[0452] -Communication TP26

[0453] Server device 2 will forward the thumbnail image received from imaging device 1 to client terminal 3.

[0454] - Handling PS7

[0455] Client terminal 3 uses all thumbnail images of the received grouped images to perform UI processing.

[0456] For example, as such Figure 26 As shown in the expanded list 37 in B, the client terminal 3 displays each thumbnail image 43 of the grouped images of continuous imaging on the display unit 77.

[0457] In addition, associated / additional information 39 can be displayed in correspondence with the grouped images of the entire continuous imaging process.

[0458] In addition, checkboxes 31 are arranged corresponding to each thumbnail image 43, so that the person in charge on the client terminal 3 side can perform a selection operation.

[0459] In addition, a main image request button 33 is displayed so that the person in charge can operate it.

[0460] Using this UI, the person in charge of client terminal 3 can also select some images from the continuous imaging images.

[0461] -Communication TP27

[0462] The client terminal 3 sends the main image request list to the server device 2. That is, in response to the user's selection of checkbox 31 and operation of the main image request button 33 in the UI processing of PS6 or PS7, the client terminal 3 generates a main image request list including a list of images corresponding to the selected thumbnails and information on requesting the main image, and sends the main image request list.

[0463] -Communication TP28

[0464] Server device 2 will forward the list of main image requests received from client terminal 3 to imaging device 1.

[0465] -Communication TP29

[0466] In response to receiving the main image request list, imaging device 1 sends the main image of one or more images listed in the main image request table to server device 2.

[0467] -Communication TP30

[0468] Server device 2 will forward the main image received from imaging device 1 to client terminal 3.

[0469] Therefore, Figure 26 The main image 35 in C is delivered to the client terminal 3.

[0470] As described above, assuming a group of images captured in successive imaging, a thumbnail image representing the group of images captured in successive imaging is transmitted, allowing selection of the group of images captured in successive imaging. Furthermore, individual images within the group of images captured in successive imaging can be selected.

[0471] In this way, instead of forwarding all the large number of main images continuously captured by imaging device 1 to client terminal 3, the main images of the group of images of continuous imaging required by the person in charge of client terminal 3 can be selected and forwarded, and the series of forwarding is very efficient.

[0472] Notice, Figure 25 The diagram illustrates communication TP21 to communication TP30 assuming continuous imaging is performed by imaging device 1. However, when imaging device 1 does not perform continuous imaging and capture, for example, communication TP1 to communication TP10 is performed by any of the first to sixth transmission modes described above.

[0473] Furthermore, even when performing continuous imaging, it is conceivable, for example, in Figure 25 The communication TP21 was executed at a time point prior to this. Figure 4 The diagram shows communication TP1, processing PS1, communication TP2, processing PS2, communication TP3, and communication TP4, and executes the process for images requested by the thumbnail image request list from communication TP4. Figure 25 The processing after TP21 communication.

[0474] As for the processing of the seventh transmission mode during imaging by imaging device 1, one can conceive of... Figure 27 Examples.

[0475] In step S140, the camera control unit 18 branches the processing based on whether continuous imaging is performed (whether a continuous imaging mode is set).

[0476] When imaging is not continuous, the camera control unit 18 performs the processing from steps S101 to S104. This is similar to... Figure 6 The processing in the process.

[0477] During continuous imaging, as a loop process LP30, the camera control unit 18 repeats the processing from steps S141 to S144 the same number of times as the number of image captures performed through continuous imaging.

[0478] In step S141, the camera control unit 18 controls the individual image capture processing in continuous imaging. For example, the camera control unit 18 causes the imaging element unit 12 and the camera signal processing unit 13 to perform image signal processing for still images, etc.

[0479] In step S142, the camera control unit 18 causes the camera signal processing unit 13 to perform encoding processing of the thumbnail image and the main image.

[0480] In step S143, the camera control unit 18 causes the recording control unit 14 to record the thumbnail image and the main image on the recording medium.

[0481] In step S144, the camera control unit 18 updates the group information management file. This is a process that updates the metadata associated with the group images being imaged sequentially, for the purpose of managing the file for sequentially imaged group images.

[0482] After performing the above processing on a number of consecutively imaged images, the camera control unit 18 proceeds to step S145 and causes the communication unit 16 to perform processing to send metadata of a thumbnail image of a first image that includes the grouped images of the consecutively imaged images.

[0483] Through the above processing, execution Figure 4 Communication TP1.

[0484] The processing of the imaging device 1 when receiving an image request is as follows: Figure 28 As shown in the image.

[0485] It is used when receiving a request for a thumbnail image of the entire packet image in communication TP25, when receiving a request list of the main image in communication TP28, and in... Figure 4 The processing of receiving a list of thumbnail images in TP5 communication.

[0486] In step S120, the camera control unit 18 checks the image request list received from the server device 2.

[0487] Then, when an image request list is received, the camera control unit 18 performs loop processing LP21.

[0488] In the loop processing LP21, the camera control unit 18 repeats the processing described below for the number of images related to the requests listed in the image request list.

[0489] First, in step S131, the camera control unit 18 determines whether the image request list is a thumbnail image request list.

[0490] In the case that the image request list is a thumbnail image request list, in step S122, the camera control unit 18 causes the recording control unit 14 to perform the reading of a thumbnail of one of the requested images from the recording medium.

[0491] Then, in step S123, the camera control unit 18 performs the process of sending the thumbnail image read from the recording medium to the server device 2 via the communication unit 16.

[0492] Repeat this operation for the number of images requested.

[0493] If the received request information is not a thumbnail image request list, in step S132, the camera control unit 18 determines whether the image request list is a main image request list. If it is a main image request list, in step S124, the camera control unit 18 causes the recording control unit 14 to read a main image included in the request from the recording medium.

[0494] Then, in step S125, the camera control unit 18 performs the process of sending the main image read from the recording medium to the server device 2 via the communication unit 16.

[0495] Repeat this operation for the number of images requested.

[0496] If the received request information is neither a thumbnail image request list nor a main image request list, in step S133, the camera control unit 18 determines whether it is a request for the entire group of images. In this case, the camera control unit 18 executes loop processing LP22.

[0497] In the loop processing LP22, steps S126 and S127 are repeated according to the number of images constituting the corresponding grouped images of the sequential imaging. Specifically, since a thumbnail image of the first image has already been sent, repeating the processing for the second and subsequent images is sufficient.

[0498] In step S126, the recording control unit 14 performs the reading of thumbnail images of untransmitted images from the group of images of continuous imaging from the recording medium.

[0499] Then, in step S127, the camera control unit 18 performs the process of sending the thumbnail image read from the recording medium to the server device 2 via the communication unit 16.

[0500] When the above processing is performed, the thumbnail image or main image associated with the image request list is sent.

[0501] Image transmission is performed as communication TP25, communication TP29 (or communication TP9) and communication TP5.

[0502] Note that the processing of received metadata by server device 2 is as follows: Figure 13 As shown, the processing when receiving an image request is as follows: Figure 8 As shown, and the processing when receiving images is as follows: Figure 10 As shown in the image.

[0503] <9. Switching Processing Example I>

[0504] Although the above primarily assumes still images as the basis for describing the first to seventh transmission modes, server device 2 can appropriately switch these transmission modes to achieve suitable communication depending on the situation or the object being imaged.

[0505] Examples of switching described below include, but are not limited to, switching the transmission mode itself, such as switching between a first transmission mode and a fourth transmission mode, switching between a fifth transmission mode and a sixth transmission mode as needed, and automatically switching from a first transmission mode to a seventh transmission mode, for example, when the imaging device is performing continuous imaging.

[0506] For example, in the first sending mode, the initial information forwarding process is performed twice, such that in the first round trip, meta-information and additional information are sent to perform a selection in the client terminal 3, and in the second round trip, a thumbnail image is sent to perform a selection in the client 3. However, there are examples where the main image forwarding process is performed only after one round trip, and examples where the main image forwarding process is performed from the beginning without performing the initial information forwarding process. As examples, there are also examples of switching between reducing or increasing the sending mode.

[0507] The following description will focus primarily on the switching of transmission modes, but it should be understood that it also applies to the switching of processes within a transmission mode.

[0508] Figure 29 Example I of the switching process is shown.

[0509] -Processing PS10

[0510] Server device 2 determines the execution status and, based on that status, determines the sending mode to be executed. Note that, as mentioned above, it is also possible to determine to change the process in a specific sending mode (e.g., omitting the initial information forwarding process, performing the initial information forwarding process only once, etc.).

[0511] -Communication TP40

[0512] Server device 2 notifies imaging device 1 of the determined transmission mode.

[0513] -Processing PS11

[0514] Imaging device 1 performs setting processing for image delivery according to the notified transmission mode.

[0515] For example, the camera control unit 18 performs the following settings. In the first to fourth transmission modes, only metadata is set to be transmitted during image forwarding. In the fifth and sixth transmission modes, metadata including audio memo data is set to be transmitted during image forwarding. In the seventh transmission mode, settings are made to transmit a thumbnail image of the first image of a group of images in continuous imaging.

[0516] Alternative locations may exist where settings are made to omit the initial message forwarding process in a specific sending mode.

[0517] -Communication TP41

[0518] Server device 2 notifies client terminal 3 of the determined sending mode.

[0519] -Processing PS12

[0520] Client terminal 3 performs UI processing that represents the notified sending mode according to the notified sending mode.

[0521] Note that the order of communication TP40 and communication TP41 described above can be reversed. Furthermore, it is conceivable that one or both of communication TP40 and communication TP41, along with their corresponding processes PS11 and PS12, may not be executed.

[0522] Figure 30 An example of the processing of server device 2 for performing such switching processing is shown.

[0523] In step S300, the control unit 2a performs a status determination. For example, it determines conditions such as the communication status (such as the current forwarding rate and packet error rate), the number of connected imaging devices 1, and the processing load of the server device 2 itself.

[0524] In step S301, control unit 2a branches the processing based on whether a switch needs to be performed according to the result of the situation determination. If no switch is performed, control unit 2a terminates the process. Figure 30 However, in the case of a switch, the process proceeds to step S302.

[0525] In step S302, the control unit 2a determines the transmission mode based on the determined situation. For example, the following examples can be conceived, in which a first transmission mode is set when the communication status with the imaging device 1 is good, and a third transmission mode is set when the communication status deteriorates; a first transmission mode or a second transmission mode is set when the processing load of the server device 2 itself is light, and a third transmission mode is set when the processing load is heavy, etc.

[0526] In step S303, the control unit 2a performs the process of notifying the imaging device 1 of the transmission mode, and in step S304, it performs the process of notifying the client terminal 3 of the transmission mode.

[0527] When switching transmission modes in such processing, the appropriate transmission mode is selected based on the communication status, the status of the processing load, etc.

[0528] The description will provide specific examples of how to switch over various situations.

[0529] First, there is a determination regarding timing.

[0530] - For the first and second sending modes, server device 2 needs to generate additional information. Furthermore, in the sixth sending mode, text conversion processing is required. Therefore, when no information for generating additional information is created, and when DB access is not possible, or when text conversion processing is not possible, the third, fourth, and fifth sending modes are set; and when additional information can be generated, the first, second, and sixth sending modes are set.

[0531] We can also consider determining the processing load of server device 2.

[0532] When requesting image lists from a large number of client terminals 3, it is desirable to reduce the amount of data sent in order to lower the network load on server device 2. For example, it is desirable to send text data instead of audio data. Therefore, it is conceivable to switch to a third sending mode when the network load on server device 2 is high, and to switch to a fifth sending mode when the network load on server device 2 is low.

[0533] Furthermore, when receiving images from a large number of cameras, it is desirable to reduce the amount of data to be transmitted in order to reduce the network load on server device 2. For example, in the case of continuous image imaging, it is desirable to reduce the amount of data by transmitting the first image. Therefore, it is conceivable to switch to the seventh transmission mode when the network load on server device 2 is high, and to switch to the third transmission mode when the network load on server device 2 is low.

[0534] Furthermore, when receiving a large amount of metadata from the imaging device 1 side, the data creation processing load on the server device 2 side increases. Therefore, it is conceivable to switch the processing to the third and fourth transmission modes when the interface processing or CPU 71 processing load on the server device 2 side is high, and to switch the processing to the first, second, and fifth transmission modes when the interface processing or CPU 71 processing load on the server device 2 side is low.

[0535] Another approach is to determine the situation from the perspective of use cases.

[0536] The environment of image capture sites differs between outdoor sports, where network conditions may be poor, and indoor / stadium sports, where network conditions are abundant. For example, one might consider switching processing to a third transmission mode in situations with poor network conditions, and to a fifth transmission mode in situations with abundant network conditions.

[0537] Furthermore, in poor network conditions, the transmission described in the first to sixth transmission modes is performed, such as metadata transmission, thumbnail image transmission, and main image transmission. However, in rich network conditions, it is conceivable to reduce the transmission stages by performing thumbnail image transmission and main image transmission (i.e., performing one round trip of the initial information forwarding process) or by performing main image transmission from the beginning (without performing the initial information forwarding process).

[0538] There are sports like track and field or swimming where databases of time, location, and athletes can be created based on the competition schedule; and there are sports like marathons and triathlons where databases of time, location, and athletes cannot be created based on the competition schedule. It is conceivable to use the first sending mode when the subject activity is a sport where a database can be created, and to switch to the third, fourth, fifth, and sixth sending modes when the subject activity is not a sport where a database can be created.

[0539] Note that the process of switching the transmission mode according to the type of sports to be imaged will be described later as examples II and III of the switching process.

[0540] The determination of the situation also took into account the perspective of 5G or 4G environments.

[0541] The goal is to reduce the amount of data sent based on the network environment. Therefore, an example could be conceived where, in a 4G environment, the system switches between the third and fourth transmission modes, and in a 5G environment, it switches between the first, second, fifth, and sixth transmission modes.

[0542] At the same time, Figure 29 and Figure 30 In the example, server device 2 sends a pattern notification to client terminal 3, which enables client terminal 3 to display, for example... Figure 31 A and Figure 31 The mode information 44 shown in B can be presented to the person in charge as to which sending mode will be used for image forwarding. Figure 31 A is an example of the displayed state before PS12 forwarding begins, and Figure 31 B is, for example Figure 16 An example of handling display status in PS2, etc.

[0543] As pattern information 44, for example, such as Figure 31 As shown in each example in C, one can imagine the content and order of the information sent to client terminal 3.

[0544] In addition, Figure 29 and Figure 30In the example, server device 2 automatically performs the switchover through situation determination processing, but it can request permission from the person in charge of client terminal 3 to switchover.

[0545] For example, such as Figure 32 As shown, a confirmation dialog box 45 is displayed on client terminal 3, and the sending mode is switched when the person in charge accepts the switch. This figure is an example of requesting approval from the person in charge when performing a switch based on the network environment.

[0546] For this reason, server device 2, for example, in Figure 30 In step S301, the following is performed: Figure 33 The processing shown.

[0547] In step S310, control unit 2a performs network status determination processing.

[0548] For example, when it is determined that the forwarding rate has decreased and information forwarding has reduced, the control unit 2a proceeds from step S311 to step S313 and sends a handover recommendation notification to the client terminal 3. In response, the client terminal 3 displays as follows: Figure 32 The confirmation dialog box 45 is shown. Then, in step S314, a response notification is awaited from the client terminal 3.

[0549] When the person in charge performs the operation to accept the switch and receives notification of this intention from client terminal 3, the process from... Figure 30 Step S314 proceeds to steps S301 and S302, and the transmission mode is switched.

[0550] When the person in charge performs an operation to reject the switch and provides a notification of this intention from client terminal 3, the process from... Figure 30 In step S314, the process proceeds to step S301 without switching the transmission mode, and... Figure 30 The processing is now complete.

[0551] When the network state determination process in step S310 determines, for example, that the forwarding rate has increased and the state of high-speed information forwarding has been restored, the control unit 2a proceeds from step S312 to step S315 and sends a switching recommendation notification to the client terminal 3. In response, the client terminal 3 displays a confirmation dialog box 45 prompting the user to switch to the transmission mode under good communication conditions. Then, in step S316, the control unit 2a waits for a response notification from the client terminal 3.

[0552] When the person in charge performs the operation to accept the switch and receives notification of this intention from client terminal 3, the process from... Figure 30 Step S316 proceeds to steps S301 and S302, and the sending mode is switched.

[0553] When the person in charge performs an operation to reject the switch and provides a notification of this intention from client terminal 3, the process from... Figure 30 In step S316, the process proceeds to step S301 without switching the transmission mode, and... Figure 30 The processing is now complete.

[0554] In this way, the sending mode switching process can be performed based on the determined results of the situation and the intention of the person in charge of the client terminal 3.

[0555] As a result, inappropriate switching can be prevented based on the situation on the three sides of the client terminal.

[0556] <10. Switching Processing Example II>

[0557] Reference Figure 34 Description of switching process example II.

[0558] First, as a process for PS20, for example, the sports information (competition type information) of the imaging object is input through the operation of the camera operator, and the competition type is included in the imaging device 1 as meta information.

[0559] As a communication TP50, the imaging device 1 sends metadata including sports information of the imaged object to the server device 2 at a certain time. For example, the transmission time may immediately follow processing PS20, or it may be during the initial transmission of metadata in a first transmission mode, etc.

[0560] As a processor of PS21, server device 2 determines the transmission mode based on the sports information included in the received metadata.

[0561] Subsequently, notifications setting the transmission modes of communication TP40 and communication TP41 to be executed are performed similarly to those in Example I of the switching process described above. Similarly, processing PS11 in imaging device 1 and processing PS12 in client terminal 3 are also executed.

[0562] The processing of server device 2 for such a switching process example II is as follows: Figure 35 As shown in the image.

[0563] In step S320, the control unit 2a checks whether the imaging device 1 has notified the imaging object, i.e., received the sports information.

[0564] Having received sports information, the control unit 2a proceeds to step S321 and determines whether the competition being imaged is one where additional information can be obtained. For example, in track and field competitions where a schedule has been determined, additional information such as the athletes as subjects in the image can be obtained through a database search. However, in marathons, soccer, and similar events, it is difficult to identify athletes appearing in the image based on the image's metadata (e.g., time or location) through a database search.

[0565] Therefore, in the case of a competition where additional information can be obtained through DB search, control unit 2a proceeds to step S322 and selects the first sending mode.

[0566] On the other hand, in the case of a race where it is difficult to obtain additional information through DB search, the control unit 2a proceeds to step S323 and selects the third sending mode.

[0567] Then, the control unit 2a proceeds from step S322 or S323 to step S324, notifying the imaging device 1 of the transmission mode, and in step S325 notifying the client terminal 3 of the transmission mode.

[0568] <11. Switching Process Example III>

[0569] Reference Figure 36 Description of switching process example III.

[0570] As a communication TP50, a notification of project information is given from the client terminal 3. Project information may include, for example, information that associates the camera ID used to identify imaging device 1 with when and for which competition imaging device 1 performs imaging. Alternatively, it may include information that associates camera operator information with when and for which competition the camera operator performs imaging.

[0571] Server device 2 performs the process of registering the received project information as processing PS30.

[0572] As a communication TP51, the imaging device 1 sends metadata, including time information, camera ID, or camera operator information, to the server device 2 at a certain timing. For example, if the sending timing is when the metadata is initially sent in, for example, a first sending mode, this is sufficient.

[0573] As a processor of PS31, server device 2 refers to the registered project information based on the time information, camera ID, or camera operator information included in the received metadata, and estimates the competition type to be imaged by imaging device 1. Then, it determines the transmission mode based on whether the competition type is one that can provide additional information.

[0574] Subsequently, notifications setting the transmission modes to communication TP40 and communication TP41 are executed similarly to those in switching processing example I. Processing PS11 in imaging device 1 and processing PS12 in client terminal 3 are also executed similarly.

[0575] The processing of server device 2 for such a switching process example III is as follows: Figure 37 and Figure 38 As shown in the image.

[0576] exist Figure 37 In step S330, the control unit 2a checks if the project information has been received from the client terminal 3. If it has been received, the process proceeds to step S331, and project information registration processing is performed. That is, thereafter, the project information is stored in a predetermined storage unit (e.g., storage unit 79) so that the project information can be referenced.

[0577] exist Figure 38 In step S332, the control unit 2a checks whether it has received metadata from the imaging device 1. In this case, it confirms that metadata including time information, camera ID, or camera operator information has been received.

[0578] Having received the corresponding metadata, control unit 2a proceeds to step S333 and estimates the type of competition to be imaged. That is, the competition type is estimated by comparing the metadata with the project information.

[0579] In step S321, control unit 2a determines whether the estimated imaging object competition is one where additional information can be obtained. Then, if the competition is one where additional information can be obtained through a DB search, control unit 2a proceeds to step S322 and selects a first transmission mode. Conversely, if the competition is one where additional information is difficult to obtain through a DB search, control unit 2a proceeds to step S323 and selects a third transmission mode. Then, control unit 2a proceeds from either step S322 or S323 to step S324, notifying imaging device 1 of the transmission mode, and in step S325, notifying client terminal 3 of the transmission mode.

[0580] <12. Eighth Sending Mode>

[0581] Although the above mainly describes the first to seventh transmission modes and their switching, there are also suitable transmission modes, especially when the forwarded image is a moving image. Furthermore, the switching process described above assumes switching to a transmission mode suitable for moving images.

[0582] Below, as the eighth, ninth, and tenth transmission modes, transmission modes suitable for moving images will be described.

[0583] Figure 39 The eighth transmission mode is shown.

[0584] -Communication TP71

[0585] Imaging device 1 sends a proxy motion image to server device 2. The proxy motion image is a motion image with, for example, lower resolution and smaller data volume than the main motion image to be delivered.

[0586] -Communication TP72

[0587] Server device 2 sends the received proxy motion image to client terminal 3.

[0588] -Processing PS40

[0589] Client terminal 3 uses the received proxy motion image to perform UI processing.

[0590] For example, client terminal 3 performs the following on display unit 77: Figure 40 The display shown is as follows: Specifically, it displays the proxy motion image reconstruction image 46 and the timeline 47 of the motion image. In timeline 47, representative images and speech waveforms for each time point extracted from the proxy motion image are displayed. Furthermore, on timeline 47, the person in charge can specify the required interval DK.

[0591] In addition, a main motion image request button 48 is provided, allowing the person in charge to operate it.

[0592] This UI screen is executed so that the person in charge of client terminal 3 can first check the reproduced image of the proxy motion image and specify the range required as the main motion image.

[0593] The person in charge operates the main motion image request button 48 after the specified interval DK.

[0594] In the PS40 processing, the CPU 71 of the client terminal 3 controls the display of such a UI screen and senses the input of the corresponding person in charge.

[0595] -Communication TP73

[0596] Client terminal 3 sends a request to server device 2 for a partial clip of the main motion picture. That is, client terminal 3 generates a request for information on the motion picture of the range DK specified by the person in charge in processing the UI of PS40, and sends the information.

[0597] -Communication TP74

[0598] Server device 2 forwards the request for partial clipping of the main motion image received from client terminal 3 to imaging device 1.

[0599] -Processing PS41

[0600] In response to receiving a request for a partial clip of the main moving image, the imaging device 1 performs a process to create a clip obtained by cutting out a specified interval from the main moving image.

[0601] -Communication TP75

[0602] Imaging device 1 sends a portion of the main motion image generated in processing PS41 to server device 2.

[0603] -Communication TP76

[0604] Server device 2 will forward a partial clip of the main motion image received from imaging device 1 to client terminal 3.

[0605] Therefore, the main motion images of the scene required by the person in charge are delivered to the client terminal 3.

[0606] As described above, instead of forwarding the entire main motion image from the beginning, the proxy motion image is forwarded as initial information forwarding, and the required interval is specified on the client terminal 3 side. Therefore, as the main image forwarding process, a portion of the main motion image within the required interval is forwarded.

[0607] Figure 41 An example of processing is shown when motion images are recorded by imaging device 1 for the eighth transmission mode.

[0608] As Figure 41 In the loop processing LP50, the camera control unit 18 of the imaging device 1 performs motion image recording processing. That is, before the recording stop request is sensed in step S401, the control of steps S402 and S403 is performed for each frame.

[0609] In step S402, the camera control unit 18 causes the camera signal processing unit 13 to perform encoding processing of frame image data and audio data.

[0610] In step S403, the camera control unit 18 causes the recording control unit 14 to record frame image data, audio data, and metadata on the recording medium.

[0611] For example, when a request to stop recording is made by the camera operator or other means, the camera control unit 18 proceeds to step S405 and performs control to read a proxy motion image of the recorded motion image. For example, the recording control unit 14 performs motion image reading, and the camera signal processing unit 13 performs low-resolution conversion, etc., to generate a proxy motion image.

[0612] Then, in step S406, the camera control unit 18 controls the communication unit 16 to send proxy motion images.

[0613] Through the above processing, execution Figure 39 The communication TP71.

[0614] Figure 42 The image shows the use of Figure 39 The communication of the TP72 server device 2 is processed.

[0615] In step S501, the control unit 2a of the server device 2 verifies the proxy motion image received from the imaging device 1.

[0616] Then, when the proxy motion image is received, the control unit 2a performs the process of sending the proxy motion image to the client terminal 3 in step S502.

[0617] In response, the client terminal 3 performs UI processing as described above as processing PS40, and as communication TP73, sends a request for a partial clip of the main motion image.

[0618] The processing of the server device 2 for communication TP74 in response to a request for partial clipping of the main moving image is similar to that described above. Figure 8 The processing in the process.

[0619] Figure 43 The image shows a processing example of the imaging device 1 used to process the PS41 and the communication TP75.

[0620] In step S430, the camera control unit 18 verifies that it has received a motion image request from the server device 2.

[0621] Then, when a list of motion image requests is received, the camera control unit 18 performs loop processing LP60.

[0622] In the loop processing LP60, the camera control unit 18 repeats the processing steps S431 to S442 as many times as required (the number of requested motion images) according to the content of the motion image request.

[0623] First, in step S431, the camera control unit 18 determines whether the motion image request content is a proxy motion image or a main motion image (a request for partial clipping of the main motion image), and branches the processing accordingly.

[0624] Note that the request for a proxy motion image occurs in the ninth and tenth transmission modes, which will be described later, and the processing from steps S432 to S434 will be described later.

[0625] In the event of a request for partial clipping of the main moving image, the camera control unit 18 performs the processing steps S440, S441, and S442.

[0626] In step S440, the camera control unit 18 causes the recording control unit 14 to read the main motion image from the recording medium.

[0627] In step S441, the camera control unit 18, for example, causes the camera signal processing unit 13 to perform a process of cutting out the requested region from the main moving image read from the self-recording medium.

[0628] Then, in step S442, the camera control unit 18 performs the process of sending the clip of the main motion image portion generated in the cutout process to the server device 2 via the communication unit 16.

[0629] Figure 44 The diagram illustrates the processing of server device 2 for communication TP76.

[0630] In step S530, the control unit 2a of the server device 2 verifies the main motion image received from the imaging device 1.

[0631] Then, when the main motion image is received, the control unit 2a performs the process of sending the received main motion image to the client terminal 3 in step S531.

[0632] Therefore, a portion of the main motion image is delivered to the client terminal 3.

[0633] <13. Ninth Sending Mode>

[0634] Reference Figure 45 Describe the ninth transmission mode.

[0635] -Communication TP81

[0636] Imaging device 1 sends a sound-only file containing the motion image to server device 2. The sound-only file is a file containing only the audio data recorded during the motion image recording.

[0637] -Communication TP82

[0638] Server device 2 sends the received audio file to client terminal 3.

[0639] -Process PS42

[0640] Client terminal 3 uses the received audio file to perform UI processing.

[0641] For example, client terminal 3 performs the following on display unit 77: Figure 46The display shown in A. In this case, since only a file containing only sound is received, the proxy motion image cannot be reproduced, and image 46 is displayed instead, resulting in a blank image. As a timeline 47, only the audio waveform is displayed.

[0642] In addition, a proxy motion image request button 49 is displayed, allowing the person in charge to operate it.

[0643] This UI is executed by the person in charge of client terminal 3 to first specify the approximate required interval based on the timeline 47 of the voice waveform. For example, the interval of audience excitement can be determined based on the voice waveform, etc.

[0644] The person in charge operates the proxy motion image request button 49 after the DKA in the specified interval.

[0645] In the PS42 processing, the CPU 71 of the client terminal 3 controls the display of such a UI screen and senses the input of the corresponding person in charge.

[0646] -Communication TP83

[0647] Client terminal 3 sends a request to server device 2 for a partial clip of the proxy motion picture. Specifically, client terminal 3 generates a request for information about the proxy motion picture of the DKA interval specified by the person in charge in the UI processing of PS42, and sends that information.

[0648] -Communication TP84

[0649] Server device 2 will forward the request for partial clipping of the proxy motion image received from client terminal 3 to imaging device 1.

[0650] -Processing PS43

[0651] In response to receiving a request for a partial clip of a proxy moving image, the imaging device 1 performs a process of creating a clip of the proxy moving image obtained by cutting out a specified interval.

[0652] -Communication TP85

[0653] Imaging device 1 will send clips of the proxy motion image generated in processing PS43 to server device 2.

[0654] As a communication TP72, server device 2 sends the received proxy motion images to client terminal 3.

[0655] Communication TP72 and subsequent communication are similar Figure 39 The communication TP72 and subsequent communication are shown in the diagram.

[0656] In client terminal 3, PS40 is processed as follows: Figure 46 The UI display shown in B. This is related to... Figure 40 The content displayed is similar to that displayed in [the context].

[0657] However, this situation is different from Figure 39 The difference in the example is that the proxy motion image is not a proxy motion image of the entire motion image recorded by imaging device 1, but a proxy motion image of the interval specified in processing PS42.

[0658] Since communication TP73 and subsequent communications are redundant, their descriptions will be omitted.

[0659] That is, in the ninth transmission mode, firstly, a range is roughly specified using a file containing only sound, and a proxy moving image is transmitted according to the specified range. Then, the client terminal verifies the proxy moving image and specifies the required range. Therefore, as part of the main image forwarding process, a portion of the main moving image from the required section is forwarded.

[0660] Figure 47 An example of processing is shown when motion images are recorded by imaging device 1 for the ninth transmission mode.

[0661] Similar to Figure 41 As a loop process LP50, the camera control unit 18 of the imaging device 1 performs motion image recording processing.

[0662] Subsequently, for example, when a request to stop recording is made through the operation of a camera operator, the camera control unit 18 proceeds to step S410 and performs control to read only the audio data used for the recorded motion picture. For example, the recording control unit 14 reads only the audio data from the motion picture file, and the camera signal processing unit 13 generates a file containing only sound.

[0663] Then, in step S411, the camera control unit 18 controls the communication unit 16 to send a file containing only sound.

[0664] Through the above processing, execution Figure 45 The communication TP81.

[0665] Figure 48 The diagram illustrates the processing of server device 2 for communication TP82.

[0666] In step S511, the control unit 2a of the server device 2 checks the file containing only sound received from the imaging device 1.

[0667] Then, when a file containing only sound is received, the control unit 2a performs the process of sending the file containing only sound to the client terminal 3 in step S512.

[0668] In response, the client terminal 3 performs UI processing as described above as processing PS42, and as communication TP83, sends a request for a partial clip of the proxy motion image.

[0669] The processing of the server device 2 for communication TP84 in response to a request for partial clipping of a proxy moving image is similar to that described above. Figure 8 The processing in the process.

[0670] According to the above Figure 43 The PS43 performs the processing for imaging device 1 and the TP85 performs the processing for communication.

[0671] That is, upon receiving a request for partial clipping of a proxy moving image, processing from... Figure 43 The process proceeds from step S431 to step S432.

[0672] In step S432, the camera control unit 18 causes the recording control unit 14 to read the proxy motion image from the recording medium.

[0673] In step S433, the camera control unit 18, for example, causes the camera signal processing unit 13 to perform a process of cutting out the requested region from the proxy motion image read from the self-recording medium.

[0674] Then, in step S434, the camera control unit 18 performs the process of sending the clipped portion of the proxy motion image generated in the cut-out process to the server device 2 via the communication unit 16.

[0675] Figure 45 The processing of server device 2 and imaging device 1 after communication TP72 is similar to the processing described regarding the eighth transmission mode.

[0676] <14. Tenth Sending Mode>

[0677] Reference Figure 49 Describe the tenth sending mode.

[0678] -Communication TP91

[0679] Imaging device 1 sends a meta-time series file of the motion image to server device 2. The meta-time series file is a file that can present the metadata (e.g., imaging location information) generated during the recording of the motion image in a time series format.

[0680] -Communication TP92

[0681] Server device 2 sends the received meta-time series file to client terminal 3.

[0682] -Process PS44

[0683] Client terminal 3 uses the received meta-time series file to perform UI processing.

[0684] For example, client terminal 3 performs the following on display unit 77: Figure 50 The illustration shows a display based on a meta-time series file. For example, where the time-series changes in imaging location information can be determined as a meta-time series file, a map image 50 and time series information 51 based on the imaging location information are displayed. In the map image 50, each imaging location is represented by a marker. The time series information 51 represents time and imaging location information in a time series relation to each other. As a timeline 47, only timelines representing interval lengths are displayed. In timeline 47, it is desirable to display the location of each time point.

[0685] It also displays a proxy motion image request button 49, which allows the person in charge to operate.

[0686] This UI is executed so that the person in charge of client terminal 3 can roughly specify the range in which the subject is believed to appear based on the imaging location. By knowing that an activity occurs at a certain time and place, the person in charge can determine the range of motion images that can be imaged for that activity.

[0687] The person in charge operates the proxy motion image request button 49 after the specified interval DKP.

[0688] In the PS44, the CPU 71 of the client terminal 3 controls the display of such a UI screen and senses the input of the corresponding person in charge.

[0689] As a communication TP83, the client terminal 3 generates a request for information of the proxy motion image of the DKP range specified by the person in charge in the UI processing of PS44, and sends the information.

[0690] Because communication TP83 and subsequent communication are similar Figure 45 The communication of the ninth transmission mode shown is therefore omitted from description.

[0691] That is, in the tenth transmission mode, firstly, a range is roughly specified using time-series information, and a proxy motion image is transmitted according to the specified range. Then, the client terminal verifies the proxy motion image and specifies the required range. Therefore, as part of the main image forwarding process, a portion of the main motion image within the required range is forwarded.

[0692] Figure 51 An example of processing is shown when motion images are recorded by imaging device 1 for the tenth transmission mode.

[0693] Similar to Figure 41As a loop process LP50, the camera control unit 18 of the imaging device 1 performs motion image recording processing.

[0694] Subsequently, for example, when a recording stop is requested via operation by a camera operator, the camera control unit 18 proceeds to step S420 and performs control to read time-series data for the metadata of the recorded motion image. For example, the metadata of the motion image file is read by the recording control unit 14, and the time-series file is generated by the camera signal processing unit 13.

[0695] Then, in step S421, the camera control unit 18 controls the communication unit 16 to send the time series file.

[0696] Through the above processing, execution Figure 49 The communication TP91.

[0697] Figure 52 The diagram illustrates the processing of server device 2 for communication TP92.

[0698] In step S521, the control unit 2a of the server device 2 verifies the time series file received from the imaging device 1.

[0699] Then, when the time series file is received, the control unit 2a performs the process of sending the time series file to the client terminal 3 in step S522.

[0700] In response, the client terminal 3 performs UI processing as described above as processing PS44, and as communication TP83, sends a request for a partial clip of the proxy motion image.

[0701] The subsequent processing of server device 2 and imaging device 1 is similar to the processing described regarding the eighth and ninth transmission modes.

[0702] <15. Summary and Variation Examples>

[0703] According to the above embodiments, the following effects can be obtained.

[0704] In this embodiment, the control unit 2a of the server device 2 performs initial information forwarding processing: in response to receiving associated information, it obtains additional information based on the associated information and sends the associated information and additional information to the client terminal 3. The associated information is information related to the image captured by the imaging device 1 and has a smaller data volume than the image. Furthermore, the control unit 2a performs main image forwarding processing: based on a request information generated through selection processing performed in the client terminal 3 based on the associated information or additional information, it sends the image sent from the imaging device 1 to the client terminal 3.

[0705] By performing such processing, server device 2 can appropriately forward the high-definition main image required by client terminal 3, while achieving image forwarding without increasing network load and system storage capacity load at the site where imaging, such as news reports, needs to be delivered immediately afterward, and without causing forwarding delays.

[0706] Furthermore, since presentation based on associated information can be performed in the client terminal 3, an environment can be provided in which the person in charge can easily and appropriately select the desired image.

[0707] Note that specific examples of information processing devices, such as server device 2 or client terminal 3, include various examples such as personal computer devices, tablet information processing devices, mobile phone devices, gaming devices, audio devices, video devices, communication devices, television devices, and server devices. Devices capable of performing calculations as information processing, such as devices integrated with microcomputers, can be implemented as information processing devices according to this disclosure.

[0708] Furthermore, despite Figure 1 MEC 5 is shown in the figure, but in this embodiment, MEC 5 may have the function of replacing server device 2.

[0709] In addition, MEC 5 may have part of the functionality (transmission function) of imaging device 1.

[0710] An example has been described in which the server device 2 of this embodiment obtains additional information by performing a DB search based on associated information (e.g., meta information) during the initial information forwarding process (see First Sending Mode).

[0711] By obtaining more detailed additional information from metadata about the main image via database search, more easily understandable information suitable for image selection can be provided to the responsible parties on the client terminal side. Therefore, the selection operation is optimized, and network communication efficiency is also improved.

[0712] In this embodiment, an example is described in which the associated information includes an image representing an image captured by the imaging device 1 and having a small amount of data (e.g., a thumbnail image), and the server device 2 obtains additional information by performing image recognition processing on the image included in the associated information during the initial information forwarding process (see the second transmission mode).

[0713] Furthermore, when information about the subject is obtained through image analysis and used as supplementary information, it can provide the person in charge on the client terminal 3 with more easily understood information suitable for image selection, which also helps to optimize the selection operation and improve network communication efficiency.

[0714] As described above, as an initial information forwarding process, the server device 2 in this embodiment performs a switching process for multiple sending modes, including first and second sending modes with additional information added, and sending modes without additional information added (second to tenth sending modes).

[0715] Because there are environments, situations, and image (subject) content that are unsuitable for forwarding additional information, server device 2 is not always suitable for performing the processing of obtaining additional information. Therefore, in some cases, server device 2 does not perform the processing of obtaining additional information, thus allowing it to perform processing appropriate to the situation and environment. This also helps reduce server load and improve network efficiency.

[0716] In this embodiment, the following example has been described, in which server device 2 performs the process of sending audio information included in the associated information to client terminal 3 as a predetermined sending mode, which is one of a plurality of sending modes (see fifth sending mode).

[0717] When an audio memo associated with a still or moving image is sent from an imaging device, server device 2 forwards the audio memo to client terminal 3 during initial information forwarding processing, allowing the person in charge of client terminal 3 to perform image selection by reproducing the audio memo. Therefore, an environment in which more desired image selection can be performed can be provided.

[0718] In this embodiment, the following example has been described, in which server device 2 performs the process of converting the audio information included in the associated information into text data and sending the text data to client terminal 3 as a predetermined sending mode, which is one of a plurality of sending modes (see sixth sending mode).

[0719] In the case of sending an audio memo associated with a still image or moving image from an imaging device, server device 2 converts the audio memo into text data during the initial information forwarding process and forwards the text data to client terminal 3. By converting it into text data, the data volume can be reduced to that of audio information.

[0720] Furthermore, since the audio memo content is displayed via text, the person in charge of client terminal 3 does not need to reproduce the audio memo, and it provides an environment where image selection can be performed more easily.

[0721] In this embodiment, an example of server device 2 performing a handover process based on the determination result of the communication state has been described (see Handover Process Example I).

[0722] By switching transmission modes based on changes in communication conditions such as the communication speed of the network transmission path, packet rate, 4G / 5G status, and the communication environment depending on the location of the imaging device, image forwarding can be achieved in accordance with the current situation.

[0723] In particular, the communication environment can be easily fluctuated outdoors when relaying sports images. Furthermore, large sporting events involve numerous journalists, and congestion along the transmission path must also be considered. Selecting an appropriate transmission mode based on these factors is crucial for maintaining proper real-time relay.

[0724] Network status may be assumed to change, particularly between imaging device 1 and server device 2. This is due to the addition of factors such as the location and movement of camera operators, and the area, location, facilities, and circumstances of activities (such as sports).

[0725] Therefore, regarding the presence or absence of forwarding of additional information, one could also consider switching the sending mode from the perspective of whether or not the associated information used to obtain the additional information is forwarded.

[0726] For example, when additional information is obtained through image analysis, the amount of data can be adjusted according to the communication status by switching between requesting a thumbnail image from imaging device 1 and not sending a thumbnail image because no additional information has been obtained.

[0727] Furthermore, the switching process is not limited to sending modes that include additional information (e.g., the first or second sending mode) and those that do not include additional information (e.g., the third to the seventh sending modes), and for example, switching can be performed between the third to the seventh sending modes based on the communication state. For example, switching can be performed between the third and fourth sending modes, or between the fifth and third sending modes.

[0728] Similarly, switching can be performed between the first transmission mode and the second transmission mode.

[0729] Furthermore, switching from one of the first to seventh transmission modes to one of the eighth to tenth transmission modes can be performed by switching from still image forwarding to moving image forwarding. Additionally, during moving image forwarding, switching from the eighth transmission mode to the tenth transmission mode can be performed based on the communication status.

[0730] In this embodiment, an example of the server device 2 performing a switching process based on information about the imaging object that has been notified from the imaging device 1 has been described (see Switching Process Example II).

[0731] Depending on the type of competition, it may be impossible to obtain appropriate additional information. For example, in competitions where the additional information is the contestant's name, the contestant's name can be obtained by performing a DB search based on the date, time, and location of the image in a competition with a defined schedule. However, there are competitions where the contestant's name cannot be obtained. For images from competitions where additional information cannot be obtained, performing the processing to obtain the additional information may reduce processing efficiency. Therefore, it is preferable to switch the transmission mode based on the competition type information, as shown in Switching Processing Example II.

[0732] In this embodiment, an example has been described in which the server device 2 performs the process of estimating the imaging object of the imaging device 1 and performs the switching process based on the information of the estimated imaging object (see Switching Process Example III).

[0733] There are situations where switching the transmission mode based on the competition type is appropriate. For example, in switching processing example III, project information is pre-registered. Therefore, the competition type of the imaging object can be estimated based on metadata such as time information, camera ID, and camera operator information, which are associated information of the image. By switching the transmission mode via this estimation process, forwarding appropriate to the competition type can be performed.

[0734] In this embodiment, an example of the server device 2 performing a process to notify the client terminal 3 or the imaging device 1 of the sending mode of the initial information forwarding process set by the switching process has been described (see switching process examples I, II and III).

[0735] For example, imaging device 1 can set the content of the information to be sent based on the notification of the transmission mode. For example, it can be set whether to send metadata or thumbnail images in the initial transmission, or what type of metadata to send.

[0736] Note that the switching process includes not only switching between a transmission mode for acquiring additional information and a transmission mode for not acquiring additional information, but also, for example, switching between which transmission mode from the first to the tenth transmission modes to use, and whether or not to perform the initial information forwarding process within the first to the tenth transmission modes. For example, the main image can be forwarded from the beginning without performing the initial information forwarding process based on the communication state, etc. Therefore, notifying the imaging device 1 of the transmission mode is useful for cooperative control.

[0737] Furthermore, for example, by notifying the client terminal 3 of the initial information forwarding processing sending mode, it is possible to perform actions such as... Figure 31 The display shown allows the person in charge to easily understand what type of forwarding is being performed.

[0738] In this embodiment, an example has been described where the associated information is metadata associated with an image captured by imaging device 1.

[0739] In other words, it is information associated with the image on the imaging device 1 side and sent to the server device 2. For example, various types of information are assumed, such as information about the imaging device 1, information about the camera operator, information about the location and time of image capture, image analysis information on the imaging device 1 side, various parameters about imaging operation and processing, sensing information of various sensors in the imaging device 1 at the time of image capture, information about image data / files, thumbnail images, screen pin images, and text information and audio memo information associated with the image of the camera operator.

[0740] This metadata is used to present the image content and obtain additional information about the image.

[0741] In this embodiment, the associated information has been described as an example of an image captured by imaging device 1, or an image with a smaller data volume.

[0742] Examples of images with smaller data volumes include thumbnail images, screen pin images, and proxy motion images.

[0743] For example, thumbnails of images with smaller data volumes than the main image, proxy motion images, etc., are very useful information for the person in charge of client terminal 3 to determine the content of the main image. Furthermore, it is also useful as information for obtaining additional information through image analysis.

[0744] In this embodiment, an example has been described where the associated information represents an image of a group of images captured sequentially by the imaging device 1 (see seventh transmission mode).

[0745] When a series of consecutively captured images are set as grouped images, the image that represents the entire group as a single image (e.g., a thumbnail of the first image, etc.) is set as associated information.

[0746] Serial imaging involves a large number of images and large amounts of data, but with essentially the same subject content. Therefore, to improve processing efficiency and reduce network load, it is desirable to represent the images using representative thumbnail images and send such thumbnail images during the initial information forwarding process.

[0747] In this embodiment, an example has been described in which the associated information is audio information recorded together with the motion image by capturing the motion image (see Ninth Transmission Mode).

[0748] For example, in the initial information forwarding process, server device 2 receives a file containing only sound that was recorded simultaneously with the motion picture recording as associated information, and sends this information to client terminal 3.

[0749] The audio information of the moving images is used by the person in charge of client terminal 3 to listen to verify the content of the moving images and specify useful information for the required scene.

[0750] In this embodiment, an example has been described where the associated information is time-series data during the capture of motion images (see Tenth Transmission Mode).

[0751] The time-series data of the motion images is used by the person in charge of client terminal 3 to verify the content of the motion images and specify useful information for the required scene.

[0752] The following example has been described, wherein, as initial information forwarding processing, the server device 2 of this embodiment performs the following processes: sending first associated information to the client terminal 3 with or without additional information; forwarding request information generated by selection processing in the client terminal 3 based on the sending of the first associated information to the imaging device 1; and sending second associated information sent from the imaging device 1 to the client terminal 3 according to the request information (see first to seventh, ninth and tenth sending modes).

[0753] That is, as initial information forwarding processing, the information is sent multiple times before the main image is sent to the client terminal 3.

[0754] As an initial information forwarding process, for example, server device 2 sends metadata to client terminal 3 for the first time to enable client terminal 3 to make a coarse selection, and sends a thumbnail image for the second time to enable client 3 to make a final selection, so that client terminal 3 can make a more appropriate and easily understandable selection without excessively increasing the amount of data to be forwarded.

[0755] Note that in this embodiment, the following example has been described, wherein as the initial information forwarding process, two round trips are mainly performed. However, as the initial information forwarding process before the transmission of the main image, only one round trip may be performed, or three or more round trips may be performed.

[0756] In this embodiment, the first associated information, which is the information with a smaller data volume than the second associated information, is the information that performs the initial information forwarding process multiple times.

[0757] For example, one could conceive of the following example, where the first associated information is metadata representing imaging time, imaging location, etc., and the second associated information is a thumbnail image, etc.

[0758] In the first round trip of the initial information forwarding process, first associated information, representing imaging time, imaging location, etc., and having a relatively small amount of data, is sent to client terminal 3. This allows information representing many forwardable images to be presented without increasing the forwarding load. For example, in the second round trip, based on the first selection by client terminal 3, thumbnail images are forwarded to present more detailed image content. Therefore, since thumbnail images only need to be forwarded for images reduced to a certain size, the person in charge of client terminal 3 can more clearly understand the image content and make selections without increasing the forwarding load.

[0759] The imaging device 1 of this embodiment includes a camera control unit 18, which performs association information transmission processing to send association information to an external server device 2. The association information is information associated with the captured image and has a smaller data volume than the image. It also performs main image transmission processing to send an image represented by the request information to the server device 2 according to the request information received from the server device 2.

[0760] Therefore, an imaging device 1 can be implemented to perform operations suitable for the image forwarding system of this embodiment.

[0761] The program in this embodiment is a program that causes, for example, a CPU, DSP, or a device including them, to perform the processing of the server device 2 described above.

[0762] That is, the program in this embodiment is a program for causing the information processing device to perform the following processes: initial information forwarding processing, in response to receiving associated information, obtaining additional information based on the associated information, and sending the associated information and additional information to the client terminal 3, wherein the associated information is information associated with the image captured by the imaging device 1 and has a smaller data amount than the image; and main image forwarding processing, sending the image sent from the imaging device 1 to the client terminal 3 according to the request information generated by the selection processing performed in the client terminal 3 based on the sending of the associated information and additional information.

[0763] Using such a program, the aforementioned server device 2 can be implemented by various computer devices.

[0764] Furthermore, the program in this embodiment is a program that causes, for example, a CPU, DSP, or a device including them to perform the processing of the imaging device 1.

[0765] That is, the program is a program for causing the processor, etc., which is a control unit, to perform the following processes: association information sending process, sending association information to an external server device 2, the association information being information associated with the captured image and having a smaller data amount than the image; and main image sending process, sending the image represented by the request information received from the server device 2 to the server device 2 according to the request information received from the server device 2.

[0766] Using such a program, the aforementioned imaging device 1 can be realized.

[0767] Such programs can be pre-recorded in an HDD or in the ROM of a microcomputer with a CPU, the HDD being a recording medium built into a device such as a computer.

[0768] Alternatively, it can be temporarily or permanently stored (recorded) on removable recording media, including floppy disks, compact disc read-only memory (CD-ROM), magneto-optical (MO) discs, digital multifunction discs (DVDs), Blu-ray discs (registered trademark), magnetic disks, semiconductor memory, memory cards, etc. Such removable recording media can be provided as so-called software packages.

[0769] Furthermore, such programs can be installed on personal computers or other devices from removable recording media, or downloaded from download sites via networks such as local area networks (LANs) or the Internet.

[0770] Furthermore, such a program is applicable to providing the server device 2 or imaging device 1 of this embodiment in a wide range of applications. For example, by downloading the program to portable terminal devices such as smartphones or tablets, mobile phones, personal computers, gaming devices, video devices, and personal digital assistants (PDAs), smartphones and the like can be used as the server device 2 or imaging device 1 of this disclosure.

[0771] Note that the effects described in this specification are illustrative only and not limiting, and may provide other effects.

[0772] Note that this technology can also be configured as described below.

[0773] (1) An information processing device, comprising:

[0774] Control unit, the control unit performs:

[0775] Initial information forwarding processing, in response to receiving associated information, obtaining additional information based on the associated information, and sending the associated information and the additional information to the client terminal, wherein the associated information is information associated with an image captured by the imaging device and has a smaller data size than the image; and

[0776] The main image forwarding process sends the image from the imaging device to the client terminal based on the request information generated by the selection process performed in the client terminal through the transmission of the associated information or the additional information.

[0777] (2) The information processing device according to (1),

[0778] The control unit obtains the additional information by performing a database search based on the associated information during the initial information forwarding process.

[0779] (3) The information processing device according to (1) or (2),

[0780] The associated information includes images representing images captured by the imaging device and images with smaller data volumes, and

[0781] The control unit obtains the additional information by performing image recognition processing on the images included in the associated information during the initial information forwarding process.

[0782] (4) The information processing apparatus according to any one of (1) to (3),

[0783] Wherein, the control unit

[0784] As part of the initial information forwarding process, a switching process for multiple sending modes is performed, including a sending mode with the additional information added and a sending mode without the additional information added.

[0785] (5) The information processing equipment according to (4),

[0786] Wherein, the control unit

[0787] The process of sending the audio information included in the associated information to the client terminal is performed as a predetermined sending mode, which is one of the plurality of sending modes.

[0788] (6) The information processing device according to (4),

[0789] Wherein, the control unit

[0790] The execution involves sending the text data obtained by converting the audio information included in the associated information into text data to the client terminal as a predetermined sending mode, which is one of the plurality of sending modes.

[0791] (7) The information processing apparatus according to any one of (4) to (6),

[0792] Wherein, the control unit

[0793] The switching process is performed based on the determination of the communication status.

[0794] (8) The information processing apparatus according to any one of (4) to (7),

[0795] Wherein, the control unit

[0796] The switching process is performed based on information about the imaging object that is notified from the imaging device.

[0797] (9) The information processing apparatus according to any one of (4) to (8),

[0798] Wherein, the control unit

[0799] The process of estimating the imaging object of the imaging device is performed, and the switching process is performed based on the information of the estimated imaging object.

[0800] (10) The information processing apparatus according to any one of (4) to (9),

[0801] Wherein, the control unit

[0802] The process of notifying the client terminal or the imaging device of the sending mode of the initial information forwarding process set by the switching process is executed.

[0803] (11) The information processing apparatus according to any one of (1) to (10),

[0804] The associated information includes metadata associated with the image captured by the imaging device.

[0805] (12) The information processing apparatus according to any one of (1) to (11),

[0806] The associated information includes images captured by the imaging device and images with smaller data volumes.

[0807] (13) The information processing apparatus according to any one of (1) to (12),

[0808] The associated information includes images representing grouped images, which are multiple images continuously captured by the imaging device.

[0809] (14) The information processing apparatus according to any one of (1) to (13),

[0810] The associated information includes audio information recorded along with motion images captured by the imaging device.

[0811] (15) The information processing apparatus according to any one of (1) to (14),

[0812] The associated information includes time-series data of the motion image captured by the imaging device at the time of capture.

[0813] (16) The information processing apparatus according to any one of (1) to (15),

[0814] Specifically, the control unit performs the initial information forwarding process.

[0815] The process of sending the first association information to the client terminal, with or without the additional information.

[0816] The process of forwarding request information generated by the selection process performed in the client terminal based on the first associated information to the imaging device, and

[0817] The processing involves sending second associated information from the imaging device to the client terminal based on the request information.

[0818] (17) The information processing device according to (16),

[0819] The first association information includes information with a smaller data volume than the second association information.

[0820] (18) An information processing method, wherein the information processing device performs:

[0821] Initial information forwarding processing, in response to receiving associated information, obtaining additional information based on the associated information, and sending the associated information and the additional information to the client terminal, wherein the associated information is information associated with an image captured by the imaging device and has a smaller data size than the image; and

[0822] The main image forwarding process sends the image from the imaging device to the client terminal based on the request information generated by the selection process performed in the client terminal through the transmission of the associated information or the additional information.

[0823] (19) An imaging device, comprising:

[0824] Control unit, the control unit performs:

[0825] The associated information transmission process sends associated information to an external information processing device. This associated information is information related to the captured image and has a smaller data volume than the image itself.

[0826] The main image transmission process involves sending an image represented by the request information received from the information processing device to the information processing device.

[0827] (20) An image forwarding system, wherein the image forwarding system:

[0828] To achieve information communication between imaging equipment and information processing equipment, and

[0829] To achieve information communication between the information processing device and the client terminal,

[0830] The imaging device includes:

[0831] The control unit that performs the following processes:

[0832] The associated information transmission process sends associated information to the information processing device. This associated information is information related to the captured image and has a smaller data volume than the image itself.

[0833] The main image transmission process involves sending an image represented by the request information to the information processing device, based on a request information generated through selection processing performed in the client terminal that has already received the associated information via the information processing device.

[0834] The information processing device includes:

[0835] The control unit that performs the following processes:

[0836] Initial information forwarding processing, in response to receiving the associated information from the imaging device, obtaining additional information based on the associated information, and sending the associated information and the additional information to the client terminal, and

[0837] The main image forwarding process sends the image from the imaging device to the client terminal according to the request information.

[0838] Reference tag list

[0839] 1 Imaging equipment

[0840] 2 Server Equipment

[0841] 2a Control Unit

[0842] 3. Client Terminal

[0843] 4 base stations

[0844] 5 MEC

[0845] 6 Networks

[0846] 7 DB

[0847] 8 Text Conversion Engine

[0848] 18 Camera control unit

[0849] 70 Information processing equipment

[0850] 71 CPU

Claims

1. An information processing device, comprising: Control unit, the control unit performs: Initial information forwarding processing: in response to receiving associated information, additional information is obtained based on the associated information, and the associated information and the additional information are sent to the client terminal. The associated information is information associated with an image captured by the imaging device and has a smaller data volume than the image. as well as The main image forwarding process sends the image from the imaging device to the client terminal based on a request information generated through selection processing performed in the client terminal based on the associated information or the additional information. The associated information includes metadata associated with the image captured by the imaging device, and the additional information represents information searched, extracted, or generated by the information processing device based on the associated information.

2. The information processing device according to claim 1, in, The control unit obtains the additional information by performing a database search based on the associated information during the initial information forwarding process.

3. The information processing device according to claim 1, in, The associated information includes images representing images captured by the imaging device and images with smaller data volumes, and The control unit obtains the additional information by performing image recognition processing on the images included in the associated information during the initial information forwarding process.

4. The information processing device according to claim 1, in, The control unit As part of the initial information forwarding process, a switching process for multiple sending modes is performed, including a sending mode with the additional information added and a sending mode without the additional information added.

5. The information processing device according to claim 4, in, The control unit The process of sending the audio information included in the associated information to the client terminal is performed as a predetermined sending mode, which is one of the plurality of sending modes.

6. The information processing device according to claim 4, in, The control unit The execution involves sending the text data obtained by converting the audio information included in the associated information into text data to the client terminal as a predetermined sending mode, which is one of the plurality of sending modes.

7. The information processing device according to claim 4, in, The control unit The switching process is performed based on the determination of the communication status.

8. The information processing device according to claim 4, in, The control unit The switching process is performed based on information about the imaging object that is notified from the imaging device.

9. The information processing device according to claim 4, in, The control unit The process of estimating the imaging object of the imaging device is performed, and the switching process is performed based on the information of the estimated imaging object.

10. The information processing device according to claim 4, in, The control unit The process of notifying the client terminal or the imaging device of the sending mode of the initial information forwarding process set by the switching process is executed.

11. The information processing device according to claim 1, in, The metadata includes information about the imager, imaging time, and imaging location associated with the image captured by the imaging device.

12. The information processing device according to claim 1, in, The associated information includes images representing images captured by the imaging device and images with a smaller data volume.

13. The information processing device according to claim 1, in, The associated information includes images representing grouped images, which are multiple images captured consecutively by the imaging device.

14. The information processing device according to claim 1, in, The associated information includes audio information recorded along with the motion images captured by the imaging device.

15. The information processing device according to claim 1, in, The associated information includes time-series data of the motion images captured by the imaging device at the time of capture.

16. The information processing device according to claim 1, in, As part of the initial information forwarding process, the control unit executes... The process of sending the first association information to the client terminal, with or without the additional information. The process of forwarding request information generated by the selection process performed in the client terminal based on the first associated information to the imaging device, and The processing involves sending second associated information from the imaging device to the client terminal based on the request information.

17. The information processing device according to claim 16, in, The first association information includes information with a smaller data volume than the second association information.

18. An information processing method, wherein, Information processing equipment performs: Initial information forwarding processing: in response to receiving associated information, additional information is obtained based on the associated information, and the associated information and the additional information are sent to the client terminal. The associated information is information associated with an image captured by the imaging device and has a smaller data volume than the image. as well as The main image forwarding process sends the image from the imaging device to the client terminal based on a request information generated through selection processing performed in the client terminal based on the associated information or the additional information. The associated information includes metadata associated with the image captured by the imaging device, and the additional information represents information searched, extracted, or generated by the information processing device based on the associated information.

19. An imaging device, comprising: Control unit, the control unit performs: The associated information sending process sends associated information to an external information processing device. The associated information is information related to the captured image and has a smaller data volume than the image. as well as The main image transmission process involves sending the image represented by the request information received from the information processing device to the information processing device. The associated information includes metadata associated with the image captured by the imaging device, additional information obtained by the information processing device based on the associated information, and the additional information represents information searched, extracted, or generated by the information processing device based on the associated information.

20. An image forwarding system, wherein the image forwarding system: To achieve information communication between imaging equipment and information processing equipment, and To achieve information communication between the information processing device and the client terminal, The imaging device includes: The control unit that performs the following processes: The associated information transmission process sends associated information to the information processing device. This associated information is information related to the captured image and has a smaller data volume than the image itself. The main image transmission process involves sending an image represented by the request information to the information processing device, based on a request information generated through selection processing performed in the client terminal that has already received the associated information via the information processing device. The information processing device includes: The control unit that performs the following processes: Initial information forwarding processing, in response to receiving the associated information from the imaging device, obtaining additional information based on the associated information, and sending the associated information and the additional information to the client terminal, and The main image forwarding process sends the image from the imaging device to the client terminal according to the request information. The associated information includes metadata associated with the image captured by the imaging device, and the additional information represents information searched, extracted, or generated by the information processing device based on the associated information.

Images