Communication system, information processing device, and communication processing method

Abstract

A communication system according to the present invention includes: a first device capable of maintaining a communication session with a plurality of imaging devices; and a second device that starts a computational process when data related to one imaging device is received from the first device while the first device maintains the communication session, and performs a process for generating communication data to be transmitted to the one imaging device via the communication session by the first device.

Description

Communication system, information processing device, and communication processing method

[0001] The present technology relates to a communication system, an information processing device, and a communication processing method, and relates to a technology suitable for communication with a large number of imaging devices.

[0002] There are communication systems that enable remote control of an imaging device (also referred to as a "camera" in this disclosure) and back up data files, such as still image data and video data, captured by the camera to a cloud. Patent Document 1 listed below discloses a technique for transferring image data from an imaging device to an external device.

[0003] JP 2007-81489 A

[0004] For example, Picture Transfer Protocol (PTP) is known as a communication protocol for transferring data between a camera and a computer device. A PTP connection always requires one computer device to control each camera.

[0005] The need for a constantly running computer device as a controller for the cameras increases the burden on the controller device in systems with a large number of cameras. For example, when considering a backup service for a large number of cameras, the operating costs of the communication system increase significantly.

[0006] Therefore, the purpose of this technology is to realize a communication system that can handle a large number of cameras with relatively few hardware resources.

[0007] A communication system according to the present technology includes a first device capable of maintaining communication sessions with a plurality of imaging devices, and a second device that, when the first device receives data related to one of the imaging devices with which the first device maintains a communication session from the first device, starts a calculation process and performs processing to generate communication data to be transmitted to the one imaging device via the communication session of the first device. The imaging device and the first device execute a stateful communication session. In contrast, the second device and the first device communicate statelessly. The first device has a function of converting stateful communication with the imaging device into stateless communication, and the calculation process for the actual communication is performed by the second device.

[0008] FIG. 1 is an explanatory diagram of a PTP communication system according to an embodiment of the present technology; FIG. 2 is an explanatory diagram of a PTP communication system using a WebRTC (Web Real-Time Communication) communication path according to an embodiment; FIG. 3 is an explanatory diagram of a system configuration of a comparative example; FIG. 4 is an explanatory diagram of a system configuration of an embodiment; FIG. 5 is an explanatory diagram of a system configuration of an embodiment; FIG. 6 is a flowchart of a processing example of an embodiment; FIG. 7 is an explanatory diagram of a system configuration of a modified example of an embodiment; FIG. 8 is a block diagram of a configuration of a camera used in an embodiment; FIG. 9 is a block diagram of an information processing device that can function as a relay server or a separation processing module according to an embodiment.

[0009] The embodiments will be described below in the following order: <1. Configuration of communication system> <2. Example of processing of communication system> <3. Example of configuration of camera and information processing device> <4. Summary and modified examples>

[0010] 1. Communication System Configuration FIG. 1 shows a comparison between a normally assumed PTP communication system and a PTP communication system according to an embodiment.

[0011] Images and commands are transmitted between the camera 1 and the computer device 2, for example, via wired PTP communication using a USB (Universal Serial Bus) cable, or via PTP-IP (Picture Transfer Protocol over Internet Protocol) communication via a wireless LAN (Local Area Network) as shown in the upper part of Figure 1. In this disclosure, the term "image" is used to collectively refer to still images and videos. In this disclosure, PTP communication is used to include PTP-IP communication, and the PTP communication in the following description primarily refers to PTP-IP communication.

[0012] The computer device 2 is assumed to be a personal computer used by a general user 91, or a cloud server for a specific service provided to a user 90 of the camera 1, such as a file backup service.

[0013] For example, by sending a command via PTP communication, a user 91 of the computer device 2 can receive and display an image (preview image) being captured by the camera 1 on the computer device 2, and can perform remote operations such as operating the shutter at any timing or manipulating the settings of the camera 1. Alternatively, the user 90 of the camera 1 can take a picture while transferring the captured image to the computer device 2 as a cloud server for backup.

[0014] For example, PTP communication commands include the following: Push shooting button: shutter operation instruction Get device property: get device (camera 1) properties Get device status: get device status Change device settings: change device settings (shutter speed, etc.) Get Content List: get content list Transfer File: file transfer

[0015] However, in the above-described PTP communication, camera 1 and computer device 2 must be performing stateful communication. In other words, the PTP communication session between camera 1 and computer device 2 must be maintained in an active state. For example, when camera 1 is turned on and a PTP communication session is initiated with computer device 2, the PTP communication session is maintained until camera 1 is turned off. Computer device 2 stores information about camera 1 in its memory, and after issuing a command to camera 1, it waits for a response.

[0016] In contrast to this, in this embodiment, the functions of the computer device 2 in PTP communication are configured to be divided into a relay server 3 as a first device and a separation processing module 4 as a second device, as shown in the lower part of Fig. 1. The relay server 3 and the separation processing module 4 are each configured by a computer device (information processing device) equipped with a processor, memory, etc.

[0017] In this case, PTP communication between camera 1 and relay server 3 is always connected and stateful, but the relay server 3 and separation processing module 4 are not always connected and are stateless. Separation processing module 4 does not store any information about communications that have been performed in memory. For example, when a message (such as a command response) from camera 1 arrives via relay server 3, a calculation process for message processing is started, and information about previous communications with camera 1 is restored from the accompanying information as needed, and the necessary processing is then performed and the next response is returned. In other words, when data arrives from camera 1, relay server 3 first supplies accompanying information (session ID, device ID) to separation processing module 4 to start separation processing module 4. After startup, no trigger is required for separation processing module 4, and processing alternates between separation processing module 4 and camera 1 via relay server 3.

[0018] In this embodiment, PTP communication between camera 1 and relay server 3 is performed over a WebRTC communication path. For example, as shown in the upper part of Figure 2, camera 1 and relay server 3 create a WebRTC communication path via cloud server 9, allowing image files and other data stored in the camera to be transferred to a specified server (not shown) using a PTP command. Hereinafter, PTP communication using WebRTC will be referred to as "PTP over WebRTC communication." Note that, for the sake of simplicity, in Figures 1, 5, and 8, the cloud server 9 is omitted and the notation "PTP over WebRTC" is used, as shown in the lower part of Figure 2.

[0019] In this embodiment, an example of PTP over WebRTC communication is described, but the camera 1 and the relay server 3 may communicate via a communication path of another standard, such as a local network, etc. However, using PTP over WebRTC communication is advantageous for providing services corresponding to a large number of cameras 1 via the relay server 3 as shown in Figure 4.

[0020] The communication system of this embodiment is provided with a relay server 3 and a separation processing module 4, and the relay server 3 is a device that can maintain a PTP communication session with multiple cameras 1. The separation processing module 4 is a device that starts a calculation process when it receives data related to one of the cameras 1 with which the relay server 3 is maintaining a communication session from the relay server 3. The separation processing module 4 then performs processing to generate communication data to be transmitted to one of the cameras 1 via the PTP communication session by the relay server 3.

[0021] Such a configuration is advantageous for providing services corresponding to a large number of cameras 1. Figure 3 shows a typical PTP communication system as shown in the upper part of Figure 1, in which services are provided to a large number of cameras 1. The computer device 2 is, for example, a server device on the cloud side, and serves as a PTP client.

[0022] As described above, the computer device 2 and camera 1 must maintain a communication session, so the computer device 2 is always running for the camera 1. This means that the processing load on the computer device 2 is high, and as the amount of traffic increases (the number of cameras 1 serving the service increases), the number of computer devices 2 on the cloud side must be increased. This means that the cost of providing the service increases significantly depending on the number of cameras 1 connected.

[0023] 4 is a schematic diagram of the configuration of this embodiment. The relay server 3 is responsible for maintaining active PTP communication with each camera 1. However, the relay server 3 only transfers data received from the camera 1 to the separation processing module 4, and receives data to be sent to the camera 1 from the separation processing module 4 and sends it to the camera 1; it does not perform any calculation processes related to actual communication. In this way, one relay server 3 can maintain stateful PTP communication with a large number of cameras 1. For example, one relay server 3 can handle approximately 500 to 1,000 cameras 1.

[0024] The separation processing module 4 starts an arithmetic process by a required application program only when necessary, such as when data is received from a certain camera 1, in response to a trigger from the relay server 3. The figure shows arithmetic functions 4a that are started to execute an arithmetic process. Each arithmetic function 4a executes an arithmetic process, such as response processing for the camera 1, and generates communication data to be sent to the camera 1. This communication data is sent to the camera 1 via the relay server 3. In this case, each arithmetic function 4a is only started temporarily when necessary. Therefore, the processing load on the separation processing module 4 is not expected to increase.

[0025] In other words, the communication system in the lower part of FIG. 1 can be said to convert stateful communication into stateless communication by the relay server 3.

[0026] A specific example of a mechanism by which the separation processing module 4 is stateless and does not need to store progress information of each session in memory will be described with reference to FIG.

[0027] When the relay server 3 receives, for example, response data to a command from the camera 1 via PTP over WebRTC communication, it passes the received data (PTP binary) along with accompanying information such as a session ID (identification) and a device ID to the separation processing module 4. The separation processing module 4 then starts a processing process in response to this.

[0028] However, as described above, the separation processing module 4 does not store information necessary for communication with the camera 1. Therefore, the necessary information is stored in an external database 5, such as a database server 5a or a cache server 5b in the figure.

[0029] For example, the database server 5a stores history information of past sessions, such as information on how many image files stored in the camera 1 have been backed up. In other words, this is information on past processing in the services provided. The cache server 5b also stores short-term information on the current PTP session with each camera 1. For example, this information includes information on what commands have been sent so far during the current PTP session, and this information becomes unnecessary when the PTP session ends due to the camera 1 being powered off.

[0030] The separation processing module 4 reads information from the database server 5a or cache server 5b using the session ID as a key, and restores the session state related to the received data received this time from a certain camera 1. Then, it executes processing corresponding to the received data, and generates communication data (PTP binary) to be sent to the camera 1. The generated communication data is handed over to the relay server 3, and is sent from the relay server 3 to the camera 1 via PTP-over-WebRTC communication. The separation processing module 4 stores history information related to this communication in the database server 5a or cache server 5b, and terminates the calculation process. The separation processing module 4 itself does not store the communication session information, but discards it.

[0031] With this configuration, the separation processing module 4 can handle stateless communication with the camera 1. Furthermore, the separation processing module 4 only needs to activate the processing process when communication arrives, so the processing load does not become excessive.

[0032] In the communication system of this embodiment, when the application logic is on the side that acts as the controller for the camera 1 and it is necessary to maintain the session in a stateful manner, the following advantages can be obtained by separating the controller side into the relay server 3 and the separation processing module 4.

[0033] First, the relay server 3 is the only server that is constantly connected to the camera 1, and the relay server 3 can handle many cameras 1 as shown in FIG. 4, so system costs can be significantly reduced.

[0034] It is also possible to simultaneously execute multiple applications in the separation processing module 4. This is because the calculation function 4a of the separation processing module 4 is activated by the relay server 3 when necessary, and the relay server 3 is configured to flexibly control which application to activate for which connection in PTP over WebRTC communication. For example, it is possible to select a shooting control application or a backup application depending on the device or session. This makes it possible to realize a communication system that provides multiple types of services.

[0035] Furthermore, multiple versions of an application can be executed simultaneously, and seamless version upgrades can be performed on a connection-by-connection basis. This is also possible because the calculation function 4a of the separation processing module 4 is activated by the relay server 3 when necessary. For example, version 1 of an application is used during a PTP session with one camera 1, but version 2 can be used for a PTP session with another camera 1. It is also possible to use version 1 of an application with a certain camera 1 until the camera 1 is powered down, but then use version 2 the next time communication begins.

[0036] 6 and 7 show an example of processing by the communication system according to the embodiment. The figures show the processing by the camera 1, the relay server 3, and the separation processing module 4 as an example of processing for providing a backup service for files in the camera 1. The figures also show the processing by the associated device linkage management system 6, the database 5, the content management server 7, and the upload server 8.

[0037] The device linkage management system 6 is a device that manages WebRTC connections. The database 5 is the database server 5a or cache server 5b described above. The content management server 7 is a device that manages the upload of image files. The upload server 8 is a device that stores uploaded image files.

[0038] 6 shows the process from when the camera 1 is powered on to when PTP over WebRTC communication starts. When the camera 1 is powered on in step S101, the camera 1 notifies the device linkage management system 6 that the power is on.

[0039] When the device collaboration management system 6 receives a power-on notification in step S401, it checks the backup settings for the camera 1 in step S402, and if there are no problems, it creates a WebRTC channel in step S403. The device collaboration management system 6 then notifies the camera 1 and relay server 3 of the WebRTC channel information.

[0040] In response to receiving WebRTC channel information from the device collaboration management system 6, the camera 1 initiates a WebRTC connection in step S102. In response to receiving WebRTC channel information from the device collaboration management system 6, the relay server 3 also initiates a WebRTC connection in step S201. Although details of the WebRTC connection process are omitted, the WebRTC connection between the camera 1 and the relay server 3 is thereby completed (steps S103, S202).

[0041] Next, camera 1 performs PTP processing in step S104. Relay server 3 also starts PTP initialization processing in step S203. In this case, relay server 3 passes the device ID and session ID to separation processing module 4 and starts a processing process for PTP initialization in separation processing module 4.

[0042] In step S301, the separation processing module 4 generates an initialization PTP binary and executes an initialization process. In this process, the separation processing module 4 communicates with the camera 1 via the relay server 3 the required number of times. The relay server 3 simply relays communication between the separation processing module 4 and the camera 1. After the above process, the separation processing module 4 completes PTP initialization in step S302. By performing PTP initialization, the camera 1 and the relay server 3 are connected via PTP-over-WebRTC communication. Thereafter, this PTP-over-WebRTC communication is maintained until the camera 1 is powered off.

[0043] With the communication session maintained as described above, the separation processing module 4 proceeds to the backup processing, which is shown in FIG.

[0044] In step S320, separation processing module 4 transmits a content list acquisition command to relay server 3. In step S205, relay server 3 relays the content list acquisition command and transmits it to camera 1. In response to the content list acquisition command, camera 1 generates a content list, which is a list of image files in camera 1, in step S120 and transmits it to relay server 3. In step S206, relay server 3 transmits the device ID and session ID along with the received content list to separation processing module 4.

[0045] As a result, the separation processing module 4 receives the content list in step S321 and can determine the image files to be backed up. In this case, the separation processing module 4 accesses the database 5, obtains the necessary information using the session ID as a key, and restores the state of the communication session. The separation processing module 4 also stores the content list in the database 5 using the device ID and session ID as keys.

[0046] In Figure 7, the processing of the database 5 only shows the saving of the content list as step S501, but the separation processing module 4 will save and restore the state every time a PTP command is processed, and the database 5 will perform the process of saving and reading information each time.

[0047] After determining the files to be backed up, the separation processing module 4 generates an upload destination URL (Uniform Resource Locator) in step S330. In this case, the separation processing module 4 requests the content management server 7 to issue an upload destination URL. In response, the content management server 7 issues the upload destination URL in step S601 and sends it to the separation processing module 4.

[0048] When separation processing module 4 receives the upload destination URL, in step S331 it transmits the upload destination URL and information on the file to be backed up to relay server 3, and in step S207 relays these and transmits them to camera 1. As a result, camera 1 is notified of the upload destination URL and information on the file to be backed up.

[0049] In response to receiving the upload destination URL, the camera 1 uploads the specified backup image file. In this case, the camera 1 performs processing to send the backup target file specified by the separation processing module 4 to the specified upload destination URL. At this time, the camera 1 sends the image file to the upload server 8 using, for example, the HTTP PUT method. Therefore, the relay server 3 does not relay the upload. The upload server 8 receives the binary data of the file and performs backup processing.

[0050] When the camera 1 has completed uploading and transmitting the specified file, it notifies the user in step S122 that the file upload has been completed. This is a notification of a completion event. In step S208, the relay server 3 adds the device ID and session ID to the received notification and forwards it to the separation processing module 4. In step S332, the separation processing module 4 performs processing to complete content generation. In this case, the separation processing module 4 requests content generation from the content management server 7. In step S602, the content management server 7 performs processing to ensure that the file currently uploaded to the upload server 8 is managed as backup content.

[0051] The process from step S330 to step S602 enclosed by the dashed line in FIG. 7 is repeated a number of times corresponding to the number of files to be backed up.

[0052] 6 and 7, processing is executed as a file backup service for the camera 1. The relay server 3 and the separation processing module 4 can perform the above processing for each of the multiple cameras 1.

[0053] So far, we have explained an example of processing for file backup, but other processing can also be performed using the communication system of the embodiment. For example, there is processing for remotely controlling multiple cameras 1. For example, Figure 8 shows an example of the configuration of Figure 5, in which commands can be sent from an external computer device 10 to the relay server 3.

[0054] An application 11 for remote control is started in the computer device 10, and operation information for the camera 1 is sent in response to, for example, a user operation.

[0055] When the relay server 3 receives operation information from the computer device 10, the relay server 3 determines the target camera 1 and the PTP session with that camera 1, and transfers the operation information along with the device ID and session ID to the separation processing module 4. This causes the separation processing module 4 to start a corresponding calculation process, restore the session state using information from the database server 5a and cache server 5b, and generate a corresponding PTP binary, such as a PTP command corresponding to the operation information.

[0056] The separation processing module 4 transmits the PTP command to the relay server 3, and the relay server 3 transmits it to the camera 1. As a result, the camera 1 executes an operation according to the operation information intended by the user.

[0057] For example, when a relay server 3 and multiple cameras 1 maintain PTP over WebRTC communication, such communication makes it possible for the computer device 10 to operate individual cameras 1 or perform similar operations on multiple cameras 1.

[0058] 3. Configuration Examples of Camera and Information Processing Device> Configuration examples of the camera 1 in the above embodiment and configuration examples of the information processing device that serves as the relay server 3 and the separation processing module 4 will be described.

[0059] Fig. 9 shows an example configuration of camera 1. Camera 1 includes a camera control unit 30, an imaging unit 31, a display unit 32, an operation unit 33, a sensor unit 34, a distance measurement unit 35, a storage control unit 36, a memory unit 37, and a communication unit 38. Note that the configuration in Fig. 9 is merely an example showing the main parts of camera 1, and the camera may include components not shown, and it is not necessarily required to include all of the components shown.

[0060] The imaging unit 31 includes various lenses and an image sensor that constitute an incident optical system. The image sensor includes a light receiving element, a gain processing circuit for the signal received by the light receiving element and photoelectrically converted, and an A / D conversion circuit for converting the signal into digital data.

[0061] The light receiving elements in the image sensor are configured as, for example, a CCD (Charge Coupled Device) type or a CMOS (Complementary Metal Oxide Semiconductor) type. The image sensor performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc. on the electrical signals obtained by photoelectrically converting the received light, and then performs A / D (Analog / Digital) conversion processing. The captured image signals are then output as digital data to the camera control unit 30.

[0062] The display unit 32 is composed of a display such as a liquid crystal display (LCD) panel or an organic electroluminescence (EL) panel and a display drive circuit for the display unit 32. The display unit 32 displays various types of information on the display screen based on instructions from the camera control unit 30. For example, the display unit 32 displays a preview image (monitor image) during shooting, various icons and controls as a user interface, and the like.

[0063] The operation unit 33 collectively refers to input devices that allow the user to input various operations. Specifically, these include physical operators such as a shutter button, and a touch panel that detects operations on operators (buttons, icons, sliders, etc.) displayed on the display of the display unit 32. The operation unit 33 detects user operations, and sends a signal corresponding to the input operation to the camera control unit 30.

[0064] The sensor unit 34 indicates various sensors mounted on the camera 1. The sensor unit 34 may be, for example, an IMU (inertial measurement unit), which may detect angular velocity using a three-axis angular velocity (gyro) sensor for pitch, yaw, and roll, and may detect acceleration using an acceleration sensor. The sensor unit 34 may also be equipped with an illuminance sensor. Various pieces of information detected by the sensor unit 34, such as illuminance information and IMU data, are supplied to the camera control unit 30.

[0065] The distance measurement unit 35 includes a light emitting unit, a light receiving unit, and a calculation control unit that controls light emission and performs calculations based on light receiving signals, and calculates the distance (depth) to the subject. The calculated distance information is supplied to the camera control unit 30.

[0066] The storage control unit 36 ​​performs processing to record image files such as still image data and video data, attribute information for the image files, thumbnail images, and the like, onto a recording medium such as a non-volatile memory. The storage control unit 36 ​​may take a variety of different forms. For example, the storage control unit 36 ​​may be implemented as a circuit unit that performs recording and playback on a flash memory built into the camera 1, or as a card recording and playback circuit that performs recording and playback access on a memory card (e.g., a portable flash memory) that can be attached to and detached from the camera 1. The storage control unit 36 ​​may also be implemented as a hard disk drive (HDD) or solid state drive (SSD) built into the camera 1.

[0067] The communication unit 38 comprehensively represents devices that communicate with external devices. For example, it represents components that perform network communication, mobile phone communication, wired communication, short-range wireless communication, etc. In the present embodiment, the communication unit 38 processes PTP over WebRTC communication and file upload communication.

[0068] The camera control unit 30 is configured by a microcomputer equipped with a CPU (Central Processing Unit). The memory unit 37 stores information used by the camera control unit 30 for processing. This memory unit 37 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, etc. The memory unit 37 may be a memory area built into the microcomputer chip that serves as the camera control unit 30, or may be configured by a separate memory chip.

[0069] The RAM in the memory unit 37 is used to temporarily store data, programs, etc. as a working area when the CPU of the camera control unit 30 processes various data. The ROM and flash memory (non-volatile memory) in the memory unit 37 are used to store the OS (Operating System) that the CPU uses to control each unit, application programs for various operations, various setting information, etc.

[0070] The camera control unit 30 executes programs stored in the ROM, flash memory, etc. of the memory unit 37, thereby controlling the entire camera 1. For example, the camera control unit 30 controls the zoom, focus, aperture adjustment, etc. of the optical system of the imaging unit 31, controls the shutter speed of the image sensor, processes signals on captured image data, performs imaging operations and recording operations in response to user operations, plays back recorded image files, etc.

[0071] The diagram shows an imaging processing unit 30a, a UI (User Interface) processing unit 30b, and a communication control unit 30c as functions provided by software in the camera control unit 30. The imaging processing unit 30a performs processing on captured image data captured by the imaging unit 31 and input, such as RAW correction processing, Y / C development processing, resolution conversion processing, codec processing, and file generation processing.

[0072] Image data as moving images or still images output after processing by the imaging processing unit 30a is stored in a storage medium by the storage control unit 36, or transmitted to an external device by the communication unit 38. The UI processing unit 30b detects display control of the display unit 32 and operation information by the operation unit 33, and executes processing accordingly. The communication control unit 30c is shown as having a function of controlling communication processing such as PTP over WebRTC communication and file upload communication.

[0073] The communication system of this embodiment can provide various services to the camera 1 configured as above.

[0074] 10 shows an example of the configuration of an information processing device 70 that can function as, for example, the relay server 3 or the separation processing module 4. Note that the database 5, the device linkage management system 6, the content management server 7, the upload server 8, the computer device 10, and the like can also be realized by the information processing device 70 described below.

[0075] The information processing device 70 can be configured as, for example, a dedicated workstation, a general-purpose personal computer, a mobile terminal device, or the like.

[0076] The CPU 71 of the information processing device 70 executes various processes according to programs stored in a ROM 72 or a nonvolatile memory unit 74 such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), or programs loaded from a storage unit 79 to a RAM 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes as appropriate. The CPU 71 realizes various control and calculation functions according to the programs.

[0077] In addition, a processor other than the CPU 71 may also be provided, such as a GPU (Graphics Processing Unit), a GPGPU (General-purpose computing on graphics processing units), or an AI (Artificial Intelligence) processor.

[0078] The CPU 71, ROM 72, RAM 73, and nonvolatile memory unit 74 are interconnected via a bus 83. The bus 83 is also connected to an input / output interface 75.

[0079] An input unit 76 consisting of operators and operation devices is connected to the input / output interface 75. For example, the input unit 76 may be various operators and operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, a remote controller, etc. A user operation is detected by the input unit 76, and a signal corresponding to the input operation is interpreted by the CPU 71.

[0080] The input / output interface 75 is also connected integrally or separately to a display unit 77 such as a liquid crystal panel or an organic EL panel, and an audio output unit 78 such as a speaker.

[0081] The display unit 77 displays various information as a user interface. The display unit 77 is configured, for example, by a display device provided in the housing of the information processing device 70 or a separate display device connected to the information processing device 70. The display unit 77 displays various images on the display screen based on instructions from the CPU 71. The display unit 77 also displays various operation menus, icons, messages, etc. based on instructions from the CPU 71.

[0082] The input / output interface 75 may be connected to a storage unit 79 configured with an SSD, HDD, or the like, or a communication unit 80 configured with a modem, or the like. The storage unit 79 can be used to store various types of data. A database can also be constructed in the storage unit 79. The communication unit 80 performs necessary communication processing. In the case of the information processing device 70 serving as the relay server 3, the communication unit 80 performs PTP-over-WebRTC communication with the camera 1, communication with the separation processing module 4, and the like. In the case of the information processing device 70 serving as the separation processing module 4, the communication unit 80 performs communication with the relay server 3, communication with the database 5, communication with the content management server 7, and the like.

[0083] A drive 82 is also connected to the input / output interface 75 as needed, and a removable recording medium 81 such as a flash memory, a memory card, a magnetic disk, an optical disk, or a magneto-optical disk is appropriately attached. The drive 82 allows data files such as image files and various computer programs to be read from the removable recording medium 81. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the audio output unit 78. In addition, the computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as needed.

[0084] In this information processing device 70, software can be installed via network communication by the communication unit 80 or via a removable recording medium 81. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, or the like.

[0085] Such an information processing device 70 can constitute the relay server 3, the separation processing module 4, or the database 5. The processing of each device described in Figures 6 and 7 can be realized by the hardware configuration of the information processing device 70 in Figure 10 and the software installed therein.

[0086] For example, when data relating to one of the cameras 1 with which a communication session is maintained is received from the relay server 3 that can maintain a communication session with multiple cameras 1, the CPU 71 can start a calculation process based on a software program and perform processing to generate communication data to be transmitted to the one camera 1 via the communication session by the relay server 3. Such an information processing device 70 functions as the separation processing module 4.

[0087] Furthermore, the CPU 71 can perform processing for maintaining a PTP communication session with a plurality of cameras 1 based on a software program. Such an information processing device 70 functions as the relay server 3.

[0088] 4. Summary and Modifications The communication system according to the above embodiment can achieve the following effects. The communication system according to the embodiment includes a relay server 3 as a first device capable of maintaining communication sessions with multiple imaging devices, and a separation processing module 4 as a second device. When the separation processing module 4 receives data related to one of the cameras 1 with which the relay server 3 is maintaining a communication session from the relay server 3, the separation processing module 4 starts a calculation process and performs processing to generate communication data to be transmitted to the camera 1 via the communication session by the relay server 3. In other words, the camera 1 and the relay server 3 execute a stateful communication session. In contrast, the separation processing module 4 and the relay server 3 communicate statelessly. The relay server 3 has a function for converting stateful communication with the camera 1 to stateless communication, and the calculation process for the actual communication is performed by the separation processing module 4. This makes it possible to maintain active PTP connections with a small number of relay servers 3, while the actual processing related to the PTP communication is performed by the separation processing module 4. Therefore, even if the number of supported cameras 1 increases significantly, an extremely small number of relay servers 3 can be used, thereby reducing system operating costs. For example, a system can be constructed in which one relay server 3 is sufficient to handle a large number of cameras 1, such as 500 or 1000 cameras.

[0089] In the embodiment, when an opportunity for communication with camera 1 arises, relay server 3 transmits a device ID indicating the communication destination device and a session ID indicating the communication session to separation processing module 4. Then, upon obtaining the device ID and session ID, separation processing module 4 starts an operation process corresponding to the communication session. In other words, separation processing module 4 does not keep a processing process resident for each session. The processing process is executed only at the start of a session or when there is an opportunity to receive data from camera 1 or to transmit data to camera 1. This prevents the processing load on separation processing module 4 from becoming excessive.

[0090] In the embodiment, an example has been described in which the separation processing module 4 performs processing to restore progress information of a corresponding communication session based on data read from an external storage device according to the session ID. The separation processing module 4 restores progress information of the session up to that point from the database 5 (database server 5a or cache server 5b) using the session ID acquired from the relay server 3. This makes it possible to generate response data according to the progress of the session up to that point, even in a stateless environment. In other words, the separation processing module 4 does not need to store progress information for each session in memory, which reduces the resource burden.

[0091] In the embodiment, an example has been described in which the separation processing module 4 starts an arithmetic process corresponding to the communication session in response to acquiring a device ID and a session ID from the relay server 3, and performs processing to generate response communication data to the communication data from the camera 1. That is, when the relay server 3 receives data from the camera 1, the separation processing module 4 receives this together with the session ID and device ID. In other words, after PTP initialization is complete, an arithmetic process is started in the separation processing module 4 and response processing is executed only when communication arrives from the camera 1. Therefore, the processing load on the separation processing module 4 corresponding to one PTP session can be made extremely small.

[0092] In the embodiment, the separation processing module 4 transmits the generated communication data for the camera 1 to the relay server 3. When the separation processing module 4 transmits the communication data to the relay server 3, the communication data is transmitted to the camera 1 via the PTP session maintained between the relay server 3 and the camera 1. In other words, the communication data generated by the separation processing module 4 can be transmitted to the camera 1 via a stateful PTP session.

[0093] In the embodiment, an example has been given in which separation processing module 4 acquires a content list from camera 1 to determine files to be backed up, and generates communication data notifying the determined files to be backed up and their upload destinations. This makes it possible to realize a communication system that provides a file upload service in camera 1 as a system including relay server 3 and separation processing module 4.

[0094] The upload destination is set to the destination (upload server 8) to which the camera 1 sends the backup target file without communicating with the relay server 3. As a result, the file from the camera 1 is sent to the upload server 8 by, for example, the HTTP PUT method, and images, etc., can be backed up. By not going through the relay server 3, the load of transferring the uploaded file on the relay server 3 is also reduced.

[0095] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0096] The present technology may also be configured as follows: (1) A communication system comprising: a first device capable of maintaining a communication session with a plurality of imaging devices; and a second device that, when the first device receives data related to one of the imaging devices with which the first device is maintaining a communication session from the first device, starts an arithmetic process and performs processing to generate communication data to be transmitted to the one imaging device via the communication session of the first device. (2) The communication system described in (1) above, in which, when an opportunity for communication with an imaging device arises, the first device transmits to the second device device identification information indicating a communication destination device and session identification information indicating a communication session, and the second device starts an arithmetic process corresponding to the communication session in response to obtaining the device identification information and the session identification information from the first device. (3) The communication system described in (2) above, in which the second device performs processing to restore progress information of the communication session based on data read from an external storage device in accordance with the session identification information. (4) The communication system according to any one of (1) to (3), wherein the first device transmits communication data from the imaging device to the second device together with device identification information indicating a communication destination device and session identification information indicating a communication session, and the second device, in response to acquiring the device identification information and the session identification information from the first device, starts an arithmetic process corresponding to the communication session and generates communication data in response to the communication data from the imaging device. (5) The communication system according to any one of (1) to (4), wherein the second device transmits the generated communication data for the imaging device to the first device. (6) The communication system according to any one of (1) to (5), wherein the second device acquires a content list of the imaging device and determines files to be backed up, and generates communication data notifying the determined files to be backed up and their upload destinations. (7) The communication system according to (6), wherein the upload destination is a destination to which the imaging device transmits the files to be backed up without communicating with the first device.(8) The communication system according to any one of (1) to (7) above, wherein PTP (Picture Transfer Protocol) commands are communicated between the first device and the imaging device over a communication path conforming to the WebRTC (Web Real-Time Communication) communication standard. (9) An information processing device that, when receiving data related to one of a plurality of imaging devices from a device capable of maintaining a communication session with the imaging devices and the data is transmitted to the one imaging device via the communication session held by the device, starts an arithmetic process and generates communication data to be transmitted to the one imaging device via the communication session held by the device. (10) A communication system having a first device and a second device, wherein the first device maintains a communication session with a plurality of imaging devices, and the second device, when receiving data related to one of the imaging devices and the data is transmitted to the one imaging device via the communication session held by the first device, starts an arithmetic process and generates communication data to be transmitted to the one imaging device via the communication session held by the first device.

[0097] REFERENCE SIGNS LIST 1 Camera 2 Computer device 3 Relay server 4 Separation processing module 4a Calculation function 5 Database 5a Database server 5b Cache server 6 Device linkage management system 7 Content management server 8 Upload server 9 Cloud server 10 Computer device 11 Application 70 Information processing device

Claims

1. A communication system comprising: a first device capable of maintaining communication sessions with a plurality of imaging devices; and a second device that, when the first device receives data related to one of the imaging devices with which the first device is maintaining a communication session, starts a calculation process and performs processing to generate communication data to be transmitted to the one imaging device via the communication session of the first device.

2. The communication system of claim 1, wherein when an opportunity for communication with an imaging device arises, the first device transmits device identification information indicating the communication destination device and session identification information indicating the communication session to the second device, and the second device starts a calculation process corresponding to the communication session in response to obtaining the device identification information and session identification information from the first device.

3. The communication system according to claim 2, wherein the second device performs a process of restoring progress information of the communication session based on data read from an external storage device in accordance with the session identification information.

4. The communication system described in claim 1, wherein the first device transmits communication data from the imaging device to the second device together with device identification information indicating the communication destination device and session identification information indicating the communication session, and the second device, upon obtaining the device identification information and session identification information from the first device, starts a calculation process corresponding to the communication session and performs processing to generate communication data in response to the communication data from the imaging device.

5. The communication system according to claim 1, wherein the second device transmits the generated communication data for the imaging device to the first device.

6. The communication system according to claim 1, wherein the second device performs the process of acquiring a content list of the imaging device and determining files to be backed up, and the process of generating communication data notifying the determined files to be backed up and the upload destination.

7. The communication system according to claim 6, wherein the upload destination is a destination to which the imaging device transmits the backup target file without communicating with the first device.

8. The communication system according to claim 1, wherein PTP (Picture Transfer Protocol) commands are communicated between the first device and the imaging device over a communication path conforming to the WebRTC (Web Real-Time Communication) communication standard.

9. An information processing device that, when receiving data related to one of the imaging devices that is maintaining a communication session from a device that can maintain a communication session with multiple imaging devices, starts a calculation process and performs processing to generate communication data to be transmitted to the one imaging device via the communication session by the device.

10. A communication processing method in which a communication system having a first device and a second device maintains a communication session between the first device and a plurality of imaging devices, and when the second device receives data related to one of the imaging devices with which the communication session is maintained from the first device, the second device starts a calculation process to generate communication data to be transmitted to the one imaging device via the communication session by the first device.

Images