Information processing system, imaging device, method, and program

Abstract

To transfer a yet-to-be transferred image captured by an imaging device.SOLUTION: An information processing system, which is an embodiment of the present invention, is an information processing system that includes an imaging device having a communication function and a server. The imaging device includes: an image state management unit that manages information indicating that an image captured by the imaging device has not been transferred yet; and an image transfer control unit that controls the yet-to-be transferred image to be re-transferred to the server according to any of a plurality of re-transfer triggers.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an imaging device, a method, and a program. [Background technology]

[0002] Conventionally, imaging devices capable of transferring captured images to an external device are known. However, such imaging devices may be unable to transfer images. For example, Patent Document 1 describes a method in which, if it is detected that the battery output voltage is below a predetermined voltage during file transmission, the power is turned off after saving unsent image data, and the saved unsent image data is sent when the power is turned back on. Summary of the Invention [Problem to be solved by the invention]

[0003] However, there are various reasons why an imaging device may be unable to transfer an image, and it has been desired to be able to retransfer an image regardless of the reason.

[0004] Therefore, an object of the present invention is to transfer images that have been captured by an imaging device but have not yet been transferred. [Means for solving the problem]

[0005] An information processing system according to an embodiment of the present invention includes an imaging device equipped with a communication function. , executes a job of a workflow application linked to the imaging device. and a server, wherein the imaging device manages information indicating that an image captured by the imaging device has not yet been transferred. If the execution of the job is completed normally, the image to be the target of the job is deleted from the list of untransferred images, and if the execution of the job is not completed normally, the image to be the target of the job is kept in the list of untransferred images. The image transfer control unit includes an image status management unit and an image transfer control unit that controls the retransfer of untransferred images to the server in response to one of a plurality of retransfer triggers.

[0006] According to the present invention, it is possible to transfer images that have been captured by an imaging device but have not yet been transferred. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram for explaining an overview of a service according to an embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an overall configuration according to an embodiment of the present invention. [Figure 3] 1 is a hardware configuration diagram of an information processing device (server) according to an embodiment of the present invention. [Figure 4] FIG. 1 is a diagram illustrating a hardware configuration of a device (omnidirectional imaging device) according to an embodiment of the present invention. [Figure 5] FIG. 2 is a functional block diagram of a server and a device according to an embodiment of the present invention. [Figure 6] FIG. 10 is a sequence diagram of a process for associating a device with an application according to an embodiment of the present invention. [Figure 7] FIG. 2 is a sequence diagram of a process related to a basic operation according to an embodiment of the present invention. [Figure 8] FIG. 10 is a sequence diagram of a management process for a transferred / non-transferred state of an image according to an embodiment of the present invention. [Figure 9] FIG. 10 is a sequence diagram of a retransfer process when a retransfer trigger occurs according to an embodiment of the present invention. [Figure 10] FIG. 10 is a sequence diagram of processing relating to a photography priority mode according to an embodiment of the present invention. [Figure 11] 10 is a diagram illustrating an example of a screen according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0009] FIG. 1 is a diagram illustrating an overview of a service according to an embodiment of the present invention. A user (a customer user, A1 in FIG. 1) uses a device 20 to access cloud services provided by servers 11 to 15 (hereinafter also referred to as server 10). A device partner (B in FIG. 1) registers a workflow (WF) application to a tenant (a group of devices using cloud services) to which the user belongs. A workflow (WF) application developer (C (cloud service partner) in FIG. 1) uses a WF application development tool to develop a workflow (WF) application that allows QR Code (registered trademark) linkage settings and sets the type of data that can be input (e.g., still images or videos). A2 is a customer administrator, and D is a device vendor. In the following, a cloud service provided by server 10 is used as an example of a service provided to device 20 via a network. However, the service used by device 20 is not limited to this, and any service that uses functions provided via a network may be used. For example, by installing server 10 within the same network as device 20, server 10 may provide on-premise services to device 20.

[0010] <System configuration> 2 is a diagram showing the overall configuration according to one embodiment of the present invention. A system (also referred to as an information processing system) 1 includes a server 10 and a device 20. The server 10 and the device 20 are connected to each other so as to be able to communicate with each other via a communication network N. Each of these will be described below.

[0011] <<Server>> A server (an example of an information processing device) 10 performs processing to link an application provided by a cloud service with a device 20 .

[0012] The devices described in the example are merely one of several computing environments for implementing the embodiments disclosed herein. In one embodiment, server 10 includes multiple computing devices, such as a server cluster, configured to communicate with each other via any type of communications link, including a network, shared memory, etc., and to perform the processes disclosed herein.

[0013] <<Device>> The device 20 is an imaging device with a communication function. The device 20 is a device used by a user who uses a cloud service. For example, the device 20 is a device (e.g., an omnidirectional imaging device) that has an imaging function but does not have a display function.

[0014] It should be noted that device 20 is not limited to a spherical imaging device as long as it is a device equipped with a communication function. Device 20 may be, for example, a PJ (Projector), an IWB (Interactive White Board: a white board with an electronic blackboard function that allows mutual communication), an output device such as digital signage, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, medical equipment, a network home appliance, an automobile (Connected Car), a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, a desktop PC, or the like.

[0015] <Hardware configuration> FIG. 3 is a hardware configuration diagram of an information processing device (in the case of a server) 10 according to one embodiment of the present invention.

[0016] As shown in FIG. 3, the server 10 is constructed by a computer, and as shown in FIG. 3, it is equipped with a CPU 1001, a ROM 1002, a RAM 1003, a HD 1004, an HDD (Hard Disk Drive) controller 1005, a display 1006, an external device connection I / F (Interface) 1007, a network I / F 1008, a data bus 1009, a keyboard 1010, a pointing device 1011, a DVD-RW (Digital Versatile Disk Rewritable) drive 1013, and a media I / F 1015.

[0017] Of these, the CPU 1001 controls the overall operation of the server 10. The ROM 1002 stores programs used to drive the CPU 1001, such as the IPL. The RAM 1003 is used as a work area for the CPU 1001. The HD 1004 stores various data, such as programs. The HDD controller 1005 controls the reading and writing of various data from and to the HD 1004 under the control of the CPU 1001. The display 1006 displays various information, such as a cursor, menu, window, text, or image. The external device connection I / F 1007 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories, printers, etc. The network I / F 1008 is an interface for data communication using the communication network N. The bus line 1009 is an address bus, data bus, etc. for electrically connecting the components, such as the CPU 1001, shown in FIG. 3.

[0018] The keyboard 1010 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 1011 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 1013 controls reading and writing of various data from a DVD-RW 1012, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 1015 controls reading and writing (storing) of data from a recording medium 1014, such as a flash memory.

[0019] FIG. 4 is a diagram showing the hardware configuration of a device (in the case of an omnidirectional imaging device) 20 according to an embodiment of the present invention.

[0020] In the following, the omnidirectional imaging device 20 is assumed to be an omnidirectional (all-round) omnidirectional imaging device using two imaging elements, but the number of imaging elements may be any number greater than or equal to two. Also, it does not necessarily have to be a device dedicated to omnidirectional imaging; a regular digital camera, smartphone, or the like may be equipped with an omnidirectional imaging unit as a retrofit, so that it has substantially the same functions as the omnidirectional imaging device 20.

[0021] As shown in FIG. 4 , the omnidirectional imaging device 20 includes an imaging unit 2001, an image processing unit 2004, an imaging control unit 2005, a microphone 2006, a sound processing unit 2007, a CPU (Central Processing Unit) 2009, a ROM (Read Only Memory) 2010, an SRAM (Static Random Access Memory) 2011, a DRAM (Dynamic Random Access Memory) 2012, an operation unit 2013, an external device connection I / F 2014, a communication unit 2015, an antenna 2015a, an electronic compass, a gyro sensor, an acceleration sensor, and a concave terminal for a Micro USB.

[0022] Of these, imaging unit 2001 includes wide-angle lenses (so-called fisheye lenses) 2002a and 2002b, each with a field angle of 180° or more for forming a hemispherical image, and two imaging elements 2003a and 2003b provided corresponding to each wide-angle lens. Imaging elements 2003a and 2003b include image sensors such as CMOS (Complementary Metal Oxide Semiconductor) sensors or CCD (Charge Coupled Device) sensors that convert optical images captured by fisheye lenses 2002a and 2002b into electrical image data signals for output, a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks for the image sensors, and a group of registers that set various commands and parameters required for the operation of the imaging elements.

[0023] The imaging elements 2003a and 2003b of the imaging unit 2001 are each connected to the image processing unit 2004 via a parallel I / F bus. On the other hand, the imaging elements 2003a and 2003b of the imaging unit 2001 are connected to the imaging control unit 2005 via a serial I / F bus (such as an I2C bus). The image processing unit 2004, the imaging control unit 2005, and the sound processing unit 2007 are connected to the CPU 2009 via a bus 2008. Furthermore, the bus 2008 is also connected to a ROM 2010, an SRAM 2011, a DRAM 2012, an operation unit 2013, an external device connection I / F (Interface) 2014, a communication unit 2015, an acceleration / direction sensor 2016, and the like.

[0024] The image processing unit 2004 takes in the image data output from the image sensors 2003a and 2003b via a parallel I / F bus, performs predetermined processing on each piece of image data, and then synthesizes the image data to create equirectangular projection image data.

[0025] The imaging control unit 2005 generally sets commands and the like in the registers of the imaging elements 2003a and 2003b using an I2C bus, with the imaging control unit 2005 acting as a master device and the imaging elements 2003a and 2003b acting as slave devices. Necessary commands and the like are received from the CPU 2009. The imaging control unit 2005 also uses the I2C bus to retrieve status data and the like from the registers of the imaging elements 2003a and 2003b and send it to the CPU 2009.

[0026] Furthermore, the imaging control unit 2005 instructs the image capturing elements 2003a and 2003b to output image data when the shutter button on the operation unit 2013 is pressed. Some omnidirectional imaging devices 20 have a preview display function or a function for displaying moving images on a display (for example, a smartphone display). In this case, the image data is output continuously from the image capturing elements 2003a and 2003b at a predetermined frame rate (frames / minute).

[0027] As will be described later, the imaging control unit 2005 also functions as a synchronization control means for synchronizing the output timing of image data from the imaging elements 2003a and 2003b in cooperation with the CPU 2009. Note that, although the omnidirectional imaging device 20 is not provided with a display in this embodiment, a display unit may be provided.

[0028] The microphone 2006 converts sound into sound (signal) data. The sound processing unit 2007 takes in the sound data output from the microphone 2006 via the I / F bus and performs predetermined processing on the sound data.

[0029] The CPU 2009 controls the overall operation of the omnidirectional imaging device 20 and executes necessary processes. The ROM 2010 stores various programs for the CPU 2009. The SRAM 2011 and DRAM 2012 are work memories that store programs executed by the CPU 2009, data in the middle of processing, etc. In particular, the DRAM 2012 stores image data in the middle of processing by the image processing unit 2004 and data of processed equirectangular projection images.

[0030] The operation unit 2013 is a general term for operation buttons such as a shutter button, etc. The user operates the operation unit 2013 to input various shooting modes, shooting conditions, and the like.

[0031] The external device connection I / F 2014 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a PC (Personal Computer). The data of the equirectangular projection image stored in the DRAM 2012 is recorded on an external medium via the network I / F 2014, or transmitted to an external terminal (device) such as a smartphone via the network I / F 2014 as needed.

[0032] The communication unit 2015 communicates with an external terminal (device) such as a smartphone by short-range wireless communication technology such as Wi-Fi, NFC (Near Field Communication), or Bluetooth (registered trademark) via an antenna 2015a provided in the omnidirectional imaging device 20. The communication unit 2015 can also transmit data of the equirectangular projection image to the external terminal (device) such as a smartphone.

[0033] The acceleration and orientation sensor 2016 calculates the orientation of the omnidirectional imaging device 20 from the Earth's magnetism and outputs orientation information. This orientation information is an example of related information (metadata) conforming to Exif and is used for image processing such as image correction of captured images. The related information also includes data such as the date and time the image was captured and the data size of the image data. The acceleration and orientation sensor 2016 is a sensor that detects angle changes (roll angle, pitch angle, yaw angle) associated with the movement of the omnidirectional imaging device 20. The angle changes are an example of related information (metadata) conforming to Exif and are used for image processing such as image correction of captured images. The acceleration and orientation sensor 2016 is a sensor that detects acceleration in three axial directions. The omnidirectional imaging device 20 calculates the attitude (angle with respect to the direction of gravity) of its own device (the omnidirectional imaging device 20) based on the acceleration detected by the acceleration and orientation sensor 2016. The provision of the acceleration and orientation sensor 2016 in the omnidirectional imaging device 20 improves the accuracy of image correction.

[0034] <Function block> FIG. 5 is a functional block diagram of the server 10 and the device 20 according to an embodiment of the present invention.

[0035] <<Server>> The server 10 includes an application setting unit 101, a device registration unit 102, a collaboration unit 103, a storage unit 104, and a WF application 105. The server 10 also functions as the application setting unit 101, the device registration unit 102, the collaboration unit 103, and the WF application 105 by executing a program.

[0036] The application setting unit 101 performs settings for the WF application 105. For example, the application setting unit 101 performs account settings for logging in to an online storage service.

[0037] The device registration unit 102 provides a service for connecting the device 20 to other devices and cloud services.

[0038] The linking unit 103 manages the association between the WF application 105 and the device 20, and upon receiving a request from the device 20 to execute a WF (workflow) job, requests the WF application 105 to execute the job.

[0039] The storage unit 104 is a storage in which images are temporarily stored.

[0040] The WF (workflow) application 105 is an application that combines functions to realize a workflow, and defines what input is to be processed, how it is to be processed, and how it is to be output.

[0041] <<Device>> The device 20 includes an image capturing unit 201, an image storage unit 202, an image transfer control unit 203, an image status management unit 204, a communication unit 205, and a setting value storage unit 206. The device 20 also functions as the image capturing unit 201, the image transfer control unit 203, the image status management unit 204, and the communication unit 205 by executing a program.

[0042] The photographing unit 201 takes a photograph when the shutter button of the device 20 is pressed.

[0043] The image storage unit 202 is a storage unit for saving images captured by the image capturing unit 201. Note that images transferred to the server 10 are deleted.

[0044] The image transfer control unit 203 controls the transfer of images. Specifically, the image transfer control unit 203 controls the retransfer of untransferred images to the server 10 in response to one of a plurality of retransfer triggers.

[0045] The image status management unit 204 manages information indicating that the images stored in the image storage unit 202 (that is, the images captured by the imaging device 20) have not yet been transferred.

[0046] In response to a request from the image transfer control unit 203, the communication unit 205 transfers an image or notifies the end of the image transfer.

[0047] The setting value storage unit 206 is a storage unit that stores the setting values for the operation of the plugin (specifically, the threshold for switching between ON and OFF of the shooting priority mode).

[0048] <Processing method> FIG. 6 is a sequence diagram of the linking process between the device and the application according to an embodiment of the present invention. It will be described by dividing it into a <WF app registration phase> and a <WF app execution phase>.

[0049] <WF app registration phase> The process of linking and registering the workflow (WF) application program 105 provided by the cloud service to the device 20 will be described. Note that the workflow (WF) application program 105 is an application program that performs predetermined processing in the server 10.

[0050] Note that the developer of the WF application program, 105 (C (cloud service partner) in FIG. 1), develops the WF application program 105 in which the permission for QR code linking setting and the type of inputable data (such as still image or moving image) are set using a WF application development tool. Also, the device partner (B in FIG. 1) registers the WF application program 105 in the tenant (group of devices using the cloud service) to which the user belongs.

[0051] In step 11 (S11), the server (app setting unit) 10 generates and distributes a code (for example, a QR code) including information on the application program provided by the cloud service and information used for registering the device.

[0052] Specifically, the user logs in to a site (IoT site in Figure 1) where applications are configured using a PC (personal computer), selects the WF application 105 to be linked to the device 20 (device plug-in in Figure 1), and has a QR code issued.

[0053] In step 12 (S12), it is assumed that the shutter button of the device 20 is pressed.

[0054] In step 13 (S13), the device 20 reads the QR code that was distributed by the server (application setting unit) 10 in S11 and displayed on the PC.

[0055] In step 14 (S14), device 20 acquires the connection destination of the site for acquiring information for identifying the device, a token (access token) for accessing the site, and information about the application (for example, the application ID (here, "App 1") and the input type (here, "still image")) contained in the QR code.

[0056] In step 15 (S15), the device 20 requests the server (device registration unit) 10 to register the device 20. Specifically, the device 20 notifies the server 10 of the information used for device registration, which was acquired by reading the code in S14 (more specifically, the device 20 sends the serial number of the device 20 to the connection destination using an access token).

[0057] In step 16 (S16), the server (device registration unit) 10 notifies the device 20 of information for identifying the device (device ID (here, "device A")) and a certificate.

[0058] In step 17 (S17), the device 20 requests the server (cooperation unit) 10 to register by linking the device 20 to the application. Specifically, the device 20 notifies the server (cooperation unit) 10 of the application ID (that is, "App 1"), the input type (that is, "still image"), and the device ID (that is, "Device A") together with the certificate of S16. Thereafter, the server (cooperation unit) 10 registers by linking the application ID, the input type, and the device ID.

[0059] <WF Application Execution Phase> The process of executing the workflow (WF) application will be described.

[0060] Assume that the shutter button of the device 20 is pressed in step 21 (S21).

[0061] In step 22 (S22), the device 20 obtains a URL (Uniform Resource Locator). Specifically, the device 20 obtains the URL from the server (device registration unit) 10.

[0062] [[ID=I7]] In step 23 (S23), the device 20 uploads the image captured in S21 to the URL of S22.

[0063] In step 24 (S24), the device 20 notifies the server (device registration unit) 10 of the shooting. Specifically, the device 20 notifies the server (device registration unit) 10 of the device ID (that is, "Device A") and the URL.

[0064] In step 25 (S25), the server (device registration unit) 10 sends the device ID and the URL obtained in SZ4 to the server (cooperation unit) 10.

[0065] In step 26 (S26), the server (cooperation unit) 10 downloads the image uploaded to the URL obtained in S25.

[0066] In step 27 (S27), the server (collaboration unit) 10 causes the workflow (WF) application to execute the job. Specifically, the server (collaboration unit) 10 notifies the application (i.e., "application 1") associated with the device ID of the input type (i.e., "still image").

[0067] In step 28 (S28), the WF application 105 uploads the image to an online storage service. Note that uploading and saving the image to an online storage service is an example of a service provided via a network.

[0068] FIG. 7 is a sequence diagram of a process related to a basic operation according to one embodiment of the present invention.

[0069] In step 101 (S101), when the shutter button of the device 20 is pressed, the photographing unit 201 of the device 20 takes a photograph.

[0070] In step 102 (S102), the photographing unit 201 of the device 20 stores the image photographed in S101 in the image storage unit 202 of the device 20.

[0071] In step 103 (S103), the photographing unit 201 of the device 20 notifies the image transfer control unit 203 of the device 20 that photographing has been performed. Specifically, the photographing unit 201 notifies the image transfer control unit 203 of the image ID.

[0072] In step 104 (S104), in response to the notification in S103, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the file name.

[0073] In step 105 (S105), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to acquire the URL of the upload destination of the image.

[0074] In step 106 (S106), the communication unit 205 of the device 20 acquires the URL requested in S105.

[0075] Here, the URL of the upload destination of the image will be explained. The communication unit 205 of the device 20 acquires the URL by specifying the file name of the image. In other words, a URL is acquired for each image file to be uploaded. The URL of the upload destination of the image is a temporary area, and becomes an invalid URL after a certain period of time has passed, and is deleted if an image has been uploaded.

[0076] In step 107 (S107), the communication unit 205 of the device 20 uploads the image to the storage unit 104 of the server 10 (URL of S106).

[0077] In step 108 (S108), when the upload in S107 is completed, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to execute a WF (workflow) job. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the URL.

[0078] In step 109 (S109), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to execute a WF (workflow) job. Specifically, the communication unit 205 notifies the device registration unit 102 of the URL and the device ID.

[0079] In step 110 (S110), the device registration unit 102 of the server 10 requests the collaboration unit 103 of the server 10 to execute a WF (workflow) job. Specifically, the device registration unit 102 notifies the collaboration unit 103 of the URL and the device ID.

[0080] In step 111 (S111), the collaboration unit 103 of the server 10 downloads an image from the URL that is a parameter of the request to execute a WF (workflow) job.

[0081] In step 112 (S112), the cooperation unit 103 of the server 10 requests the WF application 105 to execute a job. Specifically, the cooperation unit 103 notifies the WF application 105 of the app ID and the image.

[0082] The cooperation unit 103 of the server 10 manages which device 20 is associated with which WF (workflow) application through the app association registration performed in the <WF app registration phase> of FIG. 6, and causes the WF (workflow) application associated with the device 20 that has requested the execution of the WF (workflow) job to execute.

[0083] In step 113 (S113), when the WF application 105 receives an image from the device 20, it uploads the image from the device 20 to the online storage service according to the workflow of uploading the image to the online storage service.

[0084] In step 114 (S114), the cooperation unit 103 of the server 10 returns a WF job acceptance response indicating whether the acceptance of the execution of the WF (workflow) job has been successful. If the acceptance is successful, a job ID for identifying the job is notified as a parameter.

[0085] In step 115 (S115), the device registration unit 102 of the server 10 notifies the WF job acceptance response to the communication unit 205 of the device 20.

[0086] In step 116 (S116), the communication unit 205 of the device 20 notifies the WF job acceptance response to the image transfer control unit 203 of the device 20.

[0087] In step 117 (S117), the image transfer control unit 203 of the device 20 requests the WF application 105 to acquire the job status at any interval in response to the WF job acceptance response, and monitors whether the job is completed normally. Specifically, the image transfer control unit 203 notifies the WF application 105 of the job ID.

[0088] In step 118 (S118), the image transfer control unit 203 of the device 20 acquires the status of the job. Steps S117 to S118 are repeated until the job is completed.

[0089] FIG. 8 is a sequence diagram of a management process for the transferred / non-transferred state of an image according to one embodiment of the present invention.

[0090] In step 201 (S201), when the shutter button of the device 20 is pressed, the photographing unit 201 of the device 20 takes a photograph.

[0091] In step 202 (S202), the photographing unit 201 of the device 20 stores the image photographed in S201 in the image storage unit 202 of the device 20.

[0092] In step 203 (S203), the photographing unit 201 of the device 20 notifies the image transfer control unit 203 of the device 20 that photographing has been performed. Specifically, the photographing unit 201 notifies the image transfer control unit 203 of the image ID.

[0093] In step 204 (S204), the image transfer control unit 203 of the device 20 requests the image status management unit 204 of the device 20 to manage the transferred / non-transferred status of the image. Specifically, the image transfer control unit 203 notifies the image status management unit 204 of the image ID.

[0094] In step 205 (S205), the image status management unit 204 of the device 20 adds the image ID of S204 to a list of images that have not yet been transferred.

[0095] In step 206 (S206), in response to the notification in S203, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the file name.

[0096] In step 207 (S207), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to acquire the URL of the upload destination of the image.

[0097] In step 208 (S208), the communication unit 205 of the device 20 acquires the URL requested in S207.

[0098] Here, the URL of the upload destination of the image will be explained. The communication unit 205 of the device 20 acquires the URL by specifying the file name of the image. In other words, a URL is acquired for each image file to be uploaded. The URL of the upload destination of the image is a temporary area, and becomes an invalid URL after a certain period of time has passed, and is deleted if an image has been uploaded.

[0099] In step 209 (S209), the communication unit 205 of the device 20 uploads the image to the storage unit 104 of the server 10 (URL of S208).

[0100] In step 210 (S210), when the upload in S209 is completed, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to execute a WF (workflow) job. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the URL.

[0101] In step 211 (S211), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to execute a WF (workflow) job. Specifically, the communication unit 205 notifies the device registration unit 102 of the URL and the device ID.

[0102] In step 212 (S212), the device registration unit 102 of server 10 requests the cooperation unit 103 of server 10 to execute a WF (workflow) job. Specifically, the device registration unit 102 notifies the cooperation unit 103 of the URL and the device ID.

[0103] In step 213 (S213), the cooperation unit 103 of server 10 downloads an image from the URL, which is a parameter for the request to execute the WF (workflow) job.

[0104] In step 214 (S214), the cooperation unit 103 of server 10 requests the WF application 105 to execute a job. Specifically, the cooperation unit 103 notifies the WF application 105 of the app ID and the image.

[0105] The cooperation unit 103 of server 10 manages which device 20 is associated with which WF (workflow) application through the app association registration performed in the <WF app registration phase> of FIG. 6, and causes the WF (workflow) application associated with the device 20 that has requested the execution of the WF (workflow) job to be executed.

[0106] In step 215 (S215), when the WF application 105 receives an image from the device 20, it uploads the image from the device 20 to the online storage service according to the workflow of uploading the image to the online storage service.

[0107] In step 216 (S216), the cooperation unit 103 of server 10 returns a WF job acceptance response indicating whether the acceptance of the execution of the WF (workflow) job was successful. If the acceptance is successful, the job ID for identifying the job is notified as a parameter.

[0108] In step 217 (S217), the device registration unit 102 of server 10 notifies the WF job acceptance response to the communication unit 205 of device 20.

[0109] In step 218 (S218), the communication unit 205 of the device 20 notifies the image transfer control unit 203 of the device 20 of the WF job acceptance response.

[0110] In step 219 (S219), the image transfer control unit 203 of the device 20 requests the WF application 105 to acquire the job status at any interval in response to the WF job acceptance response, and monitors whether the job is completed normally. Specifically, the image transfer control unit 203 notifies the WF application 105 of the job ID.

[0111] In step 220 (S220), the image transfer control unit 203 of the device 20 acquires the job status. S219 to S220 are repeated until the job is completed.

[0112] In step 221 (S221), when the execution of the WF (workflow) job is completed normally, the image transfer control unit 203 of the device 20 requests the image status management unit 204 to delete the image that is the target of the job from the list of untransferred images.

[0113] In step 222 (S222), the image status management unit 204 of the device 20 deletes the image from the list of untransferred images in response to the request of S221. Note that if the execution of the WF (workflow) job does not end normally due to a network error, a failure, an error or a failure on the cloud service side, or the like, the image is not deleted from the list of untransferred images, and the untransferred state is maintained.

[0114] FIG. 9 is a sequence diagram of a retransfer process when a retransfer trigger occurs according to an embodiment of the present invention.

[0115] Assume that a trigger for retransmission occurs in step 301 (S301).

[0116] Here, the trigger for retransfer will be described. For example, the trigger for retransfer is the next time an image is captured (i.e., when an image is captured after an untransferred image has occurred), the next time a plug-in is started (i.e., when a plug-in is started after an untransferred image has occurred), when a network connection is detected (i.e., when a connection between the network and the device 20 is detected after an untransferred image has occurred), when the imaging device 20 is operated (i.e., when the imaging device 20 is operated after an untransferred image has occurred), when the remaining capacity of the image storage unit 202 of the imaging device 20 falls below a threshold (i.e., when the remaining capacity of the image storage unit 202 falls below a threshold after an untransferred image has occurred), or when a transfer request is received from the server 10 (i.e., when a transfer request is received from the server 10 after an untransferred image has occurred).

[0117] In step 302 (S302), the image transfer control unit 203 of the device 20 requests the image status management unit 204 to acquire a list of images that have not yet been transferred.

[0118] In step 303 (S303), the image transfer control unit 203 of the device 20 acquires a list of images that have not yet been transferred.

[0119] Thereafter, steps 304 to 320 are repeated for the number of images that have not yet been transferred, and the images that have not yet been transferred are transferred in sequence.

[0120] In step 304 (S304), the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the device 20 of the file name.

[0121] In step 305 (S305), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to acquire the URL of the upload destination of the image.

[0122] In step 306 (S306), the communication unit 205 of the device 20 acquires the URL requested in S305.

[0123] Here, the URL of the upload destination of the image file will be explained. The communication unit 205 of the device 20 acquires the URL by specifying the file name of the image. In other words, a URL is acquired for each image file to be uploaded. The URL of the upload destination of the image is a temporary area, and becomes an invalid URL after a certain period of time has passed, and is deleted if an image has been uploaded.

[0124] In step 307 (S307), the communication unit 205 of the device 20 uploads the image to the storage unit 104 of the server 10 (URL of S306).

[0125] In step 308 (S308), when the upload in S107 is completed, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to execute a WF (workflow) job. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the URL.

[0126] In step 309 (S309), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to execute a WF (workflow) job. Specifically, the communication unit 205 notifies the device registration unit 102 of the device 20 of the URL and the device ID.

[0127] In step 310 (S310), the device registration unit 102 of the server 10 requests the collaboration unit 103 of the server 10 to execute a WF (workflow) job. Specifically, the device registration unit 102 notifies the collaboration unit 103 of the URL and the device ID.

[0128] In step 311 (S311), the collaboration unit 103 of the server 10 downloads an image from the URL that is a parameter of the request to execute a WF (workflow) job.

[0129] In step 312 (S312), the cooperation unit 103 of the server 10 requests the WF application 105 to execute a job. Specifically, the cooperation unit 103 notifies the WF application 105 of the app ID and the image.

[0130] The cooperation unit 103 of the server 10 manages which device 20 is associated with which WF (workflow) application through the app association registration performed in the <WF app registration phase> of FIG. 6, and causes the WF (workflow) application associated with the device 20 that has requested the execution of the WF (workflow) job to execute.

[0131] In step 313 (S313), when the WF application 105 receives an image from the device 20, it uploads the image from the device 20 to the online storage service according to the workflow of uploading the image to the online storage service.

[0132] In step 314 (S314), the cooperation unit 103 of the server 10 returns a WF job acceptance response indicating whether the acceptance of the execution of the WF (workflow) job was successful. If the acceptance is successful, the job ID for identifying the job is notified as a parameter.

[0133] In step 315 (S315), the device registration unit 102 of the server 10 notifies the communication unit 205 of the device 20 of the WF job acceptance response. <00​​​​​​In step 317 (S317), the image transfer control unit 203 of the device 20, in response to the WF job acceptance response, requests the WF application 105 to acquire the job status at arbitrary intervals and monitors whether the job is completed normally. Specifically, the image transfer control unit 203 notifies the WF application 105 of the job ID.

[0136] In step 318 (S318), the image transfer control unit 203 of the device 20 acquires the status of the job. Steps S317 to S318 are repeated until the job is completed.

[0137] In step 319 (S319), when the execution of the WF (workflow) job is completed successfully, the image transfer control unit 203 of the device 20 requests the image status management unit 204 to delete the image that is the target of the job from the list of untransferred images.

[0138] In step 318 (S318), the image transfer control unit 203 of the device 20 deletes the image from the list of untransferred images in response to the request in S319.

[0139] FIG. 10 is a sequence diagram of processing relating to the photography priority mode (a mode in which images are not transferred after being photographed) according to one embodiment of the present invention.

[0140] In step 401 (S401), when the shutter button of the device 20 is pressed, the photographing unit 201 of the device 20 takes a photograph.

[0141] In step 402 (S402), the photographing unit 201 of the device 20 stores the image photographed in S401 in the image storage unit 202 of the device 20.

[0142] In step 403 (S403), the photographing unit 201 of the device 20 notifies the image transfer control unit 203 of the device 20 that photographing has been performed. Specifically, the photographing unit 201 notifies the image transfer control unit 203 of the image ID.

[0143] In step 404 (S404), the image transfer control unit 203 of the device 20 requests the image status management unit 204 of the device 20 to manage the transferred / non-transferred status of the image. Specifically, the image transfer control unit 203 notifies the image status management unit 204 of the image ID.

[0144] In step 405 (S405), the image status management unit 204 of the device 20 adds the image ID of S404 to the list of images that have not yet been transferred.

[0145] In step 406 (S406), the image transfer control unit 203 of the device 20 refers to the remaining battery power of the device 20.

[0146] In step 407 (S407), the image transfer control unit 203 of the device 20 requests the setting value storage unit 206 to acquire the threshold setting value for the photography priority mode (note that the device 20 operates in photography priority mode when the remaining battery charge falls below the threshold, and transfers untransferred images when the remaining battery charge exceeds the threshold due to charging).

[0147] In step 408 (S408), the image transfer control unit 203 of the device 20 acquires the set value of the threshold for the photography priority mode.

[0148] In step 409 (S409), the image transfer control unit 203 of the device 20 compares the remaining battery level referenced in S406 with the threshold setting value for the photography priority mode acquired in S408.

[0149] In step 410 (S410), if the comparison in S409 shows that the mode is not the photography priority mode, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image (the operation thereafter is as described in FIG. 8). On the other hand, if the comparison in S409 shows that the mode is the photography priority mode, the image is not transferred, and the untransferred state is maintained.

[0150] <Screen example> Fig. 11 is an example of a screen according to an embodiment of the present invention. Fig. 11 is a setting screen for a plug-in that captures an image and executes a workflow. Settings are made from an application on a smartphone, tablet, or the like that is connected to the device 20 wirelessly (by LAN, Bluetooth, or the like). The value set by the application (threshold setting value) is saved in a setting value storage unit 206 provided in the device 20.

[0151] The user can set a threshold for the capture-priority mode (i.e., the condition for switching to the capture-priority mode) on the screen shown in Fig. 11. The device 20 operates in the capture-priority mode when the remaining battery power falls below the threshold, and transfers any untransferred images when the remaining battery power exceeds the threshold due to charging.

[0152] <Effects> Thus, in one embodiment of the present invention, the imaging device can retransfer images that could not be transferred for some reason at a timing desired by the user (for example, in response to the occurrence of one of multiple retransfer triggers (i.e., when the cause of the image not being able to be transferred is resolved)).

[0153] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each function described above.

[0154] The present invention is not limited to the configurations described in the above embodiments, but may be combined with other elements, etc. These aspects can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0155] 1 System 10 Servers 20 devices 101 App Settings 102 Device Registration Unit 103 Collaboration Department 104 Storage section 105 WF App 201 Photography Department 202 Image storage unit 203 Image transfer control unit 204 Image Status Management Unit 205 Communications Department 206 Setting value memory section [Prior art documents] [Patent documents]

[0156] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-020452

Claims

1. An information processing system including an imaging device having a communication function and a server that executes a job of a workflow application linked to the imaging device, The imaging device is managing information indicating that the image captured by the imaging device has not yet been transferred; an image status management unit that deletes the image that is the target of the job from a list of untransferred images when the execution of the job is completed normally, and that keeps the image that is the target of the job in the list of untransferred images when the execution of the job is not completed normally; an image transfer control unit that controls the retransfer of untransferred images to the server in response to one of a plurality of retransfer triggers; An information processing system comprising:

2. The information processing system of claim 1, wherein the trigger for the re-transfer is the next time a photograph is taken, the next time a plug-in is started, when a network connection is detected, when the imaging device is operated, when the remaining capacity of the image storage unit of the imaging device falls below a threshold, or when a transfer request is received from the server.

3. 3. The information processing system according to claim 1, wherein the imaging device operates in a capture priority mode in which no image is transferred after capture when the remaining battery charge of the imaging device falls below a threshold.

4. The information processing system according to claim 3 , wherein the condition for switching to the photography priority mode is set by a user.

5. managing information indicating that an image captured by the imaging device has not yet been transferred; an image status management unit that, when the server has normally completed execution of a job of a workflow application linked to the imaging device, deletes the image that is the target of the job from a list of untransferred images, and, when the server has not normally completed execution of the job, keeps the image that is the target of the job in the list of untransferred images; an image transfer control unit that controls the retransfer of untransferred images in response to one of a plurality of retransfer triggers; An imaging device comprising:

6. A method performed by an imaging device, comprising: managing information indicating that the image captured by the imaging device has not yet been transferred; a step of deleting an image that is the target of a job from a list of untransferred images when the server has normally completed execution of a job of a workflow application linked to the imaging device, and retaining the image that is the target of the job in the list of untransferred images when the server has not normally completed execution of the job; a step of controlling the retransmission of untransmitted images in response to one of a plurality of triggers for retransmission; A method comprising:

7. Imaging device managing information indicating that the image captured by the imaging device has not yet been transferred; an image status management unit that, when the server has normally completed execution of a job of a workflow application linked to the imaging device, deletes the image that is the subject of the job from a list of untransferred images, and, when the server has not normally completed execution of the job, keeps the image that is the subject of the job in the list of untransferred images; an image transfer control unit that controls the retransfer of untransferred images in response to one of a plurality of retransfer triggers; A program to function as a

Images