Information processing system and program

Abstract

To enable an external device to access actual data generated by a device that cannot directly communicate with the external device.SOLUTION: A cloud server 22 includes a CPU 28A and a storage 28D, the storage 28D recording metadata 44M associated with image data 44 generated by an image forming device 32 that communicates with the CPU 28A, in which when the metadata 44M is requested by a machine learning device 50, the CPU 28A transmits the metadata 44M recorded in the storage 28D to the machine learning device 50 to enable the machine learning device 50 to access the image data 44.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to an information processing system and a program. [Background technology]

[0002] For example, Patent Document 1 discloses a setting screen for making settings related to new image data to be transmitted, which reflects settings stored in a storage means that are linked to the same type of image data as the new image data to be transmitted and that are linked to information of the same user as the user making the settings related to the new image data to be transmitted. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-163983 Summary of the Invention [Problem to be solved by the invention]

[0004] It is an object of the present disclosure to make actual data generated by a device that is not capable of communicating directly with an external device available to the external device. [Means for solving the problem]

[0005] An information processing system of a first aspect comprises one or more processors and one or more recording units that record attribute data associated with actual data generated by a device communicating with the processor, and when the attribute data is requested by an external device, the processor transmits the attribute data recorded in the recording unit to the external device, making the actual data available to the external device.

[0006] In a second aspect of the information processing system, in the information processing system described in the first aspect, when the external device requests the actual data related to the attribute data, the processor makes the actual data available to the external device.

[0007] An information processing system of a third aspect is the information processing system according to the second aspect, wherein the processor receives the actual data from the device that stores the actual data, and transmits the actual data to the external device.

[0008] An information processing system of a fourth aspect is the information processing system according to the second or third aspect, wherein the processor causes the device that stores the actual data to transmit the actual data to the external device.

[0009] An information processing system of a fifth aspect is the information processing system according to any one of the second to fourth aspects, wherein the processor records a history of requests for the actual data from the external device.

[0010] In a sixth aspect of the information processing system, in the information processing system described in the first aspect, when the processor receives a request for the attribute data from an external device, it transmits to the external device a method for accessing the storage area in which the actual data is stored together with the attribute data.

[0011] In a seventh aspect of the information processing system according to the sixth aspect, the processor records a history of requests for the attribute data from the external device.

[0012] An information processing system of an eighth aspect is the information processing system according to any one of the first to seventh aspects, wherein the processor records the attribute data in the recording unit each time the device generates the actual data.

[0013] A ninth aspect of the information processing system is an information processing system described in any one of the first to eighth aspects, wherein, when the actual data related to the attribute data is erased, the processor adds a history of the deletion of the actual data to the attribute data.

[0014] The program of the tenth aspect causes a computer to execute a process in which, when attribute data is requested from an external device, the attribute data recorded in the recording unit is transmitted to the external device, and actual data associated with the attribute data is made available to the external device. Effect of the Invention

[0015] According to the information processing system of the first aspect, actual data generated by a device that cannot directly communicate with an external device can be made available to the external device.

[0016] According to the information processing system of the second aspect, it is possible to make actual data stored in a device that cannot directly communicate with an external device available to the external device.

[0017] According to the information processing system of the third aspect, it is possible to make actual data stored in a device that cannot directly communicate with an external device available to the external device.

[0018] According to the information processing system of the fourth aspect, the time until the external device receives the actual data can be shortened, compared to a case in which the processor receives the actual data and then transmits it to the external device.

[0019] According to the information processing system of the fifth aspect, it is possible to inform the external device that the external device is repeatedly requesting the same actual data.

[0020] According to the information processing system of the sixth aspect, the time until the external device can use the actual data can be shortened, compared to a case in which the processor receives the actual data and then transmits it to the external device.

[0021] According to the information processing system of the seventh aspect, it is possible to notify the external device that the external device is repeatedly requesting the same actual data.

[0022] According to the information processing system of the eighth aspect, the actual data becomes available to the external device earlier than in a case where the attribute data is generated at a different time from the actual data.

[0023] According to the information processing system of the ninth aspect, when actual data related to attribute data is erased from the storage device, it is easier to complete the availability process successfully compared to when the attribute data recorded in the recording unit is retained.

[0024] According to the program of the tenth aspect, it is possible to obtain a program that enables an external device to use actual data stored in an area that cannot be accessed from the external device. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a diagram illustrating a system configuration of a machine learning system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration of an image forming apparatus according to a first embodiment. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of a cloud server according to the first embodiment. [Figure 4] 3 is a block diagram showing a relationship between a digital shadow owned by a cloud server and an image forming apparatus according to the first embodiment; FIG. [Diagram 5] 5 is a sequence diagram showing how the image forming apparatus and the digital shadow according to the first embodiment communicate with each other to create metadata associated with image data. FIG. [Figure 6] 10 is a sequence diagram showing how the image forming apparatus and the digital shadow according to the first embodiment communicate with each other and how metadata associated with image data is updated. FIG. [Figure 7]A sequence diagram showing how a machine learning device, a digital shadow, and an image forming device communicate with each other and generate a machine-learned model according to the first embodiment. [Figure 8] FIG. 11 is a flow diagram illustrating a process in which the cloud server according to the first embodiment searches for metadata included in a digital shadow. [Figure 9] A sequence diagram showing how a machine learning device, a digital shadow, and an image forming device communicate with each other and generate a machine-learned model according to the second embodiment. [Figure 10] A sequence diagram showing how a machine learning device, a digital shadow, and an image forming device communicate with each other and generate a machine-learned model according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, an example of an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensional ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0027] In the following description, a user refers to a person who creates, modifies, or deletes image data 44 of a document using the image forming device 32, or a person who generates a trained model using the machine learning device 50. The number of users is not limited, and includes cases where multiple image forming devices 32 or multiple functions are used.

[0028] [First embodiment] <Machine Learning System 10> FIG. 1 is a diagram showing a system configuration of a machine learning system 10 according to an embodiment of the present disclosure. As shown in FIG. 1, the machine learning system 10 according to an embodiment of the present disclosure is configured such that a plurality of image forming devices 32, each of which is installed at a plurality of bases 30, a machine learning device 50, and a cloud server 22 are connected via the Internet 12. In the following description, the image forming devices 32 are all the same device and will not be distinguished unless otherwise specified. In the following description, the description of the image forming device 32 includes cases where it refers to a single device and cases where it refers to any one of a plurality of devices.

[0029] (Image forming device 32) The image forming device 32 is a device called a multifunction device having multiple functions such as a printing function, a scanning function, a copying function, and a facsimile function, which are connected to each other via a network within the base 30. The image forming device 32 is an example of a "device that communicates with the processor" in this embodiment. FIG. 2 is a block diagram showing a hardware configuration of the image forming device 32 in this embodiment. As shown in FIG. 2, the image forming device 32 includes a control device 38, an image reading unit 40, an image forming unit 42, and a communication unit 34. These components are connected to each other via a control bus 36.

[0030] The control device 38 is a device that controls each part of the image forming device 32. The control device 38 has a function as a computer, and has a CPU 38A (Central Processing Unit), a ROM 38B (Read Only Memory), a RAM 38C (Random Access Memory), and a storage 38D, as shown in Fig. 2. The CPU 38A, the ROM 38B, the RAM 38C, and the storage 38D are each connected to each other by a control bus 36.

[0031] The CPU 38A is a central processing unit that executes various programs including information processing programs such as an information synchronization program and controls each part. The ROM 38B stores various programs including information processing programs and various data. The RAM 38C temporarily stores programs or data as a working area.

[0032] The storage 38D is configured with a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory, and stores various programs including an operating system, and various data. Here, the various data include data on various settings and states of the image forming apparatus 32. The information processing program may be stored in the storage 38D.

[0033] In the control device 38, the CPU 38A reads various programs including an information processing program from the ROM 38B or the storage 38D, and executes the programs using the RAM 38C as a working area. The CPU 38A executes the information processing program to realize various functions for controlling each part of the image forming device 32.

[0034] The image reading unit 40 is a component (e.g., a scanner) that reads an image of a document. The image reading unit 40 optically reads the image of the document and converts it into a digital signal to generate image data 44. In this description, as an example, the document includes characters.

[0035] The image forming unit 42 is a component that forms an image on a recording medium such as paper. The image forming unit 42 forms an image on the recording medium by, for example, an electrophotographic method that performs each process of charging, exposing, developing, transferring, and fixing. Note that the image forming unit 42 may form an image on the recording medium by another method such as an inkjet method.

[0036] The communication unit 34 is a component for communicating with other devices such as the cloud server 22 and the machine learning device 50. Specifically, the communication unit 34 communicates with other devices by using communication means such as wired, wireless, the Internet 12, an intranet, and public lines such as telephone lines. Note that the communication means may be communication means using sound, light, vibration, images, and the like.

[0037] The network that connects the image forming apparatuses 32 within the base 30 is a local network incorporating a boundary type security system such as a firewall.

[0038] In this embodiment, when an inquiry is made from image forming device 32 to cloud server 22 via communication unit 34, communication unit 34 is configured to be able to accept communication from cloud server 22 as a response to the inquiry.

[0039] In the image forming device 32, for example, a scan process is performed by the scan function, in which the image reading unit 40 reads an image of a document and generates image data 44. In addition, in the image forming device 32, for example, a facsimile process is performed by the facsimile function, in which the image data 44 generated by reading an image of a document with the image reading unit 40 is transmitted to another device, such as another multifunction device.

[0040] (Machine Learning Device 50) The machine learning device 50 according to the present embodiment is an example of the "external device" in the present embodiment, and includes a communication unit (not shown), a control device 52, and a recording device 54. The machine learning device 50 performs machine learning to detect features contained in the image data 44 by collecting and analyzing a large number of image data 44. The control device 52 is a device that controls each part of the machine learning device 50, and includes, for example, a CPU (not shown), a ROM (not shown), a RAM (not shown), and a recording device 54 (not shown). Each of these components (not shown) and the recording device 54 are connected to each other by a control bus (not shown). In addition, the machine learning device 50 collects image data 44 according to a procedure described later by the control device 52 reading a program recorded in the recording device 54. Note that the machine learning method is not particularly limited, but for example, learning related to image recognition in which a neural network is formed is assumed.

[0041] In this embodiment, the machine learning device 50 uses, for machine learning, images acquired by the image forming device 32. More specifically, the machine learning device 50 receives image data 44 acquired by the image forming device 32 via the Internet 12, and detects feature amounts from the multiple image data 44.

[0042] (Cloud Server 22) Cloud server 22 is an example of an "information processing system" in the technology of the present disclosure, and includes a control device 28 and a communication unit 24, as shown in Fig. 3. Control device 28 also has computer functions, and includes a CPU 28A, a ROM 28B, a RAM 28C, and a storage 28D, as shown in Fig. 3. These components are connected to each other via a control bus 26.

[0043] Note that specific functions and configurations of control device 28 and communication unit 24 of cloud server 22 are similar to those of control device 28 and communication unit 24 of image forming apparatus 32, except for the performance (processing capacity) of each component. That is, CPU 28A, ROM 28B, RAM 28C, and storage 28D of cloud server 22 are similar to CPU 38A, ROM 38B, RAM 38C, and storage 38D of image forming apparatus 33, except for the performance of each component. In the present embodiment, CPU 28A of cloud server 22 is an example of a "processor" in the present disclosure, and storage 28D is an example of a "recording unit" in the present disclosure.

[0044] 4, cloud server 22 stores in storage 28D data of digital shadow 32S as a virtual device corresponding to image forming apparatus 32 as a physical device connected via Internet 12. Cloud server 22 executes the behavior of digital shadow 32S by CPU 28A executing a program stored in storage 28D. In the following description, it is assumed that all virtual operations performed by digital shadow 32S are executed by CPU 28A of cloud server 22. In the description of this embodiment, although not shown or distinguished, it is assumed that digital shadow 32S is formed for each image forming apparatus 32 shown in FIG.

[0045] (Digital Shadow 32S) The digital shadow 32S is a virtual device that reflects the status of the image forming device 32 (e.g., the number of printed sheets, the operating time, etc.), and is created based on the status of the image forming device 32 recorded as data in the storage 28D. For example, when the digital shadow 32S receives an inquiry from the machine learning device 50, the digital shadow 32S notifies the machine learning device 50 of the status of the image forming device 32 by replying with information related to the model and functions of the image forming device 32 and information related to the status that the digital shadow 32S possesses.

[0046] Moreover, the digital shadow 32S records the image data 44 stored in the storage 38D of the image forming device 32 as metadata 44M. More specifically, when the image forming device 32 stores the image of the document read by the image reading unit 40 in the storage 38D as image data 44, the image forming device 32 notifies the digital shadow 32S of information contained in the image data 44. The digital shadow 32S then generates metadata 44M corresponding to the image data 44 and stores it in the storage 38M of the digital shadow 32S. The metadata 44M refers to data obtained by extracting attributes of the image included in the image data 44.

[0047] For example, if the original scanned by the image forming device 32 is a document, the image data 44 recorded in the storage 38D of the image forming device 32 contains the characters of the document as an image. Meanwhile, the metadata 44M recorded in the storage 38M by the digital shadow 32S contains data obtained by recognizing the characters contained in the image data 44 as character data using OCR (Optical Character Recognition). The metadata 44M also contains information that represents the characteristics of the image data 44. This information that represents the characteristics may be information obtained by applying a feature extraction technique known as an image processing technique. The metadata 44M also contains identification information that identifies the image forming device 32 that scanned the image data 44, and information about the scanning method used when scanning the image data 44.

[0048] The image data 44 is an example of "actual data generated by a device communicating with the processor" in this embodiment. The metadata 44M is an example of "attribute data associated with actual data" in this embodiment.

[0049] Also, as shown in FIG. 4, when the cloud server 22 communicates with the machine learning device 50, the cloud server 22 records the communication history. More specifically, in a procedure to be described later, the cloud server 22 stores the number of times that the metadata 44M was requested from the machine learning device 50 as metadata request history data 48HM in the storage 28D. Also, in a procedure to be described later, the cloud server 22 stores the number of times that the image data 44 was requested from the machine learning device 50 as image data request history data 48HR in the storage 28D. Note that the metadata request history data 48HM is an example of a "history of requests for attribute data" in this embodiment, and the image data request history data 48HR is an example of a "history of requests for actual data" in this embodiment. Also, the image data request history data 48HR may include the time when the image data 44 was requested from the machine learning device 50.

[0050] Next, how the metadata 44M is generated for the digital shadow 32S and how the metadata 44M is updated in this embodiment will be described with reference to FIGS.

[0051] First, a procedure for generating image data 44 and metadata 44M in this embodiment and recording the metadata 44M in the storage 38M of the digital shadow 32S will be described. Note that in this embodiment, the original that the user reads contains a document, and character data can be read by OCR.

[0052] (Creating 44M of metadata) 5 is a sequence diagram showing how the image forming device 32 and the digital shadow 32S according to this embodiment communicate with each other to create metadata 44M associated with the image data 44. As shown in FIG. 5, in this embodiment, the metadata 44M is created in the following procedure.

[0053] First, in sequence F102, the user causes the image forming apparatus 32 to read an original document and form image data 44.

[0054] Next, in sequence F104, the CPU 38A of the image forming device 32 records the image data 44 of the document read in sequence F102 in the storage 38D of the image forming device 32. The CPU 38A of the image forming device 32 also performs OCR on the image data 44, reads characters contained in the image data 44, and attaches the read character data to the image data 44 and records it.

[0055] Here, as shown in sequence F106, the digital shadow 32S inquires of the image forming apparatus 32 about synchronization processing, i.e., whether there is any change in the information recorded in the storage 38D of the image forming apparatus 32. Then, upon receiving the inquiry, the CPU 38A of the image forming apparatus 32 executes sequence F108.

[0056] In sequence F108, the CPU 38A of the image forming apparatus 32 generates metadata 44M from the image data 44 and character data included in the image data 44. Note that the metadata 44M may include information on the color tone, resolution, and the like of the image data 44 in addition to the character data obtained by OCR.

[0057] Next, in sequence F110, the CPU 38A of the image forming apparatus 32 transmits the metadata 44M generated in sequence F108 to the digital shadow 32S.

[0058] Then, in sequence F112, the digital shadow 32S records the metadata 44M received from the image forming device 32 in the storage 38M of the digital shadow 32S. Note that the metadata 44M of the digital shadow 32S also includes information indicating whether the image data 44 has been recorded in the storage 38D of the image forming device 32. In this embodiment, at the time of sequence F112, it is also recorded that the image data 44 has been recorded in the storage 38D of the image forming device 32.

[0059] Then, in sequence F114, the digital shadow 32S notifies the image forming apparatus 32 that the metadata 44M corresponding to the image data 44 has been recorded in the storage 38M of the digital shadow 32S.

[0060] Next, in this embodiment, a manner in which the metadata 44M recorded in the storage 38M of the digital shadow 32S is updated will be described. Note that in this embodiment, a case in which the user deletes the image data 44 stored in the above-mentioned procedure will be described as an example.

[0061] (Metadata 44M updated) 6 is a sequence diagram showing how the image forming device 32 and the digital shadow 32S according to this embodiment communicate with each other and how the metadata 44M associated with the image data 44 is updated. As shown in FIG. 6, in this embodiment, the metadata 44M is updated according to the following procedure.

[0062] First, in sequence F202, the user operates the image forming apparatus 32 to input a command to delete the image data 44 recorded in the storage 38D of the image forming apparatus 32.

[0063] Next, in sequence F204, the CPU 38A of the image forming apparatus 32 deletes from the storage 38D of the image forming apparatus 32 the image data 44 that was the target in sequence F202.

[0064] Next, as shown in sequence F206, the image forming apparatus 32 notifies the digital shadow 32S of a request to update the metadata 44M. Then, upon receiving the notification, the digital shadow 32S executes sequence F208.

[0065] In sequence F208, the CPU of the digital shadow 32S, i.e., the CPU 28A of the cloud server 22, notifies the image forming apparatus 32 of information accepting an update to the metadata 44M corresponding to the image data 44. Then, upon receiving the notification, the CPU 38A of the image forming apparatus 32 executes sequence F210.

[0066] Next, in sequence F210, the CPU 38A of the image forming apparatus 32 transmits a notification to the digital shadow 32S to delete the metadata 44M corresponding to the image data 44 deleted in sequence F204. Then, upon receiving the notification, the digital shadow 32S executes sequence F212.

[0067] Then, in sequence F212, the digital shadow 32S updates the information of the metadata 44M corresponding to the image data 44 deleted in sequence F204, based on the notification of deleting the metadata 44M received from the image forming device 32. Specifically, the digital shadow 32S records the fact that the image data 44 corresponding to the metadata 44M has been deleted from the storage 38D of the image forming device 32, with respect to information indicating whether or not the image data 44 corresponding to the metadata 44M has been recorded in the storage 38D of the image forming device 32.

[0068] Then, in sequence F214, the digital shadow 32S notifies the image forming apparatus 32 that the metadata 44M corresponding to the image data 44 has been deleted from the storage 38M of the digital shadow 32S.

[0069] As described above, the machine learning device 50 in this embodiment receives image data 44 acquired by the image forming device 32 via the Internet 12, and performs machine learning to detect features from the multiple image data 44. However, in general, it is not preferable in terms of security to allow a device outside the site 30 to directly connect to the image forming device 32 located within the firewall via the Internet 12. Therefore, in the configuration shown in FIG. 1, the machine learning device 50 cannot obtain access permission to refer to the image data 44 recorded in the storage 38D of the image forming device 32 in the site 30. In other words, as shown in FIG. 1, the image forming device 32 arranged at the site 30 in this embodiment cannot directly communicate with the machine learning device 50.

[0070] Next, we will explain the procedure in which the cloud server 22 in this embodiment works in cooperation with the machine learning device 50 to execute the procedures shown in Figures 7 and 8, thereby making the image data 44 held by the image forming device 32 available to the machine learning device 50.

[0071] (Procedure for making image data 44 available) 7 is a sequence diagram showing a state in which the machine learning device 50 according to the present disclosure, the digital shadow 32S, and the image forming device 32 communicate with each other, and the machine learning device 50 performs machine learning based on image data 44 to generate a machine-learned model. The CPU 28A of the cloud server 22, the CPU 38A of the image forming device 32, and a CPU (not shown) of the machine learning device 50 respectively read programs stored in the storage 28D, the storage 38D, and the recording device 54, thereby carrying out the series of steps shown in FIG. 7. Note that the machine learning device 50 executes sequence F302 shown in FIG. 7 when it receives an instruction from a user (not shown).

[0072] First, in sequence F302, the machine learning device 50 requests the digital shadow 32S to provide a list of image data 44 necessary for machine learning. More specifically, the machine learning device 50 requests the cloud server 22, which virtually executes the behavior of the digital shadow 32S, for image data 44 having attributes (e.g., resolution, color tone, presence or absence of text data, etc.) targeted for machine learning.

[0073] Next, in sequence F304, the cloud server 22 searches for metadata 44M of the image data 44 having the attributes of the image data 44 requested in sequence F302, and extracts the metadata 44M. More specifically, the cloud server 22 extracts the metadata 44M in accordance with the flowchart shown in FIG.

[0074] First, in step S402, CPU 28A of cloud server 22 sets any one of digital shadows 32S recorded in storage 28D as a digital shadow 32S to be searched for. Then, CPU 28A of cloud server 22 proceeds to step S404.

[0075] Next, in step S404, the CPU 28A of the cloud server 22 determines whether metadata 44M having the attribute requested in sequence F302 is recorded in the storage 38M of the digital shadow 32S searched for in step S402. If the CPU 28A of the cloud server 22 makes an affirmative determination in step S404, the process proceeds to step S406. On the other hand, if the CPU 28A of the cloud server 22 makes a negative determination in step S406, the process proceeds to step S408.

[0076] Next, in step S406, CPU 28A of cloud server 22 extracts metadata 44M recorded in storage 38M of the searched digital shadow 32S. Then, CPU 28A of cloud server 22 proceeds to step S408.

[0077] Next, in step S408, CPU 28A of cloud server 22 records a history of requests for metadata 44M for the searched digital shadow 32S in storage 28D as metadata request history data 48HM. Note that the metadata request history data 48HM recorded in step S408 may include the time when metadata 44M was requested, in addition to the determination result in step S404. Then, CPU 28A of cloud server 22 proceeds to step S410.

[0078] Next, in step S410, the CPU 28A of the cloud server 22 determines whether there is another digital shadow 32S that was not selected as a search target in step S402. If the CPU 28A of the cloud server 22 makes an affirmative determination in step S410, the process proceeds to step S402 and sets the other digital shadow 32S as the next digital shadow 32S to be searched. If the CPU 28A of the cloud server 22 makes a negative determination in step S410, the process ends extraction of the metadata 44M and proceeds to sequence F306.

[0079] Next, in sequence F306, the cloud server 22 transmits the metadata 44M, the metadata request history data 48HM, and the image data request history data 48HR extracted in sequence F304 to the machine learning device 50. Then, from the list of metadata 44M received in sequence F308, the machine learning device 50 identifies the image forming device 32 that records the image data 44 to be used for machine learning in the storage 38D.

[0080] Next, in sequence F308, the machine learning device 50 requests the cloud server 22 to respond as to whether or not image data 44 used in machine learning can be provided.

[0081] Next, in sequence F310, the cloud server 22 requests the image forming device 32, which records the image data 44 requested by the machine learning device 50 in sequence F308 in storage 38D, to respond as to whether or not the image data 44 can be provided.

[0082] Next, in sequence F312, the image forming apparatus 32 determines whether or not the image data 44 requested in sequence F310 can be provided. More specifically, the CPU 38A of the image forming apparatus 32 determines whether or not the image data 44 requested in sequence F310 can be provided from a security standpoint.

[0083] Next, in sequence F314, the image forming apparatus 32 responds to the cloud server 22 as to whether or not it can provide the image data 44 requested in sequence F310. In this description, one of the image forming apparatuses 32 responds that it can provide the image data 44.

[0084] Next, in sequence F316, the cloud server 22 responds to the machine learning device 50 regarding whether or not the image data 44 can be provided, which was received in sequence F314.

[0085] Next, in sequence F318, the machine learning device 50 requests the image data 44 from the digital shadow 32S that responded that it can provide the image data 44.

[0086] Next, in sequence F320, cloud server 22 transmits the request for providing image data 44 received in sequence F318 to image forming device 32 that stores the image data 44 in storage 38D. CPU 28A of cloud server 22 also records the history of requests for image data 44 from machine learning device 50 in sequence F318 in storage 28D as image data request history data 48HR.

[0087] Next, in sequence F322, the image forming apparatus 32 transmits to the cloud server 22 the image data 44 requested in sequence F320.

[0088] Next, in sequence F324, the cloud server 22 transmits the image data 44 received in sequence F322 to the machine learning device 50.

[0089] Next, in sequence F326, the image forming apparatus 32 transmits to the cloud server 22 a notification that the transmission of the image data 44 transmitted in sequence F322 has been completed.

[0090] Next, in sequence F328, the cloud server 22 transmits to the machine learning device 50 a notification that the transmission of the image data 44 transmitted in sequence F324 has been completed.

[0091] Next, in sequence F330, the machine learning device 50 starts executing machine learning based on the image data 44 received in sequence F328.

[0092] Next, in sequence F332, the machine learning device 50 records the created trained model in the recording device 54 when the machine learning started in sequence F330 is completed.

[0093] The machine learning device 50 may determine whether or not to execute the procedures from sequence F308 onward, based on the metadata request history data 48HM and the image data request history data 48HR received from the cloud server 22 in sequence F306. More specifically, the machine learning device 50 may be set not to request the provision of image data 44 for which there is a history of a previous request for provision, based on the metadata request history data 48HM and the image data request history data 48HR received from the cloud server 22 in sequence F306. For example, when the machine learning device 50 determines that there is no new image data 44 to be used for learning based on the metadata request history data 48HM received in sequence F306, it ends a series of procedures without executing the procedures from sequence F308 onward. Also, for example, when the machine learning device 50 determines that there is no new image data 44 to be used for learning based on the image data request history data 48HR received in sequence F306, it ends a series of procedures without executing the procedures from sequence F308 onward.

[0094] Next, the actions and effects of the cloud server 22 of the machine learning system 10 according to this embodiment will be described.

[0095] (Action and effect) Cloud server 22 of the present embodiment includes storage 28D that records metadata 44M associated with image data 44 generated by image forming device 32. When metadata 44M is requested by machine learning device 50, CPU 28A of cloud server 22 transmits metadata 44M recorded in storage 28D to machine learning device 50 and makes the image data 44 available to machine learning device 50. As a result, cloud server 22 according to the present embodiment can make image data 44 generated by image forming device 32 that cannot directly communicate with machine learning device 50 and stored in storage 38D of image forming device 32 available to machine learning device 50.

[0096] Furthermore, when image data 44 related to metadata 44M is requested by the machine learning device 50, the CPU 28A of the cloud server 22 of this embodiment makes the image data 44 available to the machine learning device 50. As a result, according to the cloud server 22 of this embodiment, it is possible to make image data 44 stored in the image forming device 32 that cannot directly communicate with the machine learning device 50 available to the machine learning device 50.

[0097] Furthermore, the CPU 28A of the cloud server 22 of this embodiment receives image data 44 from the image forming device 32 that stores the image data 44, and transmits the image data 44 to the machine learning device 50. Therefore, according to the cloud server 22 of this embodiment, it is possible to make the image data 44 stored in the image forming device 32 that cannot directly communicate with the machine learning device 50 available to the machine learning device 50.

[0098] Furthermore, the CPU 28A of the cloud server 22 of this embodiment records a history of requests for image data 44 from the machine learning device 50 in the storage 28D as image data request history data 48HR. As a result, the cloud server 22 of this embodiment can inform the machine learning device 50 that the machine learning device 50 has repeatedly requested the same image data 44, compared to a case in which the history of transmitting image data 44 to the machine learning device 50 is not recorded. In other words, the cloud server 22 of this embodiment can notify the machine learning device 50 that the requested image data 44 is already available.

[0099] Furthermore, in cloud server 22 of the present embodiment, CPU 28A records metadata 44M in storage 38M every time image forming device 32 generates image data 44. As a result, according to cloud server 22 of the present embodiment, the time when machine learning device 50 can use image data 44 can be accelerated, compared to a case where metadata 44M is generated at a time separate from the generation of image data 44.

[0100] Furthermore, in cloud server 22 of the present embodiment, when image data 44 related to metadata 44M is deleted, CPU 28A adds to metadata 44M a history of the deletion of image data 44. As a result, according to cloud server 22 according to the present embodiment, when image data 44 related to metadata 44M is deleted from the storage device, it is easier to successfully complete the availability process compared to the case where metadata 44M recorded in storage 38M is retained.

[0101] Furthermore, the program of this embodiment causes the CPU 38A to execute a process of transmitting the metadata 44M recorded in the storage 38M to the machine learning device 50 when the metadata 44M is requested by the machine learning device 50. That is, the program causes the CPU 38A to execute a process of enabling the machine learning device 50 to handle the image data 44 associated with the metadata 44M. As a result, according to the program of this embodiment, a program is obtained that allows the machine learning device 50 to use the image data 44 stored in an area that cannot be accessed by the machine learning device 50.

[0102] Next, a second embodiment of the present disclosure will be described. Note that the configuration of the machine learning system 10 according to the second embodiment is the same as that of the first embodiment, so the same reference numerals as those in the first embodiment are used and a detailed description thereof will be omitted.

[0103] [Second embodiment] As shown in FIG. 9, the machine learning system 10 according to the second embodiment differs from the first embodiment in the procedure for allowing the machine learning device 50 to use image data 44 held by the image forming device 32.

[0104] (Procedure for making image data 44 available) In FIG. 9, the procedure from sequence F502 to sequence F518 is similar to the procedure from sequence F302 to sequence F318 in the first embodiment.

[0105] In sequence F520, the cloud server 22 requests the image forming device 32 that stores the image data 44 in the storage 28D to transmit the image data 44 received in sequence F518 to the machine learning device 50. More specifically, in sequence F520, the cloud server 22 notifies the image forming device 32 that stores the image data 44 in the storage 28D of information including the network address, authentication information, etc. of the machine learning device 50. Furthermore, in sequence F520, the cloud server 22 requests the image forming device 32 to transmit the image data 44 to the machine learning device 50 together with the notification.

[0106] Next, in sequence F522, the image forming device 32 transmits the image data 44 requested in sequence F520 to the machine learning device 50.

[0107] Next, in sequence F524, the image forming device 32 transmits to the machine learning device 50 a notification that the transmission of the image data 44 transmitted in sequence F522 has been completed.

[0108] The procedure from sequence F530 to sequence F532 is similar to the procedure from sequence F330 to sequence F332 in the first embodiment.

[0109] Next, the actions and effects of the cloud server 22 of the machine learning system 10 according to this embodiment will be described.

[0110] (Action and effect) In the information processing system of this embodiment, CPU 28A of cloud server 22 causes a device that stores image data 44 to transmit the image data 44 to the machine learning device 50. As a result, according to the information processing system of this embodiment, the time until the machine learning device 50 receives the image data 44 can be shortened compared to a case in which CPU 28A of cloud server 22 receives the image data 44 and then transmits it to the machine learning device 50.

[0111] In this embodiment, the same functions and effects as those of the first embodiment can be obtained with the same configuration as that of the first embodiment.

[0112] Next, a third embodiment of the present disclosure will be described. Note that since the configuration of the machine learning system 10 according to the third embodiment is similar to that of the first or second embodiment, the same reference numerals as those of the first or second embodiment are used, and a detailed description thereof will be omitted.

[0113] [Third embodiment] In the machine learning system 10 according to the third embodiment, as shown in FIG. 10, the procedure for enabling the machine learning device 50 to use image data 44 held by the image forming device 32 differs from that of the first and second embodiments.

[0114] (Procedure for making image data 44 available) In FIG. 10, the procedure from sequence F602 to sequence F614 is similar to the procedure from sequence F302 to sequence F314 in the first embodiment.

[0115] In sequence F616, if the image data 44 received in sequence F614 can be provided, the cloud server 22 transmits a method for accessing a storage area of ​​the image forming device 32 in which the image data 44 is stored in the storage 38D to the machine learning device 50. More specifically, in sequence F616, the cloud server 22 transmits information including the network address, authentication information, etc. of the image data 44 together with the result of whether the image data 44 can be provided, received in sequence F614.

[0116] Next, in sequence F618, based on the information received in sequence F616, the machine learning device 50 requests the image forming device 32, which stores the image data 44 requested in sequence F614 in storage 38D, to provide the image data 44.

[0117] Next, in sequence F620, the image forming device 32 transmits the image data 44 requested in sequence F618 to the machine learning device 50.

[0118] Next, in sequence F622, the image forming device 32 transmits to the machine learning device 50 a notification that the transmission of the image data 44 transmitted in sequence F620 has been completed.

[0119] The procedure from sequence F630 to sequence F632 is similar to the procedure from sequence F330 to sequence F332 in the first embodiment.

[0120] Next, the actions and effects of the cloud server 22 of the machine learning system 10 according to this embodiment will be described.

[0121] In the cloud server 22 of this embodiment, when metadata 44M is requested by the machine learning device 50, the CPU 28A of the cloud server 22 transmits to the machine learning device 50 a method for accessing the storage area in which the image data 44 is stored, together with the metadata 44M. Therefore, according to the cloud server 22 of this embodiment, it is possible to shorten the time until the machine learning device 50 can use the image data 44, compared to a case in which the CPU 28A of the cloud server 22 receives the image data 44 and transmits it to the machine learning device 50.

[0122] Furthermore, the CPU 28A of the cloud server 22 of this embodiment records a history of requests for metadata 44M from the machine learning device 50 as metadata request history data 48HM. Therefore, the cloud server 22 of this embodiment can inform the machine learning device 50 that the machine learning device 50 has repeatedly requested the same image data 44, compared to a case in which the history of transmitting image data 44 to the machine learning device 50 is not recorded. In other words, the cloud server 22 of this embodiment can notify the machine learning device 50 that the requested image data 44 is already available.

[0123] In this embodiment, too, if the configuration is similar to that of the first or second embodiment, the same actions and effects as those of the first or second embodiment can be obtained.

[0124] [Other variations] In the above description, the CPU 28A of the cloud server 22 stores the metadata 44M in the storage 38M every time the image forming device 32 generates image data 44, but the technology according to the present disclosure is not limited to this. For example, the CPU 28A of the cloud server 22 may not store the metadata 44M until multiple image data 44 are generated. Also, for example, the CPU 28A of the cloud server 22 may periodically access the image forming device 32, and when it determines that new image data 44 has been generated, create the metadata 44M and store it in the storage 38M (so-called batch processing).

[0125] In the above description, CPU 28A of cloud server 22 stores the history of requests for image data 44 from machine learning device 50 as metadata request history data 48HM in storage 28D, but the technology according to the present disclosure is not limited to this. For example, CPU 28A of cloud server 22 does not have to create metadata request history data 48HM.

[0126] In the above description, the CPU 28A of the cloud server 22 stores the history of requests for metadata 44M from the machine learning device 50 as metadata request history data 48HM in the storage 28D, but the technology according to the present disclosure is not limited to this. For example, the CPU 28A of the cloud server 22 does not have to create the metadata request history data 48HM.

[0127] In addition, in the above embodiment, the machine learning device 50 is used as an example of an external device, but the technology according to the present disclosure is not limited to this. An example of an external device according to the present disclosure may be, for example, a device that simply obtains statistics on the usage status of the image forming device 32. In other words, any device that uses actual data (image data 44) generated by the image forming device 32 in this embodiment may be included in the external device according to the present disclosure.

[0128] In the above embodiment, the cloud server 22 that forms the digital shadow 32S is used as an example of an information processing system, but the technology according to the present disclosure is not limited to this. An example of the "information processing system" according to the present disclosure may be, for example, a recording device 54 that does not create the digital shadow 32S but simply records metadata 44M based on image data 44. In other words, any device that records attribute data associated with actual data generated by the image forming device 32 in this embodiment and makes the actual data available when the attribute data is requested may be included in the information processing system according to the present disclosure.

[0129] In the above embodiment, cloud server 22 is described as being configured by a single device, but may be configured by multiple devices. That is, the "information processing system" in this embodiment may be configured by a single device or may be configured by multiple devices.

[0130] In the above embodiment, the machine learning system 10 has been described as having a single cloud server 22, but the technology according to the present disclosure is not limited to this. That is, the machine learning system 10 may have multiple cloud servers 22.

[0131] In the above embodiment, the image data 44 of the image forming device 32 is used as an example of real data, but the technology according to the present disclosure is not limited to this. Examples of "real data" according to the present disclosure may include, for example, data created using a measuring device including audio data and video data, as well as text data, modeling data, and other artificially created data. Even in these cases, the "attribute data" according to the present disclosure may include the date and time of generation of the real data, the device that generated the real data, and other data created based on the real data, as long as they are capable of being associated with the real data as attribute data associated with the real data. In other words, as long as they are capable of communicating with the processor, devices such as measuring devices including audio and video, text creation devices, and modeling devices may also be included in the "device communicating with the processor" according to the present disclosure.

[0132] In addition, in the above embodiment, the processor refers to a processor in a broad sense, and includes a general-purpose processor (e.g., the aforementioned CPU, etc.) and a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0133] In addition, the operations of the processor in the above embodiment may be performed not only by one processor but also by multiple processors located at physically separate locations working together. The order of the operations of the processor is not limited to the order described in the above embodiment, and may be changed as appropriate.

[0134] In addition, the processing executed by the CPU after reading the software (program) in each of the above embodiments may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific processing. In addition, the processing may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). In addition, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0135] In addition, in each of the above embodiments, the processing program is described as being pre-stored (installed) in the storage, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0136] The above describes an embodiment of the present disclosure with reference to the attached drawings. However, it is clear that a person with ordinary knowledge in the technical field to which the present disclosure pertains can conceive of various modifications or applications within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0137] Furthermore, preferred aspects of the present disclosure will be described below.

[0138] (((1))) one or more processors; one or more recorders configured to record attribute data associated with actual data generated by a device in communication with the processor; Equipped with when the processor receives a request for the attribute data from an external device, the processor transmits the attribute data recorded in the recording unit to the external device and makes the actual data available to the external device. Information processing system.

[0139] (((2))) the processor, when the external device requests the actual data associated with the attribute data, makes the actual data available to the external device; (((1)))'s information processing system.

[0140] (((3))) The processor receives the actual data from the device that stores the actual data, and transmits the actual data to the external device. (((2)))'s information processing system.

[0141] (((4))) The processor causes the device storing the actual data to transmit the actual data to the external device. An information processing system as defined in (((2))) or (((3))).

[0142] (((5))) The processor records a history of requests for the actual data from the external device. An information processing system selected from (((2))) to (((4))).

[0143] (((6))) when the processor receives a request for the attribute data from an external device, the processor transmits to the external device a method for accessing a storage area in which the actual data is stored together with the attribute data; (((1)))'s information processing system.

[0144] (((7))) The processor records a history of requests for the attribute data from the external device. (((6)))'s information processing system.

[0145] (((8))) The processor records the attribute data in the recording unit every time the device generates the actual data. An information processing system selected from (((1))) to (((7))).

[0146] (((9))) when the actual data related to the attribute data is deleted, the processor adds a history of the deletion of the actual data to the attribute data. An information processing system selected from (((1))) to (((8))).

[0147] (((10))) When an external device requests attribute data, the attribute data recorded in the recording unit is transmitted to the external device, and actual data associated with the attribute data is made available to the external device. A program that causes a computer to carry out processing.

[0148] According to the information processing system of (((1))), actual data generated by a device that cannot directly communicate with an external device can be made available to the external device. According to the information processing system of (((2))), it is possible to make actual data stored in a device that cannot directly communicate with an external device available to the external device. According to the information processing system of (((3))), actual data stored in a device that cannot directly communicate with an external device can be made available to the external device. According to the information processing system of (((4))), the time until the external device receives the actual data can be shortened, compared to a case in which the processor receives the actual data and then transmits it to the external device. According to the information processing system of (((5))), it is possible to inform the external device that the external device is repeatedly requesting the same actual data. According to the information processing system of (((6))), the time until the actual data becomes available to the external device can be shortened, compared to a case in which the processor receives the actual data and then transmits it to the external device. According to the information processing system of (((7))), it is possible to inform the external device that the external device is repeatedly requesting the same actual data. According to the information processing system of (((8))), the time when the actual data becomes available to an external device can be accelerated, compared to a case in which attribute data is generated at a time separate from the time when the actual data is generated. According to the information processing system of (((9))), when actual data related to attribute data is erased from the storage device, it is easier to successfully complete the utilization process compared to when the attribute data recorded in the recording unit is retained. According to the program of (((10))), it is possible to obtain a program that enables an external device to use actual data stored in an area that cannot be accessed from the external device. [Explanation of symbols]

[0149] 10 Learning System 12. Internet 22 Cloud server (an example of an information system) 24 Communications Department 26 Control Bus 28 Control device 30 locations 32 Image forming equipment (example of equipment) 34 Communications Department 36 Control Bus 38 Control device 40 Image reading unit 42 Image forming section 44 Image data (example of actual data) 50 Machine learning device (an example of an external device) 52 Control device 54 Recording Devices 28A CPU (an example of a processor) 28B ROM 28C RAM 28D Storage (an example of a recording section) 32S Digital Shadow 38A CPU 38B ROM 38C RAM 38D Storage 38M Storage 44M metadata (example of attribute data) 48HM Metadata request history data (example of history) 48HR Image data request history data (example of history)

Claims

1. one or more processors; one or more recorders configured to record attribute data associated with actual data generated by a device in communication with the processor; Equipped with when the processor receives a request for the attribute data from an external device, the processor transmits the attribute data recorded in the recording unit to the external device and makes the actual data available to the external device. Information processing system.

2. the processor, when the external device requests the actual data associated with the attribute data, makes the actual data available to the external device; The information processing system according to claim 1 .

3. The processor receives the actual data from the device that stores the actual data, and transmits the actual data to the external device. The information processing system according to claim 2 .

4. The processor causes the device storing the actual data to transmit the actual data to the external device. The information processing system according to claim 2 .

5. The processor records a history of requests for the actual data from the external device. The information processing system according to claim 2 .

6. when the processor receives a request for the attribute data from an external device, the processor transmits to the external device a method for accessing a storage area in which the actual data is stored together with the attribute data; The information processing system according to claim 1 .

7. The processor records a history of requests for the attribute data from the external device. The information processing system according to claim 6 .

8. The processor records the attribute data in the recording unit every time the device generates the actual data. The information processing system according to claim 1 .

9. when the actual data related to the attribute data is deleted, the processor adds a history of the deletion of the actual data to the attribute data. An information processing system according to any one of claims 1 to 8.

10. When an external device requests attribute data, the attribute data recorded in the recording unit is transmitted to the external device, and actual data associated with the attribute data is made available to the external device. A program that causes a computer to carry out processing.

Images