Information processing system, storage medium, and information processing method

Abstract

An information processing system is provided with: one or a plurality of processors; and one or a plurality of recording units that record attribute data associated with actual data generated by a device that communicates with the processor, the processor performing a process that, in a case where the attribute data is requested from an external device, transmits the attribute data to the external device, and stores the attribute data associated with the actual data. And a transmission unit that transmits the attribute data recorded in the recording unit to the external device so that the external device can use the actual data.

Description

Technical Field

[0001] The invention relates to an information processing system, a storage medium and an information processing method. Background Art

[0002] For example, patent document 1 discloses a setting screen, wherein, in the case of providing a setting screen for making settings related to image data that becomes a new sending object, settings related to the image data that becomes a new sending object are made in a state reflecting settings stored in a storage unit and associated with image data of the same type as the image data that becomes a new sending object and stored in association with information of a user that is the same as the following user, and the user makes settings related to the image data that becomes a new sending object.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2021-163983 Summary of the invention

[0004] An object of the present invention is to enable an external device to utilize actual data generated by a device that is unable to communicate directly with the external device.

[0005] An information processing system of the first type comprises: one or more processors; and one or more recording units, wherein the recording unit records attribute data associated with actual data, the actual data being generated by a device that communicates with the processor, and the processor performs the following processing: when the attribute data is requested from an external device, the attribute data recorded in the recording unit is sent to the external device so that the external device can use the actual data.

[0006] The information processing system according to the second aspect is the information processing system according to the first aspect, wherein the processor performs processing for making the external device useable the actual data when the actual data associated with the attribute data is requested from the external device.

[0007] In the information processing system according to the third aspect, in the information processing system according to the second aspect, the processor performs a process of receiving the actual data from the device storing the actual data and transmitting the actual data to the external device.

[0008] An information processing system according to a fourth aspect is the information processing system according to the second aspect or the third aspect, wherein the processor causes the device storing the actual data to transmit the actual data to the external device.

[0009] An information processing system according to a fifth aspect is the information processing system according to any one of the second aspect to the fourth aspect, wherein the processor records a history of requesting the actual data from the external device.

[0010] The information processing system of the sixth aspect is the information processing system of the first aspect, wherein the processor performs processing such that, when the attribute data is requested from an external device, the processor transmits a method for accessing a storage area storing the actual data together with the attribute data to the external device.

[0011] An information processing system according to a seventh aspect is the information processing system according to the sixth aspect, wherein the processor records a history of requests for the attribute data from the external device.

[0012] An information processing system according to an eighth aspect is the information processing system according to any one of the first to seventh aspects, wherein the processor performs processing for recording the attribute data in the recording unit every time the device generates the actual data.

[0013] The information processing system of the ninth aspect is the information processing system of any one of the first to eighth aspects, wherein the processor performs the following processing: when the actual data associated with the attribute data is deleted, adding a history of the deletion of the actual data to the attribute data.

[0014] A storage medium of the tenth aspect stores a program causing a computer to execute the following processing: when attribute data is requested from an external device, the attribute data recorded in a recording unit is sent to the external device so that the external device can use actual data associated with the attribute data.

[0015] The information processing method of the eleventh aspect includes the step of, when attribute data is requested from an external device, transmitting the attribute data recorded in a recording unit to the external device so that the external device can use actual data associated with the attribute data.

[0016] Effects of the Invention

[0017] According to the information processing system according to the first aspect, it is possible to allow the external device to use actual data generated by a device that cannot directly communicate with the external device.

[0018] According to the information processing system according to the second aspect, it is possible to allow an external device to use actual data stored in a device that cannot directly communicate with the external device.

[0019] According to the information processing system according to the third aspect, it is possible to allow an external device to use actual data stored in a device that cannot directly communicate with the external device.

[0020] According to the information processing system according to the fourth aspect, compared with a case where the processor receives the actual data and transmits it to the external device, the time until the external device receives the actual data can be shortened.

[0021] According to the information processing system according to the fifth aspect, it is possible to notify the external device that the external device is repeatedly requesting the same actual data.

[0022] According to the information processing system according to the sixth aspect, compared with a case where the processor receives the actual data and transmits it to the external device, the time until the external device can use the actual data can be shortened.

[0023] According to the information processing system according to the seventh aspect, it is possible to notify the external device that the external device is repeatedly requesting the same actual data.

[0024] According to the information processing system of the eighth aspect, compared with a case where the attribute data is generated at a different time from the generation of the actual data, the time at which the actual data can be used by the external device can be shortened.

[0025] According to the information processing system of the ninth aspect, when actual data associated with attribute data is deleted from the storage device, it is easier to normally complete the usability processing than when the attribute data recorded in the recording unit is retained.

[0026] According to the storage medium involved in the tenth aspect, it is possible to obtain a storage medium that allows an external device to use actual data stored in an area that is inaccessible from the external device.

[0027] According to the information processing method according to the eleventh aspect, it is possible to obtain an information processing method that allows an external device to use actual data stored in an area that is inaccessible from the external device. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Embodiments of the present invention will be described in detail with reference to the following drawings.

[0029] Figure 1 is a diagram showing a system configuration of a machine learning system according to a first embodiment;

[0030] Figure 2 is a block diagram showing a hardware configuration of an image forming apparatus according to a first embodiment;

[0031] Figure 3 is a block diagram showing a hardware configuration of a cloud server according to a first embodiment;

[0032] Figure 4is a block diagram showing the relationship between a digital shadow and an image forming device included in a cloud server according to a first embodiment;

[0033] Figure 5 is a timing chart showing a state in which the image forming apparatus according to the first embodiment communicates with a digital shadow and creates metadata associated with image data;

[0034] Figure 6 is a timing chart showing a state in which the image forming apparatus according to the first embodiment communicates with the digital shadow and updates metadata associated with the image data;

[0035] Figure 7 is a timing diagram showing a state in which the machine learning device, the digital shadow, and the image forming device according to the first embodiment communicate with each other and generate a machine learning completed model;

[0036] Figure 8 is a flow chart of the cloud server searching for metadata contained in the digital shadow according to the first embodiment;

[0037] Fig. 9 is a timing diagram showing a state in which a machine learning device, a digital shadow, and an image forming device according to a second embodiment communicate with each other and generate a machine learning completed model;

[0038] Fig.10 This is a timing diagram showing a state in which the machine learning device, digital shadow, and image forming device according to the third embodiment communicate with each other and generate a machine learning completed model.

[0039] Explanation of symbols

[0040] 10-learning system, 12-Internet, 22-cloud server (an example of an information system), 24-communication unit, 26-control bus, 28-control device, 30-base, 32-image forming device (an example of a device), 34-communication unit, 36-control bus, 38-control device, 40-image reading unit, 42-image forming unit, 44-image data (an example of actual data), 50-machine learning device (an example of an external device), 52-control device, 54-recording device, 28A-CPU (an example of a processor), 28B-ROM, 28C-RAM, 28D-memory (an example of a recording unit), 32S-digital shadow, 38A-CPU, 38B-ROM, 38C-RAM, 38D-memory, 38M-memory, 44M-metadata (an example of attribute data), 48HM-metadata request history data (an example of history), 48HR-image data request history data (an example of history). DETAILED DESCRIPTION

[0041] Hereinafter, an example of an embodiment of the present invention will be described with reference to the accompanying drawings. In addition, in each of the drawings, the same reference symbols are used to mark the same or equivalent components and parts. In addition, the dimensional ratios of the drawings are exaggerated for the sake of convenience of description and may be different from the actual ratios.

[0042] In the following description, a user refers to a person who creates, corrects, or deletes the original image data 44 using the image forming device 32, or a person who generates a learning completion model using the machine learning device 50. The number of users is not limited, and it also includes the case where multiple image forming devices 32 or multiple functions are used.

[0043] [First embodiment]

[0044] <Machine Learning System 10>

[0045] Figure 1 1 is a diagram showing the system configuration of a machine learning system 10 according to an embodiment of the present invention. Figure 1 As shown, the machine learning system 10 of one embodiment of the present invention is a structure in which a plurality of image forming devices 32, machine learning devices 50, and cloud servers 22 are respectively installed at a plurality of bases 30 and connected via the Internet 12. In addition, in the following description, the image forming devices 32 are all the same devices, and are not distinguished unless otherwise specified. In addition, in the following description, the description of the image forming device 32 includes the case of referring to a single device and the case of referring to any one of the plurality of devices.

[0046] (Image Forming Device 32)

[0047] The image forming device 32 is a device called a multifunction machine that is connected to each other via a network in the base 30 and has multiple functions such as printing, scanning, copying, and faxing. The image forming device 32 is an example of a "device that communicates with the processor" in this embodiment. Figure 2 A block diagram showing the hardware configuration of the image forming device 32 in this embodiment is shown in FIG. Figure 2 As shown, the image forming apparatus 32 includes a control device 38 , an image reading unit 40 , an image forming unit 42 , and a communication unit 34 . These components are connected to each other via a control bus 36 .

[0048] The control device 38 is a device for controlling each part of the image forming device 32. The control device 38 has a function as a computer, such as Figure 2As shown, the CPU 38A (Central Processing Unit), ROM 38B (Read Only Memory), RAM 38C (Random Access Memory), and memory 38D are provided. The CPU 38A, ROM 38B, RAM 38C, and memory 38D are connected to each other via a control bus 36 .

[0049] The CPU 38A is a central processing unit and executes various programs including information processing programs such as information synchronization programs or controls various components. The ROM 38B stores various programs including information processing programs and various data. The RAM 38C temporarily stores programs or data as a work area.

[0050] The memory 38D is composed of a storage medium such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and stores various programs including an operating system and various data. Here, the various data include data such as various settings and states of the image forming device 32. In addition, the information processing program is stored in the memory 38D.

[0051] In control device 38 , CPU 38A reads various programs including information processing programs from ROM 38B or memory 38D and executes the programs using RAM 38C as a work area. CPU 38A executes the information processing programs to realize various functions of controlling each unit of image forming device 32 .

[0052] The image reading unit 40 is a component (for example, a scanner) that reads an image of a document. The image reading unit 40 optically reads an image of a document and converts it into a digital signal to generate image data 44. In this description, as an example, characters are included in the document.

[0053] The image forming unit 42 is a structural unit that forms an image on a recording medium such as paper. The image forming unit 42 forms an image on the recording medium by, for example, an electrophotographic method that performs each process of charging, exposure, development, transfer, and fixing. In addition, the image forming unit 42 may also form an image on the recording medium by other methods such as an inkjet method.

[0054] The communication unit 34 is a component for communicating with other devices such as the cloud server 22 and the machine learning device 50. Specifically, the communication unit 34 communicates with other devices by using a communication unit such as a wired, wireless, Internet 12, intranet, and public lines such as telephone lines. In addition, the communication unit may be a communication unit using sound, light, vibration, image, etc.

[0055] The network connecting the image forming apparatuses 32 in the base 30 is, for example, a local network in which a perimeter security system such as a firewall is installed.

[0056] In the present embodiment, when an inquiry is made from the image forming device 32 to the cloud server 22 via the communication unit 34 , the communication unit 34 is configured to be able to receive communication from the cloud server 22 as a response to the inquiry.

[0057] In the image forming device 32, for example, the image reading unit 40 reads the image of the document and generates image data 44, thereby executing a scan process based on the scan function. In addition, in the image forming device 32, for example, the image data 44 generated by reading the image of the document by the image reading unit 40 is sent to another device such as another multifunction machine, thereby executing a fax process based on the fax function.

[0058] (Machine Learning Device 50)

[0059] The machine learning device 50 involved in this embodiment is an example of an "external device" in this embodiment, and is equipped with a communication unit, a control device 52 and a recording device 54, which are not shown in the figure. The machine learning device 50 performs machine learning to detect the feature quantity contained in the image data 44 by collecting and analyzing a plurality of image data 44. The control device 52 is a device that controls each part of the machine learning device 50, and as an example, has a CPU, a ROM, a RAM, and a recording device 54, which are not shown in the figure. These unshown structures and the recording device 54 are each connected to each other by a control bus, which is not shown in the figure. In addition, the machine learning device 50 collects the image data 44 according to the steps described later by reading the program recorded in the recording device 54 by the control device 52. In addition, there is no particular limitation on the method of machine learning, but for example, learning related to image recognition formed with a neural network can be conceived.

[0060] In this embodiment, the machine learning device 50 uses the image acquired by the image forming device 32 for machine learning. More specifically, the machine learning device 50 receives the image data 44 acquired by the image forming device 32 via the Internet 12 and detects feature quantities based on the plurality of image data 44.

[0061] (Cloud Server 22)

[0062] The cloud server 22 is an example of an "information processing system" in the technology of the present invention. Figure 3 As shown in FIG. 2 , the control device 28 and the communication unit 24 are provided. In addition, the control device 28 has a function as a computer, such as Figure 3As shown in the figure, a CPU 28A, a ROM 28B, a RAM 28C, and a memory 28D are provided. These components are connected to each other via a control bus 26 .

[0063] In addition, the specific functions and structures of the control device 28 and the communication unit 24 of the cloud server 22 are the same as those of the control device 28 and the communication unit 24 of the image forming device 32, except for the performance (processing capacity) of each structure. That is, the CPU 28A, ROM 28B, RAM 28C, and memory 28D of the cloud server 22 are the same as the CPU 38A, ROM 38B, RAM 38C, and memory 38D of the image forming device 33, except for the performance of each structure. In addition, in the present embodiment, the CPU 28A of the cloud server 22 is an example of the "processor" in the present invention, and the memory 28D is an example of the "recording unit" in the present invention.

[0064] And, if Figure 4 As shown, the cloud server 22 stores the data of the digital shadow 32S as a virtual device corresponding to the image forming device 32 as a physical device connected via the Internet 12 in the memory 28D. Furthermore, the cloud server 22 performs the operation of the digital shadow 32S by having the CPU 28A execute the program stored in the memory 28D. In the following description, the virtual actions operated by the digital shadow 32S are all performed by the CPU 28A of the cloud server 22. Furthermore, in the description of this embodiment, although not illustrated and distinguished, the digital shadow 32S for Figure 1 Each image forming device 32 is shown formed.

[0065] (Digital Shadow 32S)

[0066] The digital shadow 32S is a virtual device that reflects the state of the image forming device 32 (for example, the number of printed sheets or the operating time, etc.), and is created based on the state of the image forming device 32 recorded as data in the memory 28D. As an example, when receiving an inquiry from the machine learning device 50, the digital shadow 32S notifies the machine learning device 50 of the state of the image forming device 32 by answering information related to the model or function of the image forming device 32 possessed by the digital shadow 32S, or information related to the state.

[0067] Furthermore, the digital shadow 32S records the image data 44 stored in the memory 38D of the image forming device 32 as metadata 44M. More specifically, when the image of the document read by the image reading unit 40 is stored in the memory 38D as image data 44, the image forming device 32 notifies the digital shadow 32S of the information of the image data 44. Furthermore, the digital shadow 32S generates metadata 44M corresponding to the image data 44 and stores it in the memory 38M of the digital shadow 32S. The metadata 44M refers to data obtained by extracting the attributes of the image contained in the image data 44.

[0068] For example, when the manuscript read by the image forming device 32 is a document, the characters of the document are recorded as an image in the image data 44 recorded in the memory 38D of the image forming device 32. On the other hand, in the metadata 44M recorded in the memory 38M by the digital shadow 32S, data is recorded in which the characters included in the image data 44 are recognized as character data by OCR (Optical Character Recognition). In addition, the metadata 44M contains information representing the characteristics of the image data 44. The information representing the characteristics can be information obtained by applying a feature extraction technology known as an image processing technology. In addition, the metadata 44M contains identification information for identifying the image forming device 32 that read the image data 44, and information related to the reading method used when reading the image data 44.

[0069] The image data 44 is an example of “actual data generated by a device communicating with a processor” in the present embodiment. The metadata 44M is an example of “attribute data associated with actual data” in the present embodiment.

[0070] And, if Figure 4 As shown, the cloud server 22 records its communication history when communicating with the machine learning device 50. More specifically, in the steps described later, the cloud server 22 stores the number of times the metadata 44M is requested from the machine learning device 50 as metadata request history data 48HM in the memory 28D. Furthermore, in the steps described later, the cloud server 22 stores the number of times the image data 44 is requested from the machine learning device 50 as image data request history data 48HR in the memory 28D. In addition, the metadata request history data 48HM is an example of the "history of requesting attribute data" in the present embodiment, and the image data request history data 48HR is an example of the "history of requesting actual data" in the present embodiment. Furthermore, the image data request history data 48HR may also include the period when the image data 44 was requested from the machine learning device 50.

[0071] Next, in this embodiment, referring to Figure 5 and Figure 6 The state of generating metadata 44M and the state of updating metadata 44M in the digital shadow 32S are described.

[0072] First, in this embodiment, the steps of generating image data 44 and metadata 44M and recording metadata 44M in memory 38M of digital shadow 32S are described. In this embodiment, a document is written on a manuscript read by a user, and character data can be read by OCR.

[0073] (Creation of metadata 44M)

[0074] Figure 5 4 is a timing chart showing a state in which the image forming device 32 according to the present embodiment communicates with the digital shadow 32S and creates metadata 44M associated with the image data 44. Figure 5 As shown, in this embodiment, metadata 44M is created through the following steps.

[0075] First, in sequence F102 , the user uses the image forming device 32 to read a document and form image data 44 .

[0076] Next, in sequence F104, the CPU 38A of the image forming device 32 records the image data 44 of the document read in sequence F102 in the memory 38D of the image forming device 32. Furthermore, the CPU 38A of the image forming device 32 performs OCR on the image data 44, reads the characters included in the image data 44, and also adds the read character data to the image data 44 and records it.

[0077] Here, as shown in sequence F106, the digital shadow 32S performs synchronization processing on the image forming device 32, that is, inquires whether there is any change in the information recorded in the memory 38D of the image forming device 32. Then, the CPU 38A of the image forming device 32 that receives the inquiry executes sequence F108.

[0078] In sequence F108, CPU 38A of image forming device 32 generates metadata 44M based on image data 44 and character data included in image data 44. Metadata 44M may include information such as color tone and resolution in image data 44 in addition to character data acquired by OCR.

[0079] Next, at sequence F110 , the CPU 38A of the image forming device 32 transmits the metadata 44M generated at sequence F108 to the digital shadow 32S.

[0080] Furthermore, at sequence F112, the digital shadow 32S records the metadata 44M received from the image forming device 32 in the memory 38M of the digital shadow 32S. In addition, the metadata 44M of the digital shadow 32S is also supplemented with information indicating whether the image data 44 is recorded in the memory 38D of the image forming device 32. In the present embodiment, at the time of sequence F112, the information that the image data 44 is recorded in the memory 38D of the image forming device 32 is also recorded.

[0081] Then, at sequence F114 , the digital shadow 32S notifies the image forming device 32 that the metadata 44M corresponding to the image data 44 is recorded in the memory 38M of the digital shadow 32S.

[0082] Next, in this embodiment, the state of updating the metadata 44M recorded in the memory 38M of the digital shadow 32S will be described. In addition, in this embodiment, the case where the user deletes the image data 44 stored in the above steps will be described as an example.

[0083] (Update of metadata 44M)

[0084] Figure 6 4 is a timing chart showing a state in which the image forming device 32 according to the present embodiment communicates with the digital shadow 32S and updates the metadata 44M associated with the image data 44. Figure 6 As shown, in this embodiment, the metadata 44M is updated through the following steps.

[0085] First, in sequence F202 , the user operates the image forming device 32 to input an instruction to delete the image data 44 recorded in the memory 38D of the image forming device 32 .

[0086] Next, in sequence F204 , the CPU 38A of the image forming device 32 deletes the image data 44 targeted in sequence F202 from the memory 38D of the image forming device 32 .

[0087] Next, as shown in sequence F206, the image forming device 32 notifies the digital shadow 32S of a request for updating of the metadata 44M. Then, the digital shadow 32S that has received the notification executes sequence F208.

[0088] In sequence F208, the CPU of the digital shadow 32S, that is, the CPU 28A of the cloud server 22, notifies the image forming device 32 that the metadata 44M corresponding to the image data 44 has been updated. Then, the CPU 38A of the image forming device 32 that has received the notification executes sequence F210.

[0089] Next, in sequence F210, the CPU 38A of the image forming device 32 sends a notification to the digital shadow 32S to delete the metadata 44M corresponding to the image data 44 deleted in sequence F204. Then, the digital shadow 32S that has received the notification executes sequence F212.

[0090] Furthermore, in sequence F212, the digital shadow 32S updates the information of the metadata 44M corresponding to the image data 44 deleted in sequence F204 according to the notification of the deletion of the metadata 44M received from the image forming device 32. Specifically, the digital shadow 32S records the fact that the information indicating whether the image data 44 corresponding to the metadata 44M is recorded in the memory 38D of the image forming device 32 is deleted from the memory 38D of the image forming device 32.

[0091] Then, at sequence F214 , the digital shadow 32S notifies the image forming device 32 that the metadata 44M corresponding to the image data 44 has been deleted from the memory 38M of the digital shadow 32S.

[0092] However, as described above, the machine learning device 50 in this embodiment receives the image data 44 acquired by the image forming device 32 via the Internet 12, and performs machine learning to detect feature quantities based on the plurality of image data 44. However, it is generally not desirable in terms of security to be able to directly connect to the image forming device 32 existing within the firewall from a device outside the base 30 via the Internet 12. Figure 1 In the structure shown, the machine learning device 50 cannot obtain access rights to refer to the image data 44 recorded in the memory 38D of the image forming device 32 in the base 30. Figure 1 As shown, in the present embodiment, the image forming device 32 disposed in the base 30 cannot communicate directly with the machine learning device 50 .

[0093] Next, the cloud server 22 and the machine learning device 50 in this embodiment cooperate to execute Figure 7 and Figure 8 The steps shown are used to explain the steps for enabling the machine learning device 50 to use the image data 44 held by the image forming device 32 .

[0094] (Step of Making Image Data 44 Available)

[0095] Figure 7The timing diagram shows the state in which the machine learning device 50, the digital shadow 32S and the image forming device 32 involved in the present invention communicate with each other and the machine learning device 50 performs machine learning based on the image data 44 to generate a machine learning completed model. The CPU 28A of the cloud server 22, the CPU 38A of the image forming device 32 and the CPU (not shown) of the machine learning device 50 implement the program stored in the memory 28D, the memory 38D and the recording device 54 respectively. Figure 7 In addition, the machine learning device 50 executes the following steps upon receiving an instruction from a user (not shown). Figure 7 Timing F302 is shown.

[0096] First, in sequence F302, the machine learning device 50 requests the digital shader 32S to provide a list of image data 44 required for machine learning. More specifically, the machine learning device 50 requests the cloud server 22 that virtually performs the operation of the digital shader 32S for image data 44 having target attributes (e.g., resolution, color tone, presence or absence of character data, etc.) in machine learning.

[0097] Next, in sequence F304, the cloud server 22 searches for metadata 44M of the image data 44 having the attributes of the image data 44 requested in sequence F302, and extracts the metadata 44M. More specifically, the cloud server 22 performs Figure 8 The flowchart shown extracts metadata 44M.

[0098] First, in step S402, the CPU 28A of the cloud server 22 sets any one of the digital shadows 32S recorded in the memory 28D as the digital shadow 32S to be searched. Then, the CPU 28A of the cloud server 22 proceeds to step S404.

[0099] Next, in step S404, the CPU 28A of the cloud server 22 determines whether the metadata 44M having the attribute requested in sequence F302 is recorded in the memory 38M of the digital shadow 32S set as the search object in step S402. And, if the determination in step S404 is affirmative, the CPU 28A of the cloud server 22 transfers to step S406. On the other hand, if the determination in step S404 is negative, the CPU 28A of the cloud server 22 transfers to step S408.

[0100] Next, in step S406, the CPU 28A of the cloud server 22 extracts the metadata 44M recorded in the memory 38M of the searched digital shadow 32S. Then, the CPU 28A of the cloud server 22 moves to step S408.

[0101] Next, in step S408, the CPU 28A of the cloud server 22 records the history of requesting the metadata 44M for the digital shadow 32S set as the search target in the memory 28D as metadata request history data 48HM. In addition, the metadata request history data 48HM recorded in step S408 may include the period when the metadata 44M was requested in addition to the determination result in step S404. Then, the CPU 28A of the cloud server 22 transfers to step S410.

[0102] Next, in step S410, the CPU 28A of the cloud server 22 determines whether there are other digital shadows 32S that are not set as search targets in step S402. If the determination in step S410 is positive, the CPU 28A of the cloud server 22 moves to step S402 and sets the other digital shadows 32S as the digital shadows 32S of the next search target. In addition, if the determination in step S410 is negative, the CPU 28A of the cloud server 22 ends the extraction of the metadata 44M and moves to sequence F306.

[0103] Next, in sequence F306, the cloud server 22 sends the metadata 44M, metadata request history data 48HM, and image data request history data 48HR extracted in sequence F304 to the machine learning device 50. Furthermore, the machine learning device 50 determines the image forming device 32 that records the image data 44 used for machine learning in the memory 38D based on the list of metadata 44M received in sequence F306.

[0104] Next, at sequence F308 , the machine learning device 50 requests the cloud server 22 to respond as to whether the image data 44 used in machine learning can be provided.

[0105] Next, at sequence F310 , the cloud server 22 requests the image forming device 32 that has recorded the image data 44 requested from the machine learning device 50 at sequence F308 in the memory 38D to respond as to whether the image data 44 can be provided.

[0106] Next, in sequence F312, the image forming device 32 determines whether the image data 44 requested in sequence F310 can be provided. More specifically, the CPU 38A of the image forming device 32 determines whether the image data 44 requested in sequence F310 can be provided in terms of safety.

[0107] Next, in sequence F314, the image forming device 32 replies to the cloud server 22 whether the image data 44 requested in sequence F310 can be provided. In the present description, any one of the image forming devices 32 replies that the image data 44 can be provided.

[0108] Next, in sequence F316 , the cloud server 22 responds to the machine learning device 50 as to whether the image data 44 can be provided, which was received in sequence F314 .

[0109] Next, at sequence F318 , the machine learning device 50 requests the digital shadow 32S to provide the image data 44 in reply to the request that can be provided.

[0110] Next, at sequence F320, the cloud server 22 transmits the request for providing the image data 44 received at sequence F318 to the image forming device 32 which stores the image data 44 in the memory 38D. Furthermore, the CPU 28A of the cloud server 22 records the history of requesting the image data 44 from the machine learning device 50 at sequence F318 in the memory 28D as image data request history data 48HR.

[0111] Next, at sequence F322 , the image forming device 32 transmits the image data 44 requested at sequence F320 to the cloud server 22 .

[0112] Next, at sequence F324 , the cloud server 22 sends the image data 44 received at sequence F322 to the machine learning device 50 .

[0113] Next, at sequence F326 , the image forming device 32 transmits a notification of completion of transmission of the image data 44 transmitted at sequence F322 to the cloud server 22 .

[0114] Next, at sequence F328 , the cloud server 22 transmits a notification of completion of transmission of the image data 44 transmitted at sequence F324 to the machine learning device 50 .

[0115] Next, at sequence F330 , the machine learning device 50 starts execution of machine learning based on the image data 44 received at sequence F328 .

[0116] Next, in sequence F332 , when the machine learning started in sequence F330 is completed, the machine learning device 50 records the created learning-completed model in the recording device 54 .

[0117] In addition, the machine learning device 50 can determine whether it is necessary to execute the steps after the sequence F308 based on the metadata request history data 48HM and the image data request history data 48HR received from the cloud server 22 in the sequence F306. More specifically, it can be set so that the image data 44 that has a history of being previously requested is not requested to be provided. For example, in the case where it is determined that there is no new image data 44 to be used for learning based on the metadata request history data 48HM received in the sequence F306, the machine learning device 50 does not execute the steps after the sequence F308 and ends a series of steps. And, for example, in the case where it is determined that there is no new image data 44 to be used for learning based on the image data request history data 48HR received in the sequence F306, the machine learning device 50 does not execute the steps after the sequence F308 and ends a series of steps.

[0118] Next, the function and effect of the cloud server 22 included in the machine learning system 10 according to the present embodiment will be described.

[0119] (Effect)

[0120] The cloud server 22 of the present embodiment includes a memory 28D, and the memory 28D records metadata 44M associated with the image data 44 generated by the image forming device 32. In addition, when the metadata 44M is requested from the machine learning device 50, the CPU 28A of the cloud server 22 sends the metadata 44M recorded in the memory 28D to the machine learning device 50, so that the machine learning device 50 can use the image data 44. Thus, according to the cloud server 22 involved in the present embodiment, the machine learning device 50 can use the image data 44 generated by the image forming device 32 that cannot directly communicate with the machine learning device 50 and stored in the memory 38D of the image forming device 32.

[0121] Furthermore, when the image data 44 associated with the metadata 44M is requested from the machine learning device 50, the CPU 28A of the cloud server 22 of the present embodiment enables the machine learning device 50 to use the image data 44. Thus, according to the cloud server 22 involved in the present embodiment, the machine learning device 50 can use the image data 44 stored in the image forming device 32 that cannot directly communicate with the machine learning device 50.

[0122] Furthermore, the CPU 28A of the cloud server 22 of the present embodiment receives the image data 44 from the image forming device 32 storing the image data 44, and transmits the image data 44 to the machine learning device 50. Therefore, according to the cloud server 22 involved in the present embodiment, the machine learning device 50 can use the image data 44 stored in the image forming device 32 that cannot communicate directly with the machine learning device 50.

[0123] Furthermore, the CPU 28A of the cloud server 22 of the present embodiment records the history of requesting the image data 44 from the machine learning device 50 in the memory 28D as image data request history data 48HR. Thus, according to the cloud server 22 involved in the present embodiment, compared with the case where the history of sending the image data 44 to the machine learning device 50 is not recorded, the machine learning device 50 can be notified that the machine learning device 50 is repeatedly requesting the same image data 44. In other words, according to the cloud server 22 involved in the present embodiment, it is possible to notify the machine learning device 50 that the image data 44 requested by the machine learning device 50 is already available.

[0124] Furthermore, whenever the image forming device 32 generates the image data 44, the CPU 28A of the cloud server 22 of the present embodiment records the metadata 44M in the memory 38M. Thus, according to the cloud server 22 of the present embodiment, the time at which the image data 44 can be used by the machine learning device 50 can be advanced compared to the case where the metadata 44M is generated at a different time from the generation of the image data 44.

[0125] Furthermore, when the image data 44 associated with the metadata 44M is deleted, the CPU 28A of the cloud server 22 of the present embodiment adds a record of the deletion of the image data 44 to the metadata 44M. Thus, according to the cloud server 22 of the present embodiment, when the image data 44 associated with the metadata 44M is deleted from the storage device, it is easier to normally complete the usability processing than when the metadata 44M recorded in the memory 38M is retained.

[0126] Furthermore, when the metadata 44M is requested from the machine learning device 50, the program of the present embodiment causes the CPU 38A to execute a process of sending the metadata 44M recorded in the memory 38M to the machine learning device 50. That is, the program causes the CPU 38A to execute a process of enabling the machine learning device 50 to process the image data 44 associated with the metadata 44M. Thus, according to the program involved in the present embodiment, a program that enables the machine learning device 50 to use the image data 44 stored in an area that is inaccessible from the machine learning device 50 can be obtained.

[0127] Next, a second embodiment of the present invention will be described. Note that the configuration of the machine learning system 10 according to the second embodiment is the same as that of the first embodiment, and therefore the same reference numerals as those of the first embodiment are used and a detailed description thereof is omitted.

[0128] [Second embodiment]

[0129] In the machine learning system 10 involved in the second embodiment, as Fig. 9 As shown, the procedure of enabling the machine learning device 50 to use the image data 44 held by the image forming device 32 is different from that of the first embodiment.

[0130] (Step of Making Image Data 44 Available)

[0131] exist Fig. 9 In the embodiment, the steps from sequence F502 to sequence F518 are the same as the steps from sequence F302 to sequence F318 in the first embodiment.

[0132] In sequence F520, the cloud server 22 requests the image forming device 32 to send the image data 44 received in sequence F518 to the machine learning device 50, and the image forming device 32 stores the image data 44 in the memory 28D. More specifically, in sequence F520, the cloud server 22 notifies the image forming device 32 storing the image data 44 in the memory 28D of information including the network address, authentication information, etc. of the machine learning device 50. In addition, in sequence F520, the cloud server 22 notifies the image forming device 32 and requests that the image data 44 be sent to the machine learning device 50.

[0133] Next, at sequence F522 , the image forming device 32 transmits the image data 44 requested at sequence F520 to the machine learning device 50 .

[0134] Next, at sequence F524 , the image forming device 32 transmits a notification of completion of transmission of the image data 44 transmitted at sequence F522 to the machine learning device 50 .

[0135] Note that the steps from sequence F530 to sequence F532 are the same as the steps from sequence F330 to sequence F332 in the first embodiment.

[0136] Next, the function and effect of the cloud server 22 included in the machine learning system 10 according to the present embodiment will be described.

[0137] (Effect)

[0138] In the information processing system of the present embodiment, the CPU 28A of the cloud server 22 causes the device storing the image data 44 to send the image data 44 to the machine learning device 50. Thus, according to the information processing system involved in the present embodiment, compared with the case where the CPU 28A of the cloud server 22 receives the image data 44 and sends it to the machine learning device 50, the time until the machine learning device 50 receives the image data 44 can be shortened.

[0139] In addition, in this embodiment, a system having the same configuration as that of the first embodiment can also obtain the same operations and effects as those of the first embodiment.

[0140] The third embodiment of the present invention will now be described. The configuration of the machine learning system 10 according to the third embodiment is the same as that of the first embodiment or the second embodiment, and thus the same reference numerals as those of the first embodiment or the second embodiment are used, and detailed descriptions thereof are omitted.

[0141] [Third Embodiment]

[0142] In the machine learning system 10 involved in the third embodiment, as Fig.10 As shown, the procedure of enabling the machine learning device 50 to use the image data 44 held by the image forming device 32 is different from that of the first embodiment and the second embodiment.

[0143] (Step of Making Image Data 44 Available)

[0144] exist Fig.10 In the embodiment, the steps from sequence F602 to sequence F614 are the same as the steps from sequence F302 to sequence F314 in the first embodiment.

[0145] In sequence F616, when the image data 44 can be provided as received in sequence F614, the cloud server 22 sends a method of accessing the storage area in the image forming device 32 storing the image data 44 in the memory 38D to the machine learning device 50. More specifically, in sequence F616, the cloud server 22 sends information including the network address, authentication information, etc. of the machine learning device 50 together with the result of whether the image data 44 can be provided as received in sequence F614.

[0146] Next, at sequence F618 , the machine learning device 50 requests the image forming device 32 storing the image data 44 requested at sequence F614 in the memory 38D to provide the image data 44 based on the information received at sequence F616 .

[0147] Next, at sequence F620 , the image forming device 32 transmits the image data 44 requested at sequence F618 to the machine learning device 50 .

[0148] Next, at sequence F622 , the image forming device 32 transmits a notification of completion of transmission of the image data 44 transmitted at sequence F620 to the machine learning device 50 .

[0149] Note that the steps from sequence F630 to sequence F632 are the same as the steps from sequence F330 to sequence F332 in the first embodiment.

[0150] Next, the function and effect of the cloud server 22 included in the machine learning system 10 according to the present embodiment will be described.

[0151] When the metadata 44M is requested from the machine learning device 50, the CPU 28A of the cloud server 22 of the present embodiment transmits a method of accessing the storage area storing the image data 44 together with the metadata 44M to the machine learning device 50. Therefore, according to the cloud server 22 involved in the present embodiment, compared with the case where the CPU 28A of the cloud server 22 receives the image data 44 and transmits it to the machine learning device 50, the time until the machine learning device 50 can use the image data 44 can be shortened.

[0152] Furthermore, the CPU 28A of the cloud server 22 of the present embodiment records the history of requesting the metadata 44M from the machine learning device 50 as metadata request history data 48HM. Therefore, according to the cloud server 22 involved in the present embodiment, compared with the case where the history of sending the image data 44 to the machine learning device 50 is not recorded, the machine learning device 50 can be notified that the machine learning device 50 is repeatedly requesting the same image data 44. In other words, according to the cloud server 22 involved in the present embodiment, it is possible to notify the machine learning device 50 that the requested image data 44 is already available.

[0153] In addition, in the present embodiment, a system having the same configuration as that of the first embodiment or the second embodiment can also obtain the same operations and effects as those of the first embodiment or the second embodiment.

[0154] [Other Modifications]

[0155] In addition, in the above description, whenever the image forming device 32 generates image data 44, the CPU 28A of the cloud server 22 stores the metadata 44M in the memory 38M, but the technology involved in the present invention is not limited to this. For example, the CPU 28A of the cloud server 22 may not store the metadata 44M until a plurality of image data 44 are generated. In addition, for example, the CPU 28A of the cloud server 22 may periodically access the image forming device 32, and when it is determined that the image data 44 is newly generated, the metadata 44M is created and stored in the memory 38M (so-called batch processing).

[0156] Furthermore, in the above description, the CPU 28A of the cloud server 22 stores the history of requesting the image data 44 from the machine learning device 50 as the metadata request history data 48HM in the memory 28D, but the technology involved in the present invention is not limited to this. For example, the CPU 28A of the cloud server 22 may not generate the metadata request history data 48HM.

[0157] Furthermore, in the above description, the CPU 28A of the cloud server 22 stores the history of requesting metadata 44M from the machine learning device 50 as metadata request history data 48HM in the memory 28D, but the technology involved in the present invention is not limited to this. For example, the CPU 28A of the cloud server 22 may not generate the metadata request history data 48HM.

[0158] Furthermore, in the above-mentioned embodiment, the machine learning device 50 is taken as an example of an external device, but the technology involved in the present invention is not limited to this. As an example of an external device involved in the present invention, for example, it can also be a device that simply obtains statistics on the utilization status of the image forming device 32. That is, as long as it is a device that uses the actual data (image data 44) generated by the image forming device 32 in this embodiment, it can be included in the external device involved in the present invention.

[0159] Furthermore, in the above-mentioned embodiment, the cloud server 22 that forms the digital shadow 32S is used as an example of an information processing system, but the technology involved in the present invention is not limited to this. As an example of the "information processing system" involved in the present invention, for example, it can also be a recording device 54 that simply records metadata 44M based on image data 44 without producing a digital shadow 32S. That is, as long as it is a device that records attribute data associated with actual data generated by the image forming device 32 in this embodiment and can use the actual data when the attribute data is requested, it can be included in the information processing system involved in the present invention.

[0160] Furthermore, in the above embodiment, the cloud server 22 is described as being composed of a single device, but may be composed of a plurality of devices. That is, the "information processing system" in this embodiment may be composed of a single device or a plurality of devices.

[0161] Furthermore, in the above embodiment, the machine learning system 10 is described as having a single cloud server 22 , but the technology involved in the present invention is not limited thereto. That is, the machine learning system 10 may also have a plurality of cloud servers 22 .

[0162] Furthermore, in the above-mentioned embodiment, the image data 44 of the image forming device 32 is used as an example of actual data, but the technology involved in the present invention is not limited to this. As an example of "actual data" involved in the present invention, for example, in addition to the image data 44, data produced using a measuring device including sound data and image data can also be used, and data artificially produced such as text data and modeling data can also be used. In these cases, as attribute data associated with the actual data, as long as it can be associated with the actual data, the date and time of generation of the actual data, the device that generated the actual data, and other data produced based on the actual data can also be included in the "attribute data" involved in the present invention. And, in other words, as long as it can communicate with the processor, devices such as measuring devices including sound and images, text production devices, modeling devices, etc. can also be included in the "device for communicating with the processor" involved in the present invention.

[0163] Furthermore, in the above-mentioned embodiments, the processor refers to a processor in a broad sense, and includes general-purpose processors (for example, the aforementioned CPU, etc.) and special-purpose processors (for example, GPU: Graphics Processing Unit (graphics processor), ASIC: Application Specific Integrated Circuit (application-specific integrated circuit), FPGA: Field Programmable Gate Array (field programmable gate array), programmable logic devices, etc.).

[0164] Furthermore, the actions of the processors in the above embodiments may be performed not only by one processor but also by a plurality of processors located in physically separate locations in collaboration. Furthermore, the order of the actions of the processors is not limited to the order described in the above embodiments but may be changed as appropriate.

[0165] In addition, the processing executed by reading the software (program) by the CPU in the above-mentioned embodiments may also be executed by various processors other than the CPU. As the processor at this time, FPGA (Field-Programmable Gate Array) and other PLD (Programmable Logic Device) whose circuit structure can be changed after manufacturing and ASIC (Application Specific Integrated Circuit) and other processors with circuit structures specially designed for executing specific processing, i.e., dedicated circuits, etc. are exemplified. Moreover, the processing may be executed by one of these various processors, or by a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs, and a combination of a CPU and an FPGA, etc.). Moreover, more specifically, the hardware structure of these various processors is a circuit composed of a combination of circuit elements such as semiconductor elements.

[0166] Furthermore, in the above-mentioned embodiments, the processing program is pre-stored (installed) in the memory, but the present invention is not limited thereto. The program may also be provided in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. Furthermore, the program may be downloaded from an external device via a network.

[0167] The embodiments of the present invention are described above with reference to the accompanying drawings, but it is obvious that any technician in the technical field described in the present invention can think of various changes or applications within the scope of the technical ideas described in the claims, and can understand that these of course also belong to the technical scope of the present invention.

[0168] In addition, the following further describes the embodiment of the present invention. (((1)))

[0170] An information processing system comprising:

[0171] one or more processors; and

[0172] one or more recording units, the recording units recording attribute data associated with actual data, the actual data being generated by a device communicating with the processor,

[0173] The processor performs processing such that, when the attribute data is requested from an external device, the attribute data recorded in the recording unit is transmitted to the external device so that the external device can use the actual data. (((2)))

[0175] The information processing system according to (((1))), wherein:

[0176] The processor performs processing for making the actual data available to the external device when the actual data associated with the attribute data is requested from the external device. (((3)))

[0178] The information processing system according to (((2))), wherein:

[0179] The processor performs a process of receiving the actual data from the device storing the actual data and transmitting the actual data to the external device. (((4)))

[0181] The information processing system according to (((2))) or (((3))), wherein:

[0182] The processor causes the device storing the actual data to transmit the actual data to the external device. (((5)))

[0184] The information processing system according to any one of (((2))) to (((4))), wherein:

[0185] The processor records a history of requesting the actual data from the external device. (((6)))

[0187] The information processing system according to (((1))), wherein:

[0188] The processor performs a process of transmitting, to the external device, a method of accessing a storage area storing the actual data together with the attribute data, when the attribute data is requested from the external device. (((7)))

[0190] The information processing system according to (((6))), wherein:

[0191] The processor records a history of requesting the attribute data from the external device. (((8)))

[0193] The information processing system according to any one of (((1))) to (((7))), wherein:

[0194] The processor performs processing to record the attribute data in the recording unit every time the device generates the actual data. (((9)))

[0196] The information processing system according to any one of (((1))) to (((8))), wherein:

[0197] The processor performs a process of adding a history of the deletion of the actual data to the attributes data when the actual data associated with the attributes data is deleted. (((10)))

[0199] A program that causes a computer to execute the following processing:

[0200] When attribute data is requested from an external device, the attribute data recorded in the recording unit is transmitted to the external device so that the external device can use actual data associated with the attribute data.

[0201] According to the information processing system of (((1))), it is possible to enable an external device to utilize actual data generated by a device that cannot directly communicate with the external device.

[0202] According to the information processing system involved in (((2))), it is possible to enable an external device to use actual data stored in a device that cannot directly communicate with the external device.

[0203] According to the information processing system involved in (((3))), it is possible to enable an external device to use actual data stored in a device that cannot directly communicate with the external device.

[0204] According to the information processing system involved in (((4))), compared with a case where the processor receives the actual data and transmits it to the external device, the time until the external device receives the actual data can be shortened.

[0205] According to the information processing system of (((5))), it is possible to notify the external device that the external device is repeatedly requesting the same actual data.

[0206] According to the information processing system of (((6))), compared with a case where the processor receives the actual data and transmits it to the external device, the time until the actual data becomes available to the external device can be shortened.

[0207] According to the information processing system of (((7))), it is possible to notify the external device that the external device is repeatedly requesting the same actual data.

[0208] According to the information processing system of (((8))), the time at which the actual data can be used by the external device can be shortened compared to a case where the attribute data is generated at a time different from the time at which the actual data is generated.

[0209] According to the information processing system of (((9))), when actual data associated with attribute data is deleted from the storage device, it is easier to normally complete the usability processing than when the attribute data recorded in the recording unit is retained.

[0210] According to the program of (((10))), it is possible to obtain a program that enables an external device to utilize actual data stored in an area that is inaccessible from the external device.

[0211] The above-mentioned embodiments of the present invention are provided for the purpose of illustration and description. In addition, the embodiments of the present invention do not fully and exhaustively include the present invention, and do not limit the present invention to the disclosed methods. Obviously, various modifications and changes are self-evident to those skilled in the art to which the present invention belongs. The present embodiment is selected and described in order to most easily understand the principles of the present invention and its application. Thus, other technicians in the art can understand the present invention through various variations optimized for specific uses assumed to be various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

Claims

1. An information processing system comprising: one or more processors; and one or more recording units, the recording units recording attribute data associated with actual data, the actual data being generated by a device communicating with the processor, The processor performs processing such that, when the attribute data is requested from an external device, the attribute data recorded in the recording unit is transmitted to the external device so that the external device can use the actual data.

2. The information processing system according to claim 1, wherein: The processor performs processing for making the actual data available to the external device when the actual data associated with the attribute data is requested from the external device.

3. The information processing system according to claim 2, wherein: The processor performs a process of receiving the actual data from the device storing the actual data and transmitting the actual data to the external device.

4. The information processing system according to claim 2 or 3, wherein: The processor causes the device storing the actual data to transmit the actual data to the external device.

5. The information processing system according to any one of claims 2 to 4, wherein: The processor records a history of requesting the actual data from the external device.

6. The information processing system according to claim 1, wherein: The processor performs a process of transmitting, to the external device, a method of accessing a storage area storing the actual data together with the attribute data, when the attribute data is requested from the external device.

7. The information processing system according to claim 6, wherein: The processor records a history of requesting the attribute data from the external device.

8. The information processing system according to any one of claims 1 to 7, wherein: The processor performs processing to record the attribute data in the recording unit every time the device generates the actual data.

9. The information processing system according to any one of claims 1 to 8, wherein: The processor performs a process of adding a history of the deletion of the actual data to the attributes data when the actual data associated with the attributes data is deleted.

10. A storage medium storing a program for causing a computer to execute the following processing: When attribute data is requested from an external device, the attribute data recorded in the recording unit is transmitted to the external device so that the external device can use actual data associated with the attribute data.

11. An information processing method comprising the following steps: When attribute data is requested from an external device, the attribute data recorded in the recording unit is transmitted to the external device so that the external device can use actual data associated with the attribute data.

Images