Electronic device, electronic system, noise determination method, and program

Abstract

To improve the accuracy of determining whether noise is normal noise not caused by an abnormality in a device or abnormal noise caused by the abnormality in the device.SOLUTION: An electronic device according to the present disclosure which is equipped with an electrical component that generates signals, includes a noise determination unit that determines the type of noise indicating that the noise contained in the signal in the electronic device is normal noise not caused by an abnormality in the electronic device, or abnormal noise caused by the abnormality in the electronic device on the basis of the operating state of the electronic device.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to an electronic device, an electronic system, a noise determination method, and a program.

Background Art

[0002] Conventionally, when it is determined that the occurrence of an abnormality in an electrical component is caused by noise, an electronic device such as an image forming apparatus that facilitates identification of the noise source has been proposed (see Patent Document 1). In this case, when the number of occurrences of noise exceeds a predetermined threshold value, the electronic device determines that an abnormality has occurred in itself and forcibly stops the operation of the device.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, since the number of occurrences of noise varies depending on the operating state of the device such as standby or job operation, even when the number of occurrences of noise within a predetermined time exceeds the threshold value, there may actually be no abnormality in the device. In such a case, if the device is forcibly stopped, the user will be confused.

[0004] The present disclosure has been made in view of the above circumstances, and an object thereof is to improve the accuracy of determining whether noise is normal noise not caused by an abnormality of the device or abnormal noise caused by an abnormality of the device.

Means for Solving the Problems

[0005] The invention according to claim 1 is an electronic device provided with an electrical component that generates a signal, and based on the operating state of the electronic device, a noise determination unit that determines the type of noise indicating that the noise included in the signal in the electronic device is normal noise not caused by an abnormality of the electronic device or abnormal noise caused by an abnormality of the electronic device.

Effects of the Invention

[0006] As described above, according to the present disclosure, there is an effect that the accuracy of determining whether the noise is normal noise not caused by an abnormality of the device or abnormal noise caused by an abnormality of the device can be improved.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Modes for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0009] 〔Overall Configuration of the System〕 First, the overall configuration of the communication system according to this embodiment will be described with reference to FIG. 1. FIG. 1 is an overall configuration diagram of the communication system according to the embodiment.

[0010] As shown in FIG. 1, the communication system 1 of this embodiment is constructed by an image forming apparatus 3, a machine learning server 5, a data management server 7, and a user terminal 9. The image forming apparatus 3, the machine learning server 5, the data management server 7, and the user terminal 9 can communicate via a communication network 100 such as the Internet or a LAN (Local Area Network). The connection form to the communication network 100 may be either wireless or wired.

[0011] The image forming apparatus 3 is a multifunction peripheral (MFP), a printer, a facsimile apparatus, or the like. The image forming apparatus 3 is equipped with a learned machine learning model Mb of AI (Artificial Intelligence) and performs predetermined inferences. For example, the image forming apparatus 3 is provided with electrical components that generate signals, and based on the operating state of the image forming apparatus 3, it determines whether the noise contained in the signals within the image forming apparatus is normal noise not caused by an abnormality of the image forming apparatus or abnormal noise caused by an abnormality of the image forming apparatus. Note that the image forming apparatus is an example of an electronic device.

[0012] The machine learning server 5 performs machine learning of the machine learning model Ma in order to generate the learned machine learning model Mb. Note that an image forming system 2 is constructed by the image forming apparatus 3 and the machine learning server 5.

[0013] The data management server 7 collects learning data used for performing machine learning in the machine learning server 5 from external devices such as the image forming apparatus 3 and provides it to the machine learning server 5.

[0014] The user terminal 9 sends a request to execute a job such as printing to the image forming apparatus 3. Note that the user terminal 9 is, for example, a PC (Personal Computer), a tablet terminal, a smart watch, or the like.

[0015] 〔Hardware Configuration〕 <Image Forming Apparatus> FIG. 2 is an electrical hardware configuration diagram of a multifunction peripheral as an image forming apparatus.

[0016] The image forming apparatus 3 includes a controller 310, a short-range communication circuit 320, an engine control device 300, an operation panel 340, and a network I / F 350.

[0017] Among these, the controller 310 has a CPU 301 which is a main part of a computer, a system memory (MEM-P) 302, a north bridge (NB) 303, a south bridge (SB) 304, an ASIC (Application Specific Integrated Circuit) 306, a local memory (MEM-C) 307 which is a storage unit, an HDD controller 308, and an HD 309 which is a storage unit, and is configured to connect between the NB 303 and the ASIC 306 by an AGP (Accelerated Graphics Port) bus 321.

[0018] Among these, the CPU 301 is a control unit that performs overall control of the MFP 3. The NB 303 is a bridge for connecting the CPU 301, the MEM-P 302, the SB 304, and the AGP bus 321, and has a memory controller that controls reading and writing to the MEM-P 302, a PCI (Peripheral Component Interconnect) master, and an AGP target.

[0019] MEM-P302 consists of a ROM302a which is a memory for storing programs and data for realizing each function of the controller 310, and a RAM302b which is used as a memory for expanding programs and data and for drawing during memory printing. Note that the programs stored in the RAM302b may be configured to be provided by recording them in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0020] SB304 is a bridge for connecting NB303 with PCI devices and peripheral devices. ASIC306 is an IC (Integrated Circuit) for image processing applications having hardware elements for image processing, and has the role of a bridge for connecting an AGP bus 321, a PCI bus 322, an HDD 308, and MEM-C307 respectively. This ASIC306 consists of a PCI target and an AGP master, an arbiter (ARB) forming the core of the ASIC306, a memory controller for controlling MEM-C307, a plurality of DMACs (Direct Memory Access Controllers) for performing operations such as rotation of image data by means of hardware logic, etc., and a PCI unit for performing data transfer via the PCI bus 322 between the scanner unit 331 and the printer unit 332. Note that interfaces for USB (Universal Serial Bus) and IEEE1334 (Institute of Electrical and Electronics Engineers 1394) may be connected to the ASIC306.

[0021] MEM-C307 is local memory used as an image buffer for copying and a code buffer. HD309 is storage for accumulating image data, font data used at the time of printing, and forms. HD309 controls the reading or writing of data to HD309 according to the control of CPU301. The AGP bus 321 is a bus interface for a graphics accelerator card proposed to speed up graphic processing. By directly accessing MEM-P302 with high throughput, the graphics accelerator card can be made faster.

[0022] In addition, the short-range communication circuit 320 is provided with a short-range communication circuit 320a. The short-range communication circuit 320 is a communication circuit such as NFC or Bluetooth.

[0023] Furthermore, the engine control device 300 is composed of a scanner unit 331 and a printer unit 332. Also, the operation panel 340 includes a panel display unit 340a such as a touch panel that displays current setting values, selection screens, etc. and receives inputs from the operator, and an operation panel 340b consisting of a numeric keypad that receives setting values of conditions related to image formation such as density setting conditions and a start key that receives a copy start instruction. The controller 310 controls the entire MFP3, for example, controls drawing, communication, inputs from the operation panel 340, etc. The scanner unit 331 or the printer unit 332 includes an image processing part such as error diffusion and gamma conversion.

[0024] Note that the MFP3 can sequentially switch and select the document box function, copy function, printer function, and facsimile function by the application switching key on the operation panel 340. When the document box function is selected, it becomes the document box mode, when the copy function is selected, it becomes the copy mode, when the printer function is selected, it becomes the printer mode, and when the facsimile mode is selected, it becomes the facsimile mode.

[0025] In addition, the network I / F 350 is an interface for performing data communication using the communication network 100. The short-range communication circuit 320 and the network I / F 350 are electrically connected to the ASIC 306 via the PCI bus 322.

[0026] <Engine control device> FIG. 3 is an electrical hardware configuration diagram of the engine control device in FIG. 2.

[0027] As shown in FIG. 3, the engine control device 330 includes a CPU (Central Processing Unit) 400, a ROM (Read Only Memory) 401, a RAM (Random Access Memory) 402, an SSD (Solid State Drive) 403, a GPU (Graphics Processing Unit) 404, a connection I / F (Interface) 405, various sensors 406, and a bus line 410.

[0028] Among these, the CPU 400 controls the overall operation of the engine control device 330. The ROM 401 stores programs used for driving the CPU 400 such as IPL. The RAM 402 is used as a work area for the CPU 400.

[0029] The SSD 403 reads or writes various data according to the control of the CPU 400. Note that an HDD (Hard Disk Drive) may be used instead of the SSD 404. The GPU 404 is a semiconductor chip that handles graphics and the like. The GPU 404 may be omitted.

[0030] The connection I / F 405 is an interface for connecting various devices or electrical components provided in the image forming apparatus 3.

[0031] The various sensors 406 are a group of sensors for detecting the operation and environment (for example, temperature, humidity) of the engine control device 330 and the like.

[0032] The bus line 410 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401.

[0033] <Machine learning server> Figure 4 is an electrical hardware configuration diagram of a machine learning server, a data management server, and a user terminal. Note that since the data management server 7 and the user terminal 9 have the same configuration as the machine learning server 5, the machine learning server 5 will be described below.

[0034] As shown in Figure 4, the machine learning server 5, as a computer, as shown in Figure 4, includes a CPU (Central Processing Unit) 500, a ROM (Read Only Memory) 501, a RAM (Random Access Memory) 502, an SSD (Solid State Drive) 503, a GPU (Graphics Processing Unit) 504, an external device connection I / F (Interface) 505, a network I / F 506, a display 507, an operation unit 508, a media I / F 509, and a bus line 510.

[0035] Among these, the CPU 500 controls the operation of the entire machine learning server 5. The ROM 501 stores programs used for driving the CPU 500 such as IPL. The RAM 502 is used as a work area for the CPU 500.

[0036] The SSD 503 reads or writes various data according to the control of the CPU 500. Note that an HDD (Hard Disk Drive) may be used instead of the SSD 504. The GPU 504 is a semiconductor chip that handles graphics, etc.

[0037] The external device connection I / F 505 is an interface for connecting various external devices. The external devices in this case are a display, a speaker, a keyboard, a mouse, a USB memory, and a printer, etc.

[0038] The network I / F 506 is an interface for data communication via the communication network 100.

[0039] The display 507 is a type of display means such as a liquid crystal or an organic EL (Electro Luminescence) that displays various images.

[0040] The operation unit 508 is a keyboard, a pointing device, etc., and is an example of input means for receiving input operations such as characters, numerical values, and various instructions.

[0041] The media I / F 509 controls the reading or writing (storage) of data to / from a recording medium 509m such as a flash memory. The recording medium 509m includes DVDs, Blu-ray Disc (registered trademark), etc.

[0042] The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501.

[0043] 〔Functional configuration of the embodiment〕 Subsequently, the functional configuration of this embodiment will be described with reference to FIGS. 5 and 6.

[0044] <Learning phase> FIG. 5 is a functional block diagram of the machine learning server in the learning phase. As shown in FIG. 5, the machine learning server 5 in the learning phase has a receiving unit 50, an input unit 51, a learning unit 52, and a transmitting unit 59. The receiving unit 50 and the transmitting unit 59 are functions realized by commands from the CPU 301 in FIG. 2 based on a program. Also, the input unit 51 and the learning unit 52 are functions realized by commands from the CPU 400 or the GPU 404 in FIG. 3 based on a program.

[0045] The receiving unit 50 receives learning data from the data management server 7. The learning data includes a dataset of input data and correct answer data. The input data includes signal levels of a plurality of signals s1 to sn, operation state data of the image forming apparatus, ON timing data indicating ON timing, paper conveyance position data indicating the paper conveyance position, and circuit data indicating the impedance of a circuit. The learning data also includes, as each correct answer data (teacher data) paired with each input data, a result indicating whether it is normal noise or abnormal noise.

[0046] Among these, the signals s1 to sn are, for example, a motor lock detection signal, an interlock signal, etc. The motor lock detection signal is a signal for notifying that the rotation of the motor has stabilized. The interlock signal is a signal indicating the open / closed state of the interlock. Since the interlock is a switch that mechanically turns ON or OFF a relatively high voltage, there is a high possibility that large noise or inrush current will occur at the timing when it is turned ON.

[0047] The operation state data of the image forming apparatus is, for example, the operation state of the engine control device 330, such as a warming-up state, a standby state, an energy-saving state, a printing state, etc. Also, when classifying each state in detail, there are further a plurality of different states. FIG. 7 is a diagram for explaining the detailed operation state of the image forming apparatus. Since the control method of the image forming apparatus 3 is different for each operation state of the image forming apparatus 3, there are differences in the generation of noise levels, etc., in each state, which is relevant to the inference of noise determination.

[0048] The ON timing data is data indicating the timing at which electrical components such as a motor and a solenoid are turned ON within the engine control device 330. The ON timing data is, for example, data indicating the timing at which a relay switch is turned ON, the timing at which a heater is turned ON, and the timing at which a high-voltage power supply is turned ON. Since there is a high possibility that relatively large noise will be generated during the operation of the electrical component, the state of the load control state (ON or OFF) is relevant to the inference of noise determination.

[0049] The paper passing position data is detected within the engine control device 330 and indicates the paper passing position in the image forming apparatus during printing. Since the printing paper holds the electric charge generated by friction with other papers and can become a noise source by discharging the electric charge while passing the paper, it is relevant to the inference of noise determination.

[0050] The circuit data is, for example, external circuit data indicating the impedance of the circuit of the electrical component which is the source of the signals s1~s from the image forming apparatus 3 outside the engine control device 330. Since there are circuit characteristics such that a signal is liable to receive noise when the impedance is high and a signal is less liable to receive noise when the impedance is low, it is relevant to the inference of noise determination.

[0051] The input unit 51 acquires a plurality of signals s1~sn, the operation state data of the image forming apparatus 3, the ON timing data indicating the ON timing, the paper passing position data indicating the paper passing position in the image forming apparatus 3, and the circuit data from the reception unit 50 and inputs them to the learning unit 52.

[0052] The learning unit 52 has a machine learning model Ma and generates a learned machine learning model Mb capable of high-precision output by machine learning using a machine learning algorithm such as a neural network.

[0053] The machine learning model Ma of the present embodiment infers the type of noise indicating that the noise included in each of the signals s1~sn is normal noise not caused by an abnormality of the image forming apparatus 3 or abnormal noise caused by an abnormality of the image forming apparatus 3 and outputs the data. For example, the learning unit 52 uses the above-described signals and the like as input data and outputs the result (type of noise) indicating that the noise included in the signal (riding on the signal) is normal noise or abnormal noise as the output data.

[0054] Further, the learning unit 52 has a comparison and modification unit 53. The comparison and modification unit 53 obtains the error E between the output data from the machine learning model Ma and the correct answer data, and calculates the loss L representing the error E using a loss function. Further, the comparison and modification unit 53 updates the connection weight coefficients between the nodes of the neural network so that the loss becomes smaller (so that the loss L approaches 0) based on the loss L. The comparison and modification unit 53 updates the connection weight coefficients between the nodes of the neural network using, for example, the error backpropagation method. The error backpropagation method is a method of adjusting the connection weight coefficients between the nodes of each neural network so that the error E becomes smaller.

[0055] For example, in the present embodiment, the comparison and modification unit 53 compares the output data (normal noise or abnormal noise) output from the machine learning model Ma with the correct answer data (normal noise or abnormal noise), and changes the model parameters of the machine learning model Ma according to the error. Thereby, the learning unit 52 can perform machine learning of the machine learning model Ma and generate a learned machine learning model Mb.

[0056] <Inference phase> FIG. 6 is a functional block diagram of the image forming apparatus 3 in the inference phase. As shown in FIG. 6, the image forming apparatus 3 in the inference phase includes a receiving unit 30, an input unit 31, a noise determination unit 35 as an inference unit, an abnormal number counting unit 37, a display control unit 38, and a transmitting unit 39.

[0057] The receiving unit 30, the display control unit 38, and the transmitting unit 39 are functions realized by commands from the CPU 301 in FIG. 2 based on a program. Further, the input unit 31, the noise determination unit 35, and the abnormal number counting unit 37 are functions realized by commands from the CPU 400 or the GPU 404 in FIG. 3 based on a program.

[0058] The receiving unit 30 receives the data of the learned machine learning model Mb from the machine learning server 5 and passes it to the noise determination unit 35.

[0059] The input unit 31 is outside the engine control device 330 and acquires a plurality of signals s1 to sn from each electrical component or the like inside the image forming apparatus 3, detects the signal levels of the respective signals, and inputs them to the noise determination unit 35. Inside the housing of the image forming apparatus 3, noise tends to propagate to the plurality of signals s1 to sn, and noise is likely to be superimposed on the plurality of signals s1 to sn at the same time.

[0060] The noise determination unit 35 has a learned machine learning model Mb. The noise determination unit 35 inputs a plurality of signals s1 to sn from the input unit 31 and acquires each data (operation state data of the image forming apparatus, ON timing data, paper feed position data, circuit data) from inside the engine control device 330. Note that the circuit data is stored in the storage unit of the RAM 402 or the SSD 403.

[0061] Further, the noise determination unit 35 inputs, as input data to the learned machine learning model Mb, a plurality of signals s1 to sn, operation state data of the image forming apparatus, ON timing data, paper feed position data, and circuit data, and causes the learned machine learning model Mb to output, as output data, a determination result (inference result) indicating whether the type of noise included in the plurality of signals s1 to sn is normal noise or abnormal noise. Thereby, the noise determination unit 35 outputs the output data, which is the determination result, to the abnormal number counting unit 37. Note that when the output data output from the learned machine learning model Mb indicates an inference result that the noise is normal noise, the noise determination unit 35 may not output this output data to the abnormal number counting unit 37.

[0062] The abnormal number counting unit 37 counts the number of abnormal noises acquired from the noise determination unit 35, and when the number of abnormal occurrences within a predetermined time exceeds a threshold value (for example, 5 times per second), determines that an abnormality has occurred in the image forming apparatus 3.

[0063] When the display control unit 38 determines that an abnormality has occurred in the image forming apparatus 3 by the abnormal number counting unit 37, the display control unit 38 displays information indicating that an abnormality has occurred on the panel display unit 340a shown in FIG. 2.

[0064] When the abnormality counter 37 determines that an abnormality has occurred in the image forming apparatus 3, the transmission unit 39 transmits information indicating that an abnormality has occurred from the network I / F 350 shown in FIG. 2 to a server or the like of a sales office or manufacturer of the image forming apparatus 3.

[0065] 〔Processing or operation of the embodiment〕 Subsequently, the processing or operation of this embodiment will be described with reference to FIGS. 8 to 10.

[0066] <Processing of the entire communication system> First, the overall processing of the communication system 1 will be described with reference to FIG. 8. FIG. 8 is a sequence diagram showing the processing of the communication system.

[0067] S11: The data management server 7 transmits learning data for machine learning to the machine learning server 5. As a result, the receiving unit 50 of the machine learning server 5 receives the learning data.

[0068] S12: The machine learning server 5 performs learning processing of the machine learning model Ma using the learning data. This learning processing will be described later.

[0069] S13: The transmission unit 59 of the machine learning server 5 transmits data of the learned machine learning model Mb to the image forming apparatus 3. As a result, the receiving unit 30 of the image forming apparatus 3 receives the data of the learned machine learning model Mb.

[0070] S14: On the other hand, the user terminal 9 transmits a job execution request such as printing to the image forming apparatus 3. As a result, the receiving unit 30 of the image forming apparatus 3 receives the job execution request.

[0071] S15: The image forming apparatus 3 performs job processing according to the job execution request and also performs inference processing. This inference processing will be described later. Note that the image forming apparatus 3 may execute a job such as copying or scanning by direct operation of the user without receiving a job execution request from the user terminal 9.

[0072] <Learning Process> Next, with reference to FIG. 9, the learning process shown in FIG. 8 will be described in detail. FIG. 9 is a flowchart showing the processing in the learning phase.

[0073] S111: The input unit 51 in FIG. 5 inputs, as input data for machine learning, a plurality of signals s1 to sn among the learning data received by the receiving unit 50, as well as the operation state of the image forming apparatus, the ON timing, the notification position, and each data of the circuit to the learning unit 52.

[0074] S112: The learning unit 52 performs machine learning of the machine learning model Ma based on the input data from the input unit 51 and the like by machine learning using a machine learning algorithm such as a neural network, and generates a learned machine learning model Mb.

[0075] S113: The learning unit 52 determines whether or not the machine learning has ended. If not (S113; NO), the process returns to step S111 above and continues. On the other hand, if it ends (S113; YES), the learning process ends.

[0076] <Inference Process> Next, with reference to FIG. 10, the inference process shown in FIG. 8 will be described in detail. FIG. 10 is a flowchart showing the processing in the inference phase.

[0077] S131: The input unit 31 in FIG. 6 acquires each signal s1 to sn output by each of the various sensors 406 in FIG. 3 and inputs it to the noise determination unit 35. Further, the noise determination unit 35 as an inference unit inputs data (operation state of the image forming apparatus 3, ON timing, notification position) from within the engine control device 330 and also inputs circuit data stored in the RAM 402 or the like.

[0078] S132: The noise determination unit 35 uses the signals s1 to sn, the operating state of the image forming apparatus 3, the ON timing, the notification position, and the circuit data as input data, and determines (infers) whether the type of noise included in the signals s1 to sn is normal noise or abnormal noise, and uses the result as output data.

[0079] S133: The abnormal count unit 37 counts the number of times determined to be abnormal noise by the noise determination unit 35 within a predetermined time.

[0080] S134: The abnormal count unit 37 determines whether the number of occurrences of abnormal noise within a predetermined time exceeds a threshold value. If not (S134; NO), the process returns to S131.

[0081] S135: If the number of occurrences of abnormal noise within a predetermined time exceeds the threshold value in process S134 (S134; YES), the display control unit 38 causes the panel display unit 340a to display information indicating that an abnormality has occurred in the image forming apparatus 3. As a result, the user operating the image forming apparatus 3 can grasp that an abnormality has occurred in the image forming apparatus 3.

[0082] S136: The transmission unit 39 transmits information indicating that an abnormality has occurred in the image forming apparatus 3 to the user terminal 9 or the like. As a result, the user of the user terminal 9 or the like can grasp that an abnormality has occurred in the image forming apparatus 3.

[0083] 〔Another example of the machine learning server〕 Subsequently, another example of the machine learning server 5 will be described with reference to FIGS. 11 and 12. In the above embodiment, the image forming apparatus 3 performs the inference process. Here, an example will be described in which the machine learning server 5 performs the inference process as an inference server.

[0084] <Another functional configuration of the machine learning server> FIG. 11 is another functional block diagram of the machine learning server (inference server) in the inference phase.

[0085] As shown in FIG. 11, the machine learning server 5 in the inference phase has a receiving unit 50, an input unit 51, a noise determination unit 55 as an inference unit, and a transmitting unit 59. Since the receiving unit 50, the input unit 51, and the transmitting unit 59 have been described with reference to FIG. 5, the description thereof will be omitted.

[0086] The noise determination unit 55 is a function realized by an instruction from the CPU 501 or the GPU 504 in FIG. 4 based on a program. Further, the noise determination unit 55 has a learned machine learning model Mb learned by the machine learning server 5 itself.

[0087] <Other processes of the entire communication system> Subsequently, with reference to FIG. 12, the overall processing of the communication system 1 when the machine learning server 5 performs inference processing will be described. FIG. 12 is a sequence diagram showing another example 1 of the processing of the communication system.

[0088] S31: The data management server 7 transmits learning data for machine learning to the machine learning server 5. Thereby, the receiving unit 50 of the machine learning server 5 receives the learning data.

[0089] S32: The machine learning server 5 performs learning processing of the machine learning model Ma using the learning data. Since this learning processing is the same as the content described with reference to FIG. 9, the description thereof will be omitted.

[0090] S33: On the other hand, the user terminal 9 transmits a job execution request such as printing to the image forming apparatus 3. Thereby, the receiving unit 30 of the image forming apparatus 3 receives the job execution request.

[0091] S34: The image forming apparatus 3 performs job processing according to the job execution request. Note that the image forming apparatus 3 may execute a job such as copying by direct operation of the user without receiving the job execution request from the user terminal 9.

[0092] S35: The transmission unit 39 of the image forming apparatus 3 transmits an inference request to the machine learning server 5. This inference request includes each signal s1 to sn and each data (the operating state of the image forming apparatus 3, ON timing, paper feed position, and circuit) that should be used by the noise determination unit 35 shown in FIG. 6 to infer the type of noise (normal noise or abnormal noise). As a result, the reception unit 50 of the machine learning server 5 receives the inference request.

[0093] S36: The machine learning server 5 as an inference server performs an inference process. Since this inference process is the same as the content (S131, S132) described with reference to FIG. 10, the description is omitted.

[0094] S37: The transmission unit 59 of the machine learning server 5 transmits data of an inference result corresponding to the inference request in process S35 to the image forming apparatus 3. This inference result includes information indicating whether the type of noise included in the signals s1 to sn is normal noise or abnormal noise. As a result, the reception unit 30 of the image forming apparatus 3 receives the data of the inference result.

[0095] After this, in the image forming apparatus 3, the abnormal count unit 37 performs the same processes as processes S133 and S134 shown in FIG. 10. In this case, if it is NO in process S134, it returns to process S35 shown in FIG. 13. If it is YES in process S134, processes S135 and S136 shown in FIG. 10 are performed.

[0096] Note that the machine learning server 5 shown in FIG. 11 may have the same function as the abnormal count unit 37 of the image forming apparatus 3. In this case, the data of the inference result transmitted in process S37 shown in FIG. 13 includes information indicating that an abnormality has occurred in the image forming apparatus 3.

[0097] 〔Another example of the image forming apparatus〕 Subsequently, another example of the image forming apparatus 3 will be described with reference to FIGS. 13 and 14. In the above embodiment, the machine learning server 5 performed the learning process, but here, an example in which the image forming apparatus 3 performs the learning process will be described.

[0098] <Other functional configurations of the image forming apparatus> FIG. 13 is another functional block diagram of the image forming apparatus in the learning phase.

[0099] As shown in FIG. 13, the image forming apparatus 3 in the learning phase includes a receiving unit 30, an input unit 31, and a learning unit 32. Since the receiving unit 30 and the input unit 31 have been described with reference to FIG. 6, their descriptions are omitted.

[0100] The learning unit 32 is a function realized by an instruction from the CPU 401 or the GPU 404 in FIG. 3 based on a program. Also, the learning unit 32 has a machine learning model Ma in its initial state, but has a learned machine learning model Mb after machine learning. When the learning unit 32 has the learned machine learning model Mb, the learning unit 32 performs additional learning.

[0101] The learning unit 32 also has a comparison and change unit 33 that has the same function as the comparison and change unit 53. The output data (normal noise or abnormal noise) output from the machine learning model Ma or the learned machine learning model Mb is compared with the correct answer data (normal noise or abnormal noise), and the model parameters of the machine learning model Ma or the learned machine learning model Mb are changed according to the error. Thereby, the learning unit 32 can perform machine learning of the machine learning model Ma to generate a learned machine learning model Mb, or perform additional learning (reinforcement learning) on the learned machine learning model Mb.

[0102] <Other processes of the entire communication system> Subsequently, with reference to FIG. 14, the overall process of the communication system 1 when the image forming apparatus 3 performs a learning process will be described. FIG. 14 is a sequence diagram showing another example 2 of the process of the communication system.

[0103] S51: The data management server 7 transmits learning data for machine learning to the image forming apparatus 3. Thereby, the receiving unit 30 of the image forming apparatus 3 receives the learning data.

[0104] S52: The image forming apparatus 3 performs a learning process of the machine learning model Ma using learning data. Since this learning process is the same as the content described with reference to FIG. 9, the description thereof will be omitted.

[0105] S53: On the other hand, in the same manner as in process S14, the user terminal 9 transmits a job execution request such as printing to the image forming apparatus 3. As a result, the reception unit 30 of the image forming apparatus 3 receives the job execution request.

[0106] S54: In the same manner as in process S15, the image forming apparatus 3 performs a job process according to the job execution request and also performs an inference process. Note that the image forming apparatus 3 may execute a job such as copying or scanning by a direct operation of the user without receiving a job execution request from the user terminal 9.

[0107] 〔Main effects of the embodiment〕 As described above, according to the present embodiment, at least by making a determination based on the operating state of the image forming apparatus 3, it is possible to improve the accuracy of determining whether the noise of the signal output from the electrical components of the image forming apparatus 3 is normal noise not caused by an abnormality of the image forming apparatus 3 or abnormal noise caused by an abnormality of the image forming apparatus 3.

[0108] Furthermore, by considering at least one of ON timing data indicating the timing when the electrical component is turned on, circuit data indicating the impedance of the circuit of the electrical component, and paper feed position data indicating the paper feed position in the image forming apparatus, the above accuracy can be further improved.

[0109] 〔Supplementary〕 Although the embodiment has been described above, the present invention is not limited to such an embodiment, and various modifications and substitutions can be made without departing from the gist of the present invention.

[0110] (1) In the communication between the image forming apparatus 3, the machine learning server 5, the data management server 7, and the user terminal 9, other devices (such as servers and routers) may relay data and the like. For example, in this specification, for the sake of simplicity, it is described that the image forming apparatus 3 receives data (information) from the machine learning server 5 or the image forming apparatus 3 transmits data (information) to the machine learning server 5. However, each of these reception and transmission processes includes the case where other devices relay data (information).

[0111] (2) Each function of the above-described embodiments can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0112] (3) Further, as a (non-transitory) recording medium such as a DVD-ROM in which each program of the above-described embodiments is stored, a program product can be provided domestically or abroad.

[0113] (4) The processors (for example, CPU301, CPU400, GPU404, CPU500, GPU504) may be singular or plural respectively.

Explanation of Reference Numerals

[0114] 1 Communication system 2 Image forming system (an example of an electronic system) 3 Image forming apparatus (an example of an electronic device) 5 Machine learning server 7 Data management server 9 User terminal (an example of a communication terminal) 30 Receiver 31 Input section 32 Learning section 33 Comparison and change section 35 Noise determination section (inference section) 37 Abnormal constant counting section 38 Display control section 39 Transmitter 50 Receiver 51 Input section 52 Learning section 53 Comparison and change section 55 Noise determination section (inference section) 57 Abnormal constant counting section 58 Display control section 59 Transmitter 340a Panel display section (an example of a display section) Ma Machine learning model Mb Trained machine learning model

Prior art documents

Patent documents

[0115]

Patent Document 1

Claims

1. An electronic device provided with an electrical component that generates a signal, An electronic device having a noise determination unit that determines the type of noise indicating that the noise included in the signal in the electronic device is normal noise not caused by an abnormality of the electronic device or abnormal noise caused by an abnormality of the electronic device, based on the operating state of the electronic device.

2. The noise determination unit uses a learned machine learning model that takes the signal and data indicating the operating state of the electronic device as input data, infers the type of noise indicating that the noise included in the signal is normal noise not caused by an abnormality of the electronic device or abnormal noise caused by an abnormality of the electronic device, and outputs the inferred result as output data, and infers the type of noise included in the signal based on the signal and the data indicating the operating state of the electronic device, thereby determining the type of the noise. The electronic device according to claim 1.

3. The operating state is that the electronic device is in a warming-up state, a standby state, or an energy-saving state. The electronic device according to claim 2.

4. The warming-up state, the standby state, or the energy-saving state each includes a plurality of different states. The electronic device according to claim 3.

5. The electronic device is an image forming apparatus, and the operating state includes a printing state. The electronic device according to claim 3.

6. The printing state includes a plurality of different states. The electronic device according to claim 5.

7. The input data includes ON timing data indicating the timing when the electrical component is turned on and circuit data indicating the impedance of the circuit of the electrical component. The noise determination unit further infers the type of the noise based on the ON timing data or the circuit data. The electronic device according to claim 2.

8. The electronic device is an image forming apparatus, and the input data includes paper passage position data indicating the paper passage position in the image forming apparatus during printing. The noise determination unit further infers the type of the noise based on the paper passage position data. The electronic device according to claim 7.

9. An electronic device according to any one of claims 1 to 8, An electronic device having an abnormal count unit that counts the number of times determined to be abnormal noise by the noise determination unit, and determines that an abnormality has occurred in the electronic device when the number of abnormal times within a predetermined time exceeds a threshold value.

10. An electronic system constructed by an electronic device provided with an electrical component that generates a signal and an inference server that performs inference related to machine learning, using a trained machine learning model that takes the signal and data indicating the operating state of the electronic device as input data, and infers the type of noise included in the signal, whether it is normal noise not caused by an abnormality of the electronic device or abnormal noise caused by an abnormality of the electronic device, and outputs the inferred type of noise as output data, and determining the type of the noise by inferring the type of noise included in the signal based on the signal and the data indicating the operating state of the electronic device. An electronic system having a noise determination unit.

11. A noise determination method executed by an electronic device provided with an electrical component that generates a signal, wherein the electronic device executes a noise determination process for determining the type of noise indicating that the noise included in the signal in the electronic device is normal noise not caused by an abnormality of the electronic device or abnormal noise caused by an abnormality of the electronic device based on the operating state of the electronic device. Noise determination method.

12. A program for causing a computer to execute the method according to claim 11.

Images