Jam precursor estimation device, jam precursor estimation method, and recording medium

Abstract

A jam precursor estimation device (100) is a jam precursor estimation device that estimates a precursor of a jam in a paper feeding device (200), and includes a sound collecting section (112) that collects rubbing sounds of papers (10) against each other that are generated when the papers (10) are fed into the inside of the paper feeding device (200) from a holding section (270) that holds a plurality of papers (10), an estimation section (114) that estimates a precursor of a jam in the paper feeding device (200) based on an output result obtained by inputting information related to the rubbing sounds into a learned machine learning model, i.e., a learned model, and an output section (116) that outputs a signal to stop feeding the papers (10) into the inside of the paper feeding device (200) to the paper feeding device (200) in a case where the estimation section (114) estimates a precursor of a jam.

Description

Technical Field

[0001] This disclosure relates to a paper jam prediction device, a paper jam prediction method, and a recording medium. Background Technology

[0002] For example, in paper feeding devices that supply paper to image reading devices such as printers or image copying devices (so-called scanners), paper jams sometimes occur due to factors such as re-feeding, oblique feeding, or staples. Depending on the severity of the paper jam, sometimes not only is the operation halted, but the paper is also damaged and becomes unusable. Therefore, technology for detecting paper jams in advance is required.

[0003] For example, Patent Document 1 discloses a technique in which ultrasonic waves transmitted from an ultrasonic transmitting unit provided on one part of a medium support are received by an ultrasonic receiving unit provided on another part of the medium support, and the presence or absence of medium levitation relative to the mounting surface (hereinafter referred to as paper levitation) is determined based on the sound pressure of the received ultrasonic waves.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2020-142868 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, in the technology described in Patent Document 1, an ultrasonic transmitting unit is required to determine whether paper has floated up, making it difficult to easily predict whether there are signs of paper jam (i.e., the occurrence of paper floating up).

[0009] The purpose of this disclosure is to provide a paper jam prediction device, a paper jam prediction method, and a recording medium that can easily predict signs of paper jams.

[0010] Methods used to solve problems

[0011] A paper jam prediction device according to the present disclosure is a device for predicting the signs of a paper jam in a paper feeding device. It comprises: a sound collection unit that collects friction sounds generated when paper is fed from a holding unit holding multiple sheets of paper into the interior of the paper feeding device; a prediction unit that predicts whether there are signs of a paper jam in the paper feeding device based on the output of an input obtained by inputting information related to the friction sounds into a learned machine learning model; and an output unit that, if the prediction unit predicts that there are signs of a paper jam, outputs a signal to the paper feeding device to stop feeding paper into the interior of the paper feeding device.

[0012] Invention Effects

[0013] According to this disclosure, a device for predicting the presence or absence of paper jams, a method for predicting paper jams, and a recording medium can be provided that can easily predict the presence or absence of paper jams. Attached Figure Description

[0014] Figure 1 This is a diagram illustrating an example of a paper feeding device that utilizes the paper jam prediction device of Embodiment 1.

[0015] Figure 2 This is a diagram showing an example of the conveying section of the paper feeding device according to Embodiment 1.

[0016] Figure 3 This is a diagram illustrating an example of the structure of the paper jam prediction device and paper feeding device according to Embodiment 1.

[0017] Figure 4 This is a diagram illustrating an example of the signs of a cardboard in Embodiment 1.

[0018] Figure 5 This is a flowchart illustrating the operation of the paper jam prediction device according to Embodiment 1.

[0019] Figure 6 This is a diagram illustrating an example of the structure of a paper jam prediction device according to a variation of Embodiment 1.

[0020] Figure 7 This is a diagram illustrating an example of the structure of the paper jam prediction device and paper feeding device according to Embodiment 2.

[0021] Figure 8 This is a flowchart illustrating the operation of the paper jam prediction device according to Embodiment 2.

[0022] Figure 9 This is a diagram illustrating an example of the structure of the paper jam prediction device and paper feeding device of a variation of Embodiment 2, Example 1.

[0023] Figure 10 This is a flowchart illustrating the operation of the paper jam prediction device in a variation of Embodiment 2, Example 1.

[0024] Figure 11 This is a diagram used to illustrate the machine learning models used in Examples 1 and 2.

[0025] Figure 12 This is a graph showing the results of Example 1.

[0026] Figure 13 This is a graph showing the results of Example 2.

[0027] Figure 14This is a graph comparing the inference accuracy of Examples 1 and 2 for four types of paper used in Example 2. Detailed Implementation

[0028] (Summary of this disclosure)

[0029] A paper jam prediction device according to the present disclosure is a device for predicting the signs of a paper jam in a paper feeding device. It comprises: a sound collection unit that collects friction sounds generated when paper is fed from a holding unit holding multiple sheets of paper into the interior of the paper feeding device; a prediction unit that predicts whether there are signs of a paper jam in the paper feeding device based on the output of an input obtained by inputting information related to the friction sounds into a learned machine learning model; and an output unit that, if the prediction unit predicts that there are signs of a paper jam, outputs a signal to the paper feeding device to stop feeding paper into the interior of the paper feeding device.

[0030] Therefore, the paper jam prediction device can collect the friction sounds as paper is fed from the holding section into the paper feeding device. Based on the output obtained by inputting information related to the collected friction sounds into a learned model, it can predict whether there are signs of a paper jam, such as paper floating, before a jam occurs. Therefore, unlike conventional technologies, it does not require an ultrasonic transmitter to predict whether paper floating has occurred; a sound collecting section for collecting friction sounds is sufficient. Consequently, the paper jam prediction device can easily predict the presence or absence of paper jam signs with a structure that is simpler than one equipped with an ultrasonic irradiation section.

[0031] Furthermore, the paper jam prediction device can prevent paper jams from occurring and suppress paper breakage by predicting whether there are signs of a paper jam, such as the floating of the supplied paper.

[0032] In a paper jam prediction device according to a technical solution of this disclosure, the information related to the aforementioned friction sound input to the learned model may also be an image of the spectrum of the aforementioned friction sound or an image of its frequency characteristics.

[0033] Therefore, by using a machine learning model, the paper jam prediction device can more easily extract the regularity (so-called feature quantity) of an image. Consequently, the paper jam prediction device can more easily predict whether there is a paper jam.

[0034] In a paper jam prediction device according to one aspect of this disclosure, the aforementioned friction sound can also be an inaudible sound generated by the friction between the paper supplied from the holding section and the paper held in the holding section. For example, the inaudible sound can also be a sound in the ultrasonic frequency range.

[0035] Therefore, the paper jam prediction device predicts the presence or absence of paper floating based on the non-audible sound (e.g., sound in the ultrasonic frequency range) generated during paper feeding from the holding section. This makes it less susceptible to the influence of various audible sounds, i.e., noise, generated around the device, resulting in higher accuracy. Thus, the paper jam prediction device can accurately predict the presence or absence of paper jams.

[0036] In a paper jam prediction device according to a technical solution of this disclosure, the teacher data used in the learning of the aforementioned machine learning model may also include: first data, consisting of information related to the aforementioned friction sound and annotations indicating that a paper jam has occurred; and second data, consisting of information related to the aforementioned friction sound and annotations indicating that no paper jam has occurred.

[0037] As a result, the learning accuracy of the learning unit of the paper jam prediction device is improved, so it can predict the presence or absence of paper jams with high precision.

[0038] In a paper jam prediction device according to one of the technical solutions of this disclosure, the learned model may include multiple learned models corresponding to multiple types of paper; the paper jam prediction device may also include an identification unit that identifies the type of paper supplied from the holding unit to the inside of the paper feeding device; the prediction unit inputs information related to the friction sound into the learned model corresponding to the identified type of paper based on the type of paper identified by the identification unit.

[0039] Therefore, the paper jam prediction device can switch to the learned model to be used based on the type of paper supplied from the holding section to the inside of the paper feeding device. Thus, the paper jam prediction device can accurately predict whether there is a paper jam based on the type of paper.

[0040] In a paper jam prediction device according to a technical solution of this disclosure, the identification unit can also identify the type of paper based on data obtained from at least one of an image sensor, an ultrasonic sensor, an optical sensor, a weight sensor, and machine learning.

[0041] Therefore, the paper jam prediction device can identify the type of paper using at least one of the following: a database that establishes a correspondence between data representing paper characteristics and paper types, and a learned model that takes data representing paper characteristics as input and outputs the type of the supplied paper 10. Thus, the paper jam prediction device can identify the type of paper with high accuracy.

[0042] In the paper jam prediction device of one of the technical solutions of this disclosure, the aforementioned machine learning model can also be a convolutional neural network model.

[0043] Therefore, by using a convolutional neural network model, the paper jam prediction device can more easily extract the regularity (so-called feature quantity) of an image.

[0044] Furthermore, the paper jam prediction method disclosed in this paper invention is a method for predicting the signs of a paper jam in a paper feeding device. It includes: a sound collection step, which collects friction sounds generated when paper is fed from a holding section holding multiple sheets of paper into the interior of the paper feeding device; a prediction step, which predicts whether there are signs of a paper jam in the paper feeding device based on the output results obtained by inputting information related to the friction sounds into a learned machine learning model, i.e., a learned model; and an output step, which outputs a signal to the paper feeding device to stop feeding paper into the interior of the paper feeding device if the prediction indicates that there are signs of a paper jam.

[0045] Therefore, the paper jam prediction method can predict whether there are signs of a paper jam, such as paper floating, based on the output obtained by inputting information related to the friction sound when paper is supplied from the holding section into a learned model. Thus, unlike conventional technologies, it does not require an ultrasonic transmitter to predict whether paper floating has occurred; simply collecting the friction sound is sufficient. Consequently, the paper jam prediction device can easily predict the presence or absence of paper jam signs with a structure that is simpler than one equipped with an ultrasonic irradiation unit.

[0046] Furthermore, the paper jam prediction method can prevent paper jams from occurring and suppress paper breakage by predicting whether there are signs of a paper jam, such as the floating of the supplied paper.

[0047] Furthermore, the recording medium of one of the technical solutions of this disclosure is a computer-readable, non-transitory recording medium that records a program for causing a computer to execute the above-described paper jam prediction method.

[0048] Therefore, computers can achieve the same effect as the aforementioned method for predicting paper jams.

[0049] In addition, these inclusive or specific technical solutions can also be implemented by systems, methods, devices, integrated circuits, computer programs or computer-readable CD-ROM (Compact Disc Read Only memory) recording media, or by any combination of systems, methods, devices, integrated circuits, computer programs and recording media.

[0050] Hereinafter, embodiments of the present disclosure will be specifically described with reference to the accompanying drawings. The numerical values, shapes, materials, constituent elements, arrangements and connection forms of constituent elements, steps, and order of steps shown in the following embodiments are examples and are not intended to limit the scope of the claims. Furthermore, constituent elements in the following embodiments that are not described in the independent claims representing the highest-level concept are described as arbitrary constituent elements. In addition, the figures are not necessarily strictly illustrated. In the figures, substantially identical components are given the same reference numerals, and sometimes repeated descriptions are omitted or simplified.

[0051] Furthermore, in this disclosure, terms indicating the relationship between elements such as parallel and perpendicular, terms indicating the shape of elements such as rectangles, and numerical values ​​do not merely indicate a strict meaning, but rather refer to substantially equivalent ranges, for example, including differences of a few percentage points.

[0052] (Implementation Method 1)

[0053] Hereinafter, Embodiment 1 will be described in detail with reference to the accompanying drawings.

[0054] Paper feeding device

[0055] First, refer to Figure 1 , Figure 2 and Figure 3 The paper feeding device is described. Figure 1 This is a diagram illustrating an example of a paper feeding device 200 that incorporates the paper jam prediction device 100 of Embodiment 1. Figure 2 This is a diagram showing an example of the conveying section 210 of the paper feeding device 200 according to Embodiment 1. Figure 3 This is a diagram illustrating an example of the structure of the paper jam prediction device 100 and the paper feeding device 200 according to Embodiment 1.

[0056] The paper feeding device 200 supplies paper, for example, to a paper processing device (not shown). The processing device may be a processing device that processes the supplied paper itself or applies processing to the paper, a copying device that copies information such as characters, marks, and pictures printed on the supplied paper to other recording media, or an output device that reads the information and outputs it as an analog image signal.

[0057] like Figure 1 As shown, the paper feeding device 200, for example, includes a device for holding multiple sheets of paper 10 (see reference). Figure 2The paper feed section 270 supplies paper 10 through a feed port 260, a feed roller 212 that supplies paper 10 from the feed port 260, a separation roller 214 that separates the paper 10 supplied from the feed port 260 one sheet at a time, and a retard roller 216 that rotates in the opposite direction to the rotation direction of the separation roller 214. Figure 1 middle, Figure 2 The multiple sheets of paper 10 shown are illustrated as a paper bundle 20, with diagonal lines added for easy observation. Additionally, the dotted circle represents the portion of the paper 10 that floats up when supplied from the holding section 270. This portion will be referred to hereafter as the paper-floating occurrence portion 30. The floating of the paper during supply will be described later.

[0058] Next, refer to Figure 2 The conveyor unit 210 will be described below. Figure 2 In this illustration, from an easy-to-observe point of view, the supply port 260 and the holding part 270 are omitted, but multiple sheets of paper 10 are held as a paper bundle 20 by the holding part 270, and the paper 10 is supplied from the supply port 260.

[0059] like Figure 2 As shown, the feed roller 212, the separating roller 214, and the deceleration roller 216 are components of the conveying unit 210. The conveying unit 210 separates and conveys sheets of paper 10 supplied from the holding unit 270 one by one. Hereinafter, the feed rollers 212a and 212b will be collectively referred to as feed roller 212, the separating rollers 214a and 214b will be collectively referred to as separating roller 214, and the deceleration rollers 216a and 216b will be collectively referred to as deceleration roller 216.

[0060] The feed rollers 212a and 212b are configured to move freely up and down, abutting against the uppermost sheet 10 of the multiple sheets 10 held by the holding portion 270, and pick up the uppermost sheet 10 from the multiple sheets 10 and supply it from the feed port 260. The feed roller 212 is configured to be easily repositioned according to changes in the thickness of the paper bundle 20 within the holding portion 270 that occur with the supply of paper 10. Alternatively, the feed roller 212 may be configured to abut against the lowermost sheet 10 of the multiple sheets 10 in the paper bundle 20. In this case, the feed port 260 is located below the paper bundle 20.

[0061] Separating rollers 214a and 214b separate the sheets of paper 10 supplied by the paper feed roller 212 one by one. Here, separating rollers 214a and 214b, together with deceleration rollers 216a and 216b arranged opposite to separating rollers 214a and 214b, function as a separating section for separating the sheets of paper 10 one by one. Deceleration roller 216 returns the paper 10 supplied that is overlapping the paper 10 that abuts against the separating roller 214 to the holding section 270 side.

[0062] Next, the specific operation of the conveying unit 210 will be explained. First, the feed roller 212, by rotating in the direction of arrow A, picks up the uppermost sheet of paper 10 held by the holding unit 270 and feeds it from the feed port 260 in the direction of arrow D. Next, the separating roller 214, by rotating in the direction of arrow B, feeds the paper 10 that abuts against the separating roller 214 in the direction of arrow D. At this time, the deceleration roller 216, by rotating in the direction of arrow C, returns the paper 10 that abuts against the deceleration roller 216 in the direction opposite to arrow D. Since the torque of the deceleration roller 216 is limited, when only one sheet of paper 10 is supplied, the paper 10 is fed in the direction of arrow D by the movement of the separating roller 214. Furthermore, for example, when two sheets of paper 10 are overlapped and supplied by the feed roller 212, the paper 10 that abuts against the separating roller 214 is fed in the direction of arrow D, and the paper 10 that abuts against the deceleration roller 216 is returned in the direction opposite to arrow D.

[0063] Through the above actions, the conveying unit 210 can separate the sheets of paper 10 supplied from the supply port 260 by the paper feed roller 212 and supply them to the processing device one by one. As a result, the conveying unit 210 can reduce the refeeding of the paper 10 supplied from the holding unit 270, thus reducing paper jams in the paper feeding device 200.

[0064] Next, refer to Figure 3 The functional structure of the paper feeding device 200 will be explained here. (Referring to reference...) Figure 1 and Figure 2 The structure of the description can omit or simplify the description.

[0065] like Figure 3 As shown, the paper feeding device 200 includes, for example, a conveying unit 210, a driving unit 220, a control unit 230 for controlling the operation of the driving unit 220, a storage unit 240, and a communication unit 250.

[0066] The drive unit 220 drives the paper feed roller 212, the separation roller 214, and the reduction roller 216 of the conveying unit 210 respectively. For example, the drive unit 220 includes one or more motors, which rotate the paper feed roller 212, the separation roller 214, and the reduction roller 216 according to the control signal from the control unit 230.

[0067] As described above, the control unit 230 performs information processing to control the operation of the transport unit 210. The control unit 230 may be implemented by a microcomputer, a processor, or a dedicated circuit.

[0068] The storage unit 240 is a storage device for control programs and the like executed by the storage control unit 230. The storage unit 240 is implemented, for example, by a semiconductor memory.

[0069] The communication unit 250 is a communication module (communication line) for the paper feeding device 200 to communicate with the paper jam prediction device 100 and the processing device (not shown) via a local communication network. Communication via the communication unit 250 can be wireless or wired communication, for example. There are no particular limitations on the communication standard used in the communication.

[0070] [Paper Jam Prediction Device]

[0071] [1. Overview, etc.]

[0072] Next, refer to Figure 1 and Figure 4 An overview of the paper jam prediction device 100 of Embodiment 1 will be described. Figure 4 This is a diagram illustrating an example of the signs of a cardboard cutout in Embodiment 1. Additionally, in Figure 4 Nakaya and Figure 2 Similarly, the illustration of the retaining part 270 is omitted from the viewpoint of easy observation.

[0073] The paper jam prediction device 100 is a device that predicts whether there are signs of a paper jam in the paper feed device 200. Specifically, the paper jam prediction device 100 collects the friction sounds generated when paper 10 is fed from the holding section 270 holding multiple sheets of paper 10 into the paper feed device 200, and predicts whether there are signs of a paper jam in the paper feed device 200 based on the output obtained by inputting information related to the collected friction sounds into a learned model. Furthermore, if the paper jam prediction device 100 predicts that there are signs of a paper jam, it outputs a signal to the paper feed device 200 to stop feeding paper 10 from the holding section 270 into the paper feed device 200.

[0074] Furthermore, the learned model is a machine learning model that has already been learned. The learned model is obtained through learning performed by the learning unit 140. The learned model is constructed by learning the relationship between the friction sound generated when paper 10 is fed from the holding unit 270 into the interior of the paper feeding device 200 and the presence or absence of paper jam warning signs. The information related to the friction sound input into the learned model is, for example, an image of the friction sound spectrogram or an image of its frequency characteristics.

[0075] Friction noise, for example, is the friction noise generated when paper 10 is supplied from the holding section 270, caused by the friction between the supplied paper 10 and the paper 10 held in the holding section 270. The paper 10 held in the holding section 270 also includes a portion of the paper 10 held in the holding section 270. Furthermore, friction noise can be generated, for example, when the paper 10 supplied from the holding section 270 is not supplied straight but at an angle relative to the supply port 260, caused by the friction between the supplied paper 10 and the paper 10 held in the holding section 270, or by the friction between the supplied paper 10 and the inner wall of the holding section 270 or the components surrounding the supply port 260. Additionally, friction noise can also occur when a portion of the supplied paper 10 is bent, wrinkled, or in a state different from normal when a note or sealing strip is attached to the paper 10. Friction noise can include audible sounds that can be heard by the human ear and inaudible sounds that cannot be heard by the human ear, but it can also be inaudible sounds. Inaudible sounds are, for example, sounds in the ultrasonic frequency range. When the friction sound is a sound with a frequency in the ultrasonic range, the frequency band of the friction sound can be above 60kHz and below 95kHz, or above 75kHz and below 95kHz, or above 80kHz and below 95kHz, or especially above 85kHz and below 90kHz.

[0076] The paper 10 supplied from the holding section 270 can be a single sheet or multiple sheets. The paper 10 is separated one by one by the separating roller 214 and the deceleration roller 216 (described later) and supplied to the processing device. Furthermore, the paper 10 held by the holding section 270 and rubbing against the supplied paper 10 can be either the uppermost sheet of multiple sheets held by the holding section 270, or multiple sheets including the uppermost sheet. The paper jam prediction device 100 collects the friction noise generated when paper 10 is supplied from the holding section 270 to the interior of the paper feeding device 200, and based on the output obtained by inputting information related to the collected friction noise into a learned model, predicts whether there is a sign of a paper jam in the paper feeding device 200.

[0077] The signs of a paper jam in the paper feeding device 200 are precursors to a paper jam; they are phenomena that occur before a paper jam is about to occur due to the cause of the jam. For example, such as... Figure 4 As shown, the cause of paper jams is explained as follows: the supplied paper 10 is stapled with staples 15 (hereinafter also referred to as being stapled). For example, when multiple sheets of paper 10 stapled with staples 15 are fed by the feed roller 212 in the direction of arrow D, only the sheet of paper 10 that is in contact with the separation roller 214 among the multiple sheets of paper 10 stapled with staples 15 is fed by the separation roller 214 in the direction of arrow D. At this time, the paper 10 floats up around the part that is stapled with staples 15. This phenomenon occurs because... Figure 1 and Figure 4The paper floats at point 30. Furthermore, if the paper 10 is further fed by the separating roller 214 in the direction of arrow D, the paper 10 rotates and tilts around the point where it is stapled with the staple 15. Furthermore, if the paper 10 continues to be fed further by the separating roller 214 in the direction of arrow D, a paper jam occurs. Thus, the occurrence of a portion of the paper 10 supplied from the holding section 270 floating is a precursor to a paper jam. The paper jam prediction device 100 predicts the presence or absence of a paper jam precursor based on the output obtained by inputting information related to the friction noise generated by the friction between the paper 10 supplied from the holding section 270 and the multiple sheets of paper 10 held by the holding section 270 into a learned model.

[0078] Here, the example of a paper jam caused by a staple is given, but the causes of paper jams are not limited to this. Other causes of paper jams include, for example, a portion of the supplied paper 10 being bent, a label being attached to the supplied paper 10, a portion of the supplied paper 10 being glued to other papers 10, or the supplied paper 10 having a different paper quality than other papers 10, such as due to the roughness of its surface.

[0079] Furthermore, when paper 10 is supplied from the holding section 270, a portion of the supplied paper 10 floats up near the separating roller 214, particularly between the separating roller 214 and the feed roller 212. For example, when two or more sheets of paper 10 are re-fed by the feed roller 212, the portion of the supplied paper 10 that floats up may be the front side of the part where the separating roller 214 abuts against the uppermost sheet of paper 10 among the two or more sheets of paper 10. Here, the front side refers to the direction opposite to the feeding direction of the paper 10 (the direction of arrow D in the figure). That is, the front side is the holding section 270 side when viewed from the holding section 270 towards the feed port 260.

[0080] As described above, when the separating roller 214 separates only the paper 10 that comes into contact with the separating roller 214 from the two or more sheets of paper 10 re-fed by the feed roller 212 and feeds it toward the processing device, the paper jam prediction device 100 can predict whether a portion of the supplied paper 10 has floated up (paper float) based on the output obtained by inputting information related to the friction noise between the paper 10s into a learned model. Therefore, the paper jam prediction device 100 can, for example, stop the supply of paper 10 before the paper 10 that has floated up rotates and is supplied at an angle relative to the supply direction. Thus, the paper jam prediction device 100 can not only reduce the occurrence of paper jams, but also suppress damage such as bending, wrinkling, or tearing of the supplied paper 10.

[0081] Furthermore, the friction noise between the papers 10 is generated, for example, by the friction between the supplied paper 10 and the paper 10 held in the holding portion 270. The paper 10 held in the holding portion 270 also includes a portion of the paper 10 held in the holding portion 270. Therefore, for example, when two or more sheets of paper 10 are heavily fed towards the separation roller 214 by the feed roller 212 and paper floats up near the separation roller 214, the friction noise between the papers 10 is generated by the friction between the paper 10 that abuts against the separation roller 214 and the other paper 10 that does not abut against the separation roller 214.

[0082] Furthermore, the paper jam prediction device 100 does not need to irradiate multiple sheets of paper 10 held in the holding section 270 with ultrasonic waves to predict the reflected waves of the irradiated ultrasonic waves in order to predict whether there is a paper jam (i.e., a sign of a paper jam), but instead collects the friction sound between the sheets of paper 10 as a sound in the ultrasonic field. That is, the paper jam prediction device 100 does not need to have an active ultrasonic sensor, but only a passive ultrasonic sensor, so it can predict whether there is a paper jam with a simpler structure.

[0083] [2. Structure]

[0084] Next, refer to Figure 3 The structure of the paper jam prediction device 100 will be explained.

[0085] The paper jam prediction device 100 includes an information processing unit 110, a storage unit 120, a communication unit 130, and a learning unit 140. The structure will be described below.

[0086] Information Processing Department

[0087] The information processing unit 110 performs information processing related to the prediction of paper jams. The information processing unit 110 is implemented, for example, by a microcomputer or processor. Specifically, the information processing unit 110 includes a sound collection unit 112, a prediction unit 114, and an output unit 116.

[0088] [Sound Collection Department]

[0089] The sound collecting unit 112 collects the friction sounds generated when the paper 10 is supplied from the holding part 270, which holds multiple sheets of paper 10. More specifically, the sound collecting unit 112 collects the friction sounds generated by the friction between the paper 10 supplied from the holding part 270 and the paper 10 held in the holding part 270. The sound collecting unit 112 is, for example, a microphone. In this case, the sound collecting unit 112 converts the collected friction sounds into electrical signals and outputs the electrical signals to the estimation unit 114.

[0090] Furthermore, when the sound collecting unit 112 is a microphone, it is positioned to collect the friction sounds of the paper 10 against each other. For example, the sound collecting unit 112 may be positioned closer to the holding portion 270 than the separating roller 214, i.e., closer to the separating roller 214 when viewed from the holding portion 270. More specifically, the sound collecting unit 112 may be positioned above the holding portion 270. The sound collecting unit 112 may also be positioned near the feed port 260. Near the feed port 260, for example, refers to the area from the midpoint between the feed port 260 and the separating roller 214 to the upper part of the feed roller 212. In particular, the sound collecting unit 112 may be positioned above the feed port 260 and arranged side-by-side with the feed roller 212 in a direction intersecting the direction of feeding the paper 10 from the holding portion 270. More specifically, the sound collecting unit 112 may also be positioned above the supply port 260 and at the same height as the paper supply roller 212 in a direction intersecting the direction of feeding the paper 10 from the holding unit 270. The sound collecting unit 112 can be positioned at a height that does not contact the supplied paper 10; for example, it may be positioned at the same height as the rotation axis of the paper supply roller 212 and arranged side by side with the paper supply roller 212.

[0091] Furthermore, the sound collecting unit 112 can be located at a position that can collect the friction sound of the paper 10 against each other, and is not limited to the upper part of the supply port 260. For example, the sound collecting unit 112 can also be located at the lower part of the supply port 260, or it can be located on the side of the supply port 260.

[0092] In addition, Figure 3 The diagram shows an example of a paper jam prediction device 100 having one sound collection unit 112, but it may also have two or more sound collection units 112. For example, multiple (i.e., two or more) sound collection units 112 may be provided with the paper feed roller 212 clamped in the center in a direction that intersects with the direction of feeding paper 10 from the holding part 270.

[0093] [Speculation Section]

[0094] The prediction unit 114 uses a learned machine learning model (so-called learned model) stored in the storage unit 120 to predict whether there is a sign of a paper jam, based on the output obtained by inputting information related to the friction sound collected by the sound collection unit 112 into the learned model. The specific operation of the prediction unit 114 will be described later.

[0095] Information related to the fricative sound input into the learned model may be, for example, an image of the fricative sound's spectrogram or frequency characteristics. This information may be image data in formats such as JPEG (Joint Photographic Experts Group) or BMP (Basic Multilingual Plane), but it may not be image data. In this case, the information may also be numerical data in formats such as WAV (Waveform Audio File Format) (more specifically, time-series numerical data). This information may also include, for example, at least one of the following: the frequency band of the fricative sound, the duration of the fricative sound, the sound pressure level, and the waveform.

[0096] Furthermore, the output results may include, for example, signs of paper jams, a reduction in friction, or the absolute value or relative value of the friction sound to a specified value. The reduction in friction between the paper 10 can also be information indicating whether the friction sound (more specifically, the sound pressure of the friction sound) has decreased compared to a preset specified value (e.g., if it is the absolute value of the difference in sound pressure, it is whether it has increased compared to the specified value).

[0097] [Output Department]

[0098] When the prediction unit 114 predicts that there is a sign of a paper jam, the output unit 116 outputs a signal to the paper feeding device 200 to stop the supply of paper 10 from the holding unit 270 to the inside of the paper feeding device 200.

[0099] [Storage Department]

[0100] Storage unit 120 is a storage device for computer programs and the like executed by storage information processing unit 110. Storage unit 120 can also temporarily store teacher data and data related to friction sounds collected by sound collection unit 112. Storage unit 120 updates the stored learned model with a machine learning model (so-called learned model) generated by learning unit 140. Storage unit 120 is implemented using semiconductor memory or HDD (Hard Disk Drive) or the like.

[0101] [Ministry of Communications]

[0102] Communication unit 130 is the communication path used by paper jam prediction device 100 to communicate with paper feeding device 200. Communication between communication unit 130 and paper feeding device 200 can be direct or via a relay device such as a wireless router (not shown). Communication unit 130 can be, for example, a wireless communication circuit for wireless communication or a wired communication circuit for wired communication. There are no particular limitations on the communication standard used by communication unit 130.

[0103] [Study Department]

[0104] Learning Department 140 uses teacher data for machine learning. For example, Learning Department 140 uses machine learning to create a machine learning model that takes information related to friction sounds as input and outputs a symptom of paper jamming. The output can be either a symptom of paper jamming or a reduction in friction between the paper 10s. The learned model is constructed by learning the relationship between the friction sounds between the paper 10s and the symptom of paper jamming. The symptom of paper jamming has been described above, so it is omitted here.

[0105] The teacher data used in the learning of the machine learning model includes, for example, a first dataset consisting of information related to friction sounds and annotations indicating the occurrence of paper jams (in other words, signs of paper jams), and a second dataset consisting of information related to friction sounds and annotations indicating the absence of paper jams (in other words, signs of paper jams). More specifically, the teacher data includes, for example, a first dataset labeled with signs of paper jams on images of the spectrograms or frequency characteristics of friction sounds, and a second dataset labeled with signs of paper jams on images of the spectrograms or frequency characteristics of friction sounds. More specifically, the teacher data is a dataset containing multiple groups of information related to friction sounds collected in the past and groups of information indicating whether or not paper jams have occurred.

[0106] The machine learning model is, for example, a neural network model, more specifically, a convolutional neural network (CNN) model. The machine learning model does not have to be a CNN; there is no particular limitation. However, for example, if the information related to fricatives is time-series numerical data (e.g., a spectrogram or time-series numerical data of the frequency characteristics of fricatives), it can also be a recurrent neural network (RNN) model. That is, the machine learning model can also be appropriately selected based on the form of the input data. The learned machine learning model (so-called learned model) generated by the learning unit 140 contains learned parameters that have been adjusted through machine learning. The learning unit 140 stores the generated learned model in the storage unit 120. The learning unit 140 is implemented, for example, by the processor executing the program stored in the storage unit 120.

[0107] [3. Action]

[0108] Next, the operation of the paper jam prediction device 100 will be explained. Figure 5 This is a flowchart illustrating the operation of the paper jam prediction device 100 according to Embodiment 1.

[0109] like Figure 5As shown, the sound collection unit 112 collects the friction sound between the paper 10 generated when the paper 10 is supplied from the holding unit 270 to the inside of the paper feeding device 200 (S101). Here, the sound collection unit 112 is, for example, a microphone, which converts the collected friction sound into an electrical signal and outputs the converted electrical signal to the estimation unit 114. The microphone includes a microphone device. For example, the sound collection unit 112 can be a microphone capable of collecting inaudible sounds, or a microphone capable of collecting both audible and inaudible sounds and extracting sounds in a specific frequency band. Furthermore, the sound collection unit 112 can also be a directional microphone. The sound collection unit 112 can also be, for example, a MEMS microphone. Inaudible sounds are, for example, sounds in the ultrasonic frequency range. For example, when the estimation unit 114 is input with information related to the inaudible sounds in the collected friction sound, the sound collection unit 112 can also extract the inaudible sounds (e.g., sounds in the ultrasonic frequency range) in the collected friction sound and convert them into electrical signals, and output the converted electrical signals to the estimation unit 114.

[0110] Next, the estimation unit 114 inputs information related to the friction sound collected by the sound collection unit 112 into the learned model and obtains an output result (S102). More specifically, in step S102, firstly, the estimation unit 114 acquires the electrical signal output from the sound collection unit 112 and converts the acquired electrical signal into a digital signal using PCM (Pulse Code Modulation) or the like. At this time, for example, the estimation unit 114 may also acquire the electrical signal of the friction sound collected by the sound collection unit 112, which includes audible and inaudible sounds, convert the electrical signal into a digital signal, and then extract the digital signal of the inaudible sound. Next, the estimation unit 114 generates an image of the spectrum of the friction sound or an image of its frequency characteristics based on the digital signal. In addition, the image of the spectrum of the friction sound or the image of its frequency characteristics is information related to the friction sound that is input into the learned model, but the digital signal (i.e., the time series numerical data of the spectrum of the friction sound or the frequency characteristics) can also be used as information related to the friction sound. Next, the prediction unit 114 inputs the generated information related to friction sound into the learned model and obtains the output result. As described above, the output result can be a sign of paper jamming, a reduction in friction between the paper 10s, or the absolute value of the friction sound or a relative value to a specified value.

[0111] Next, the estimation unit 114 estimates whether there is a sign of a paper jam based on the output result obtained in step S102 (S103). In step S103, if the estimation unit 114 estimates that there is a sign of a paper jam ("Yes" in S104), the output unit 116 outputs a signal to the paper feeding device 200 to stop feeding paper 10 from the holding unit 270 into the paper feeding device 200 (S105). More specifically, in step S104, if the learned model outputs "signs of a paper jam", the estimation unit 114 estimates that there is a sign of a paper jam based on this output result. Furthermore, in step S104, if the learned model outputs "reduction in friction between the paper 10s", the estimation unit 114 can also estimate that there is a sign of a paper jam based on this output result.

[0112] On the other hand, if in step S103 the estimation unit 114 estimates a sign of no paper jam (in S104, this is "No"), the paper jam prediction device 100 returns to the processing in step S101. More specifically, in step S104, if the learned model outputs "signs of no paper jam," the estimation unit 114 estimates a sign of no paper jam based on this output. Furthermore, in step S104, if the learned model outputs "reduction in friction between the paper 10s," the estimation unit 114 estimates a sign of no paper jam based on this output.

[0113] The paper jam prediction device 100 repeatedly executes the above-described processing procedure whenever paper 10 is supplied from the holding section 270.

[0114] [4. Effects, etc.]

[0115] As explained above, the paper jam prediction device 100 of Embodiment 1 is a paper jam prediction device that predicts the signs of a paper jam in the paper feeding device 200. It includes: a sound collection unit 112 that collects friction sounds generated when paper 10 is fed from the holding unit 270 holding multiple sheets of paper 10 into the paper feeding device 200; a prediction unit 114 that predicts whether there are signs of a paper jam in the paper feeding device 200 based on the output result obtained by inputting information related to the friction sounds into a learned machine learning model, i.e., a learned model; and an output unit 116 that outputs a signal to the paper feeding device 200 to stop feeding paper 10 into the paper feeding device 200 when the prediction unit 114 predicts that there are signs of a paper jam.

[0116] Therefore, the paper jam prediction device 100 can collect the friction sounds when paper 10 is supplied from the holding section 270 to the inside of the paper feeding device 200, and based on the output results obtained by inputting information related to the collected friction sounds into a learned model, it can predict whether there are signs of a paper jam, such as the paper 10 floating up. Therefore, it is not necessary to have an ultrasonic transmitting section, as in conventional technology, to predict whether the paper 10 is floating up; only a sound collecting section 112 for collecting friction sounds is needed. Therefore, the paper jam prediction device 100 can easily predict the presence or absence of paper jam signs with a structure that is simpler than a structure with an ultrasonic irradiation section.

[0117] Furthermore, since the paper jam prediction device 100 can predict whether there are signs of a paper jam, such as the floating of the supplied paper 10, it can not only prevent paper jams from occurring, but also suppress damage to the paper 10.

[0118] In the paper jam prediction device 100 of Embodiment 1, the information related to friction sound input to the learned model may be an image of the spectrum of friction sound or an image of its frequency characteristics.

[0119] Therefore, by using a machine learning model, the paper jam prediction device 100 can more easily extract the regularity (so-called feature quantity) of an image. Consequently, the paper jam prediction device 100 can more easily predict whether there is a paper jam.

[0120] In the paper jam prediction device 100 of Embodiment 1, the friction sound may also be an inaudible sound generated by the friction between the paper 10 supplied from the holding section 270 and the paper 10 held in the holding section 270. In this case, the inaudible sound may also be a sound in the frequency range of the ultrasonic wave.

[0121] Therefore, the paper jam prediction device 100 predicts the presence or absence of paper 10 based on the non-audible sound (e.g., sound in the ultrasonic frequency range) generated during the friction noise produced when paper 10 is supplied from the holding section 270. This makes it less susceptible to the influence of various audible sounds, i.e., noise, generated around the device, resulting in higher sound collection accuracy. Thus, the paper jam prediction device 100 can predict the presence or absence of paper jams with high accuracy.

[0122] In the paper jam prediction device 100 of Embodiment 1, the teacher data used in the learning of the machine learning model may also include first data consisting of information related to friction sounds and annotations indicating that a paper jam has occurred, and second data consisting of information related to friction sounds and annotations indicating that no paper jam has occurred.

[0123] As a result, the learning accuracy of the learning unit 140 of the paper jam prediction device 100 is improved, so it can predict the presence or absence of paper jams with high accuracy.

[0124] In the paper jam prediction device 100 of embodiment 1, the machine learning model may also be a convolutional neural network model.

[0125] Therefore, by using a convolutional neural network model, the paper jam prediction device is able to more easily extract the regularity (so-called feature quantity) of an image.

[0126] (Modification 1 of Implementation Method 1)

[0127] Next, refer to Figure 6 The paper jam prediction device 100a of the modified example 1 of the embodiment 1 will be described. Figure 6 This is a diagram illustrating an example of the structure of the card prediction device 100a, which is a variation of Embodiment 1. In Embodiment 1, the sound collection unit 112 is described as an example of a microphone. However, in Variation 1 of Embodiment 1, the sound collection unit 112a differs from Embodiment 1 in that it acquires an electrical signal containing the friction sound output from the microphone 300. Hereinafter, the explanation will focus on the differences from Embodiment 1, and repeated content will be simplified or omitted.

[0128] [1. Structure]

[0129] like Figure 6 As shown, in Variation 1 of Embodiment 1, the paper jam prediction device 100a is connected to the microphone 300 via the communication unit 130. The paper jam prediction device 100a includes an information processing unit 110a, a storage unit 120, a communication unit 130, and a learning unit 140. The information processing unit 110a includes a sound collection unit 112a, a prediction unit 114, and an output unit 116. Hereinafter, the sound collection unit 112a will be described.

[0130] The sound collection unit 112a acquires, for example, the friction sound collected by at least one microphone 300 as an electrical signal, and outputs the acquired electrical signal to the estimation unit 114. At this time, the sound collection unit 112a may also acquire, for example, the electrical signal output from at least one microphone 300 and information indicating the microphone 300 that output the electrical signal, and output the acquired information and the electrical signal to the estimation unit 114. Furthermore, for example, if the friction sound is an inaudible sound (e.g., a sound with a frequency in the ultrasonic range), an electrical signal representing the sound pressure level of the ultrasonic range frequency may be extracted from the acquired electrical signal and output to the estimation unit 114.

[0131] [2. Action]

[0132] In a variation of Embodiment 1, the sound collecting unit 112a acquires an electrical signal corresponding to the friction sound collected by the microphone 300. Therefore, the signal referred to in Embodiment 1... Figure 5 The processing of step S101 is different.

[0133] For example, in Figure 5 In step S101, the sound collection unit 112a acquires an electrical signal corresponding to the friction sound collected by the microphone 300. Then, the sound collection unit 112a outputs the acquired electrical signal to the estimation unit 114. In this case, the sound collection unit 112a functions as a so-called acquisition unit.

[0134] Furthermore, for example, in the case where friction sounds are collected by multiple microphones 300, Figure 5 In step S101, the sound collection unit 112a acquires an electrical signal corresponding to the friction sound collected by the plurality of microphones 300. At this time, the sound collection unit 112a can also acquire electrical signals output from the plurality of microphones 300 and information indicating which microphone 300 output the electrical signal. Furthermore, the sound collection unit 112a outputs the acquired electrical signals and information to the estimation unit 114.

[0135] As described above, in Variation 1 of Embodiment 1, the paper jam omen prediction device 100a differs from Embodiment 1 in that it acquires an electrical signal containing the friction sound collected by the microphone 300 to perform information processing on omen prediction.

[0136] [3. Effects, etc.]

[0137] In the modified example 1 of embodiment 1, the paper jam prediction device 100a is configured separately from the microphone 300, so the installation position and number of microphones 300 can be appropriately changed according to the design, and the paper jam prediction device 100a can be installed in one integrated circuit.

[0138] (Implementation Method 2)

[0139] Next, the paper jam prediction device of Embodiment 2 will be described. Figure 7 This diagram illustrates an example of the structure of the paper jam prediction device 100b and the paper feeding device 200 according to Embodiment 2. In addition to the structure of Embodiment 1, the paper jam prediction device 100b of Embodiment 2 includes a recognition unit 113a that identifies the type of paper 10 supplied from the holding unit 270 to the interior of the paper feeding device 200, and the learned models include multiple learned models corresponding to multiple types of paper. These aspects differ from Embodiment 1 and its variant 1. Hereinafter, the description will focus on the differences from Embodiment 1 and its variant 1, omitting or simplifying repetitive descriptions.

[0140] [1. Structure]

[0141] The paper jam prediction device 100b includes an information processing unit 110b, a storage unit 120, a communication unit 130, and a learning unit 140a. The information processing unit 110b includes a sound collection unit 112, a recognition unit 113a, a prediction unit 114a, and an output unit 116. The recognition unit 113a, the prediction unit 114a, and the learning unit 140a will be described below.

[0142] [Identification Department]

[0143] The identification unit 113a identifies the type of paper 10 supplied from the holding unit 270 to the interior of the paper feeding device 200. More specifically, the identification unit 113a identifies the type of paper 10 based on the friction sound generated between the paper 10 when it is supplied from the holding unit 270 to the interior of the paper feeding device 200. For example, the identification unit 113a is triggered by the collection of friction sound by the sound collecting unit 112 and obtains the friction sound from the sound collecting unit 112. Here, the type of paper 10 may include not only the category of paper 10 (e.g., photocopying paper, typewriter paper, tracing paper, thick paper, etc.) but also the size of paper 10 (e.g., A4 size, B5 size, A3 size, etc.).

[0144] For example, the identification unit 113a identifies the type of paper 10 based on the output obtained by inputting the acquired friction sound (more specifically, information related to the friction sound) into a learned model (hereinafter also referred to as the second learned model) representing the relationship between the friction sound and the type of paper 10. Furthermore, the friction sound used by the identification unit 113a may be a friction sound from a different time than the friction sound used by the estimation unit 114a. More specifically, the friction sound used in the identification of paper 10 is generated by the paper feed roller 212 (see reference 114a). Figure 1 The friction noise between the paper 10 when it begins to be supplied from the holding section 270 to the supply port 260 is the friction noise between the paper 10 before it floats up near the separating roller 214.

[0145] [Speculation Section]

[0146] The estimation unit 114a, based on the type of paper 10 identified by the recognition unit 113a, inputs information related to friction sounds into the learned model (hereinafter also referred to as the first learned model) corresponding to the identified type of paper 10. The learned models created by the learning unit 140a include multiple first learned models corresponding to multiple types of paper 10 respectively. Based on the type of paper 10 identified by the recognition unit 113a, the estimation unit 114a selects the first learned model corresponding to the identified type of paper 10 from the multiple first learned models stored in the storage unit 120. Furthermore, the estimation unit 114a acquires the friction sounds between the paper 10 collected by the sound collection unit 112 and inputs information related to the acquired friction sounds into the selected first learned model. Based on the output of the first learned model, the estimation unit 114a estimates whether there are signs of a paper jam.

[0147] [Study Department]

[0148] Learning Department 140a uses teacher data for machine learning. For example, Learning Department 140a uses machine learning to create multiple first learned models for multiple types of paper 10, with information related to friction sounds as input and signs of paper jams, such as whether or not the paper floats, as output. That is, Learning Department 140a creates a first learned model corresponding to each of the multiple types of paper 10. The teacher data, for each type of paper 10 (in other words, for each type of paper 10), includes first data consisting of information related to friction sounds and annotations indicating that paper jams have occurred, and second data consisting of information related to friction sounds and annotations indicating that paper jams have not occurred.

[0149] Furthermore, the learning unit 140a uses machine learning to create a second learned model that takes friction sound (i.e., information related to friction sound) as input and the type of paper 10 as output. The teacher data contains data consisting of information related to friction sound and annotations indicating the type of paper 10. The friction sound-related information used in the teacher data can be generated using the friction sound between the paper 10 as it is fed from the holding section 270 to the supply port 260 starting from the feed roller 212, but it can also be generated using the friction sound when no paper jam occurs. The friction sound-related information can be, for example, an image of the spectrum of the friction sound or an image of the frequency characteristics of the friction sound. In this case, the machine learning model can also be a CNN model. Alternatively, the friction sound-related information can also be time-series numerical data containing the electrical signal (e.g., a digitally converted signal) corresponding to the friction sound. In this case, the machine learning model can also be an RNN model.

[0150] [2. Action]

[0151] Next, the operation of the paper jam prediction device 100b will be explained. Figure 8This is a flowchart illustrating the operation of the paper jam prediction device 100b in Embodiment 2.

[0152] like Figure 8 As shown, the sound collecting unit 112 collects the friction sound between the paper 10 generated when the paper 10 is supplied from the holding unit 270 to the inside of the paper feeding device 200 (S201). Here, the sound collecting unit 112 is, for example, a microphone, but it can also function as an acquisition unit that acquires the friction sound collected by the microphone 300 as an electrical signal, as in the variation 1 of embodiment 1.

[0153] Next, the identification unit 113a identifies the type of paper 10 supplied from the holding unit 270 to the inside of the paper feeding device 200 based on the friction sounds collected by the sound collecting unit 112 in step S201 (S202). For example, the identification unit 113a is triggered by the collection of friction sounds by the sound collecting unit 112 and obtains friction sounds from the sound collecting unit 112. At this time, the identification unit 113a identifies the type of paper 10 based on data obtained through machine learning. For example, the identification unit 113a may also identify the type of paper 10 based on the output obtained by inputting friction sounds (more specifically, information related to friction sounds) into a second learned model representing the relationship between the friction sounds of the papers 10 and the type of paper 10.

[0154] Next, the estimation unit 114a inputs information related to the friction sound collected by the sound collection unit 112 in step S201 into the first learned model corresponding to the type of paper 10 identified by the recognition unit 113a in step S202, and obtains an output result (S203). More specifically, based on the type of paper 10 identified by the recognition unit 113a, the estimation unit 114a selects the first learned model corresponding to the type of paper 10 from the plurality of first learned models stored in the storage unit 120, and inputs information related to the friction sound into the selected first learned model. That is, the estimation unit 114a switches the first learned model according to the type of paper 10 supplied from the holding unit 270 to the inside of the paper feeding device 200.

[0155] Next, the estimation unit 114a estimates whether there is a sign of a paper jam based on the output result obtained in step S203 (S204). If the estimation unit 114a estimates that there is a sign of a paper jam in step S204 (yes in S205), the output unit 116 outputs a signal to the paper feeding device 200 to stop feeding paper 10 from the holding unit 270 into the paper feeding device 200 (S206). On the other hand, if the estimation unit 114a estimates that there is no sign of a paper jam in step S204 (no in S205), the paper jam prediction device 100b returns to the processing in step S201.

[0156] [3. Effects, etc.]

[0157] As explained above, in the paper jam prediction device 100b of Embodiment 2, the learned model includes multiple learned models (so-called first learned models) corresponding to multiple types of paper 10 respectively. The paper jam prediction device 100b also includes an identification unit 113a that identifies the type of paper 10 supplied from the holding unit 270 to the inside of the paper feeding device 200. The prediction unit 114a inputs information related to friction noise into the learned model (so-called first learned model) corresponding to the identified type of paper 10 based on the type of paper 10 identified by the identification unit 113a.

[0158] Therefore, the paper jam prediction device 100b can switch the learned model to be used based on the type of paper 10 supplied from the holding section 270 to the inside of the paper feeding device 200. Thus, the paper jam prediction device 100b can predict with high accuracy whether there is a paper jam based on the type of paper 10.

[0159] In the paper jam prediction device 100b of embodiment 2, the identification unit 113a can also identify the type of paper 10 based at least on data obtained through machine learning.

[0160] Therefore, the paper jam prediction device 100b can accurately identify the type of paper 10 being supplied based on data obtained through machine learning.

[0161] (Modification 1 of Implementation Method 2)

[0162] Next, the paper jam prediction device of the modified example 1 of embodiment 2 will be described. Figure 9 This is a diagram illustrating an example of the structure of the paper jam prediction device 100c and the paper feeding device 200 in a variation of Embodiment 2. The paper jam prediction device 100b of Embodiment 2 identifies the type of paper 10 based on friction sounds collected by the sound collection unit 112, but the paper jam prediction device 100c of Variation 1 of Embodiment 2 differs from Embodiment 2 in that it identifies the type of paper 10 based on sensing data representing characteristics of the paper 10, such as surface roughness.

[0163] [1. Structure]

[0164] The paper jam prediction device 100c includes an information processing unit 110c, a storage unit 120, a communication unit 130, a learning unit 140b, and a sensor unit 150. The information processing unit 110c includes a sound collection unit 112, a recognition unit 113b, a prediction unit 114a, and an output unit 116. The recognition unit 113b, the prediction unit 114a, the learning unit 140b, and the sensor unit 150 will be described below.

[0165] [Identification Department]

[0166] The identification unit 113b identifies the type of paper 10 supplied from the holding unit 270 to the interior of the paper feeding device 200. More specifically, the identification unit 113b identifies the type of paper 10 based on data (also called sensing data) acquired by the sensor unit 150. The sensing data is data representing the characteristics of the paper 10. The characteristics of the paper 10 include, for example, the smoothness of the paper 10 surface, the presence or absence of gloss on the surface, thickness, weight, or size. The identification unit 113b can identify the type of paper 10 using either a database that establishes a correspondence between the sensing data and the types of paper 10, or it can identify the type of paper 10 using a learned model (hereinafter also called the third learned model) that takes the sensing data as input and outputs the types of paper 10. Alternatively, the identification unit 113b can use both the database and the third learned model.

[0167] [Speculation Section]

[0168] The estimation unit 114a, based on the type of paper 10 identified by the recognition unit 113b, inputs information related to friction sounds into the first learned model corresponding to the identified paper 10. More specifically, the estimation unit 114a, based on the type of paper 10 identified by the recognition unit 113b, selects the first learned model corresponding to the identified paper 10 from among the multiple first learned models stored in the storage unit 120. Furthermore, the estimation unit 114a acquires the friction sounds between the paper 10s collected by the sound collection unit 112 and inputs information related to the acquired friction sounds into the selected first learned model. Based on the output of the first learned model, the estimation unit 114a estimates whether there are signs of a paper jam.

[0169] [Study Department]

[0170] Learning unit 140b uses teacher data to perform machine learning. Similar to implementation method 2, learning unit 140b creates a first learned model corresponding to each of the multiple types of paper 10.

[0171] Furthermore, the learning unit 140b can also use machine learning to create multiple third-learned models for multiple types of paper 10, taking at least one of the data representing the characteristics of paper 10, such as surface roughness, surface reflectance, and light transmittance, as input, and the type of paper 10 as output. The teacher data includes data consisting of information representing the characteristics of paper 10 and annotations representing the type of paper 10. The information representing the characteristics of paper 10 can be, for example, data representing at least one of the surface roughness, surface reflectance, and light transmittance of paper 10. This data can be in the form of, for example, an image or time-series numerical data.

[0172] [Sensor Department]

[0173] The sensor unit 150 acquires data (sensing data) representing the characteristics of the paper 10 supplied from the holding unit 270 to the interior of the paper feeding device 200. For example, the sensor unit 150 is activated by the collection of friction noise by the sound collecting unit 112. The sensor unit 150 includes, for example, at least one of an image sensor, an ultrasonic sensor, an optical sensor, and a weight sensor. The image sensor acquires image data representing the surface characteristics of the paper 10 by photographing the paper 10. The ultrasonic sensor acquires data representing the thickness of the paper 10 by passing ultrasonic waves through the paper 10. The optical sensor acquires data such as the smoothness of the surface of the paper 10 and the presence or absence of gloss by illuminating the surface of the paper 10 with light. The weight sensor acquires data representing the weight of the paper 10.

[0174] [2. Action]

[0175] Next, the operation of the paper jam prediction device 100c will be explained. Figure 10 This is a flowchart illustrating the operation of the paper jam prediction device in a variation of Embodiment 2, Example 1.

[0176] like Figure 10 As shown, the sound collecting unit 112 collects the friction sound between the paper 10 generated when the paper 10 is supplied from the holding unit 270 to the inside of the paper feeding device 200 (S301). Here, the sound collecting unit 112 is, for example, a microphone, but it can also function as an acquisition unit that acquires the friction sound collected by the microphone 300 as an electrical signal, as in the variation 1 of embodiment 1.

[0177] Although not illustrated, the sensor unit 150 is activated by the collection of friction sounds by the sound collection unit 112, and acquires data representing the characteristics of the paper 10.

[0178] Next, the identification unit 113b identifies the type of paper 10 supplied from the holding unit 270 to the inside of the paper feeding device 200 based on the data obtained by the sensor unit 150 (S302). For example, the identification unit 113b can identify the type of paper 10 by using a database that establishes a correspondence between sensing data and paper types, or it can identify the type of paper 10 by using a learned model (hereinafter also referred to as the third learned model) that outputs the type of paper 10 as input to sensing data. Alternatively, the identification unit 113b can also use both the database and the third learned model.

[0179] Next, the estimation unit 114a inputs information related to the friction sound collected by the sound collection unit 112 in step S301 into the learned model corresponding to the type of paper 10 identified by the recognition unit 113b in step S302, and obtains an output result (S303). More specifically, based on the type of paper 10 identified by the recognition unit 113b, the estimation unit 114a selects the first learned model corresponding to the type of paper 10 from the plurality of first learned models stored in the storage unit 120, and inputs information related to the friction sound into the selected first learned model. That is, the estimation unit 114a switches the first learned model according to the type of paper 10 supplied from the holding unit 270 to the inside of the paper feeding device 200.

[0180] Next, the estimation unit 114a estimates whether there is a sign of a paper jam based on the output result obtained in step S303 (S304). If the estimation unit 114a estimates that there is a sign of a paper jam in step S304 (yes in S305), the output unit 116 outputs a signal to the paper feeding device 200 to stop feeding paper 10 from the holding unit 270 into the paper feeding device 200 (S306). On the other hand, if the estimation unit 114a estimates that there is no sign of a paper jam in step S304 (no in S305), the paper jam prediction device 100b returns to the processing in step S301.

[0181] [3. Effects, etc.]

[0182] As explained above, in the paper jam prediction device 100c of the modified example 1 of embodiment 2, the learned model includes multiple learned models (so-called first learned models) corresponding to multiple types of paper 10 respectively. The paper jam prediction device 100c also includes an identification unit 113b that identifies the type of paper 10 supplied from the holding unit 270 to the inside of the paper feeding device 200. The prediction unit 114a inputs information related to friction noise into the learned model (so-called first learned model) corresponding to the identified type of paper 10 based on the type of paper 10 identified by the identification unit 113b.

[0183] Therefore, the paper jam prediction device 100c can switch the learned model to be used based on the type of paper 10 supplied from the holding section 270 to the inside of the paper feeding device 200. Thus, the paper jam prediction device 100c can predict with high accuracy whether there is a paper jam based on the type of paper 10.

[0184] In the paper jam prediction device 100c of the modified embodiment 2, the identification unit 113b can also identify the type of paper 10 based on data obtained by at least one of an image sensor, an ultrasonic sensor, an optical sensor, a weight sensor, and machine learning.

[0185] Therefore, the paper jam prediction device 100c can identify the type of paper 10 using at least one of a database that establishes a correspondence between data representing the characteristics of paper 10 and the type of paper 10, and a learned model (also called a third learned model) that takes data representing the characteristics of paper 10 as input and outputs the type of paper 10 supplied. Thus, the paper jam prediction device 100c can identify the type of paper 10 with high accuracy.

[0186] Example

[0187] The following examples illustrate the paper jam prediction device and method of this disclosure. However, the following examples are only one example, and this disclosure is not limited to the following examples.

[0188] Hereinafter, we will explain (1) the machine learning models used in Examples 1 and 2, (2) the prediction accuracy of each type of paper when using one learned model (the so-called first learned model), and (3) the prediction accuracy of each type of paper when using learned models (the so-called first learned models) corresponding to the eight types of paper respectively.

[0189] (1) Regarding the machine learning models used in Examples 1 and 2

[0190] Figure 11 This is a diagram used to illustrate the machine learning models used in Embodiments 1 and 2. For example... Figure 11 As shown, the machine learning model used in Examples 1 and 2 is a convolutional neural network (CNN) model. The machine learning model consists of an input layer, a convolutional layer (3×3), a ReLU (normalized linear unit) layer, a pooling layer, a fully coupled layer, a classification layer including a softmax layer, and an output layer.

[0191] (1-1) Regarding the machine learning model used in Example 1

[0192] The machine learning model used in Example 1 is one, which was trained using the following teacher data.

[0193] • Number of data points: 560

[0194] • Teacher data: A dataset including Data 1 and Data 2

[0195] Data 1: Data consisting of a spectrum graph of the sound produced by the friction between the paper pieces and annotations indicating paper jams.

[0196] The second data point consists of a spectrum graph of the sound produced by the friction between the paper pieces and annotations indicating that no paper jams occurred.

[0197] The friction noise between paper sheets is the friction noise produced when the following eight types of paper are fed into the paper feeding device. It is the friction noise between the paper sheets, and it is collected from the start of paper feeding to 30 msec. The types of paper are: premium paper 1, premium paper 2, premium paper 3, premium paper (thin paper), glossy coated paper, pressure sensitive paper base paper, tracing paper, and typewriter paper.

[0198] The teacher data used in Example 1 does not include information related to the type of paper.

[0199] The output is either "normal" (indicating no paper jam) or "abnormal" (indicating a paper jam). Alternatively, "normal" and "abnormal" can also be represented by the values ​​0 and 1.

[0200] (1-2) Regarding the machine learning model used in Example 2

[0201] The machine learning model used in Example 2 consists of 8 models. For each of the 8 types of paper, teacher data corresponding to the type of paper is selected from the teacher data mentioned above, and the models are learned separately.

[0202] (2) Regarding the prediction accuracy of each type of paper when using a single learned model (the so-called first learned model).

[0203] [Example 1]

[0204] In Example 1, using the eight types of paper described above, the friction sounds between the papers were collected as they were fed from the holding section. The spectrum of the friction sounds collected from the start of paper feeding to 30 ms was input into a learned machine learning model trained under the conditions described in (1-1) above. This operation was performed 10 times for each of the eight types of paper. The fed paper was stapled, and the number of times it was correctly predicted (i.e., the number of times it was correctly predicted) out of 10 attempts was counted. The results are expressed as follows: Figure 12 middle. Figure 12 This is a graph showing the results of Example 1.

[0205] Figure 12 The inference accuracy (%) shown indicates the number of inferences that were out of 10 attempts. Regarding the 8 types of paper, different paper materials, thicknesses, and surface roughness can be selected from the corresponding paper types available to the scanner.

[0206] like Figure 12 As shown, in Example 1, the inference accuracy deviates depending on the type of paper. It is believed that the deviation occurs because there are cases where the number of teacher data is relatively small depending on the type of paper.

[0207] (3) Regarding the prediction accuracy of each paper type when using the learned models (the so-called first learned models) corresponding to the 8 types of paper 10 respectively.

[0208] [Example 2]

[0209] In Example 2, the process was the same as in Example 1, except that four types of paper were used among the eight types mentioned above: premium paper 1, premium paper (thin paper), tracing paper, and typewriter paper, and the learned models corresponding to these four types of paper were used. The results are presented in... Figure 13 middle. Figure 13 This is a graph showing the results of Example 2. Figure 14 This is a graph comparing the inference accuracy of Examples 1 and 2 for the four types of paper in Example 2.

[0210] like Figure 13 As shown, the accuracy of the estimate for the four types of paper is over 70%.

[0211] In addition, such as Figure 14 As shown, it was confirmed that if the four learned models corresponding to the four types of paper are used in a way that switches between each type of paper, the prediction accuracy is improved.

[0212] (result)

[0213] Due to the bias in the teacher data, there was a deviation in the accuracy of the prediction in Example 1, but it was confirmed that the presence of signs of a paper jam could be predicted by using a machine learning model.

[0214] Furthermore, based on the results of Examples 1 and 2, it was confirmed that by switching the use of the machine learning model according to each type of paper, it is possible to predict the signs of paper jams with high accuracy, regardless of the type of paper.

[0215] (Other implementation methods)

[0216] The above description, based on the aforementioned embodiments, outlines one or more technical solutions of this disclosure regarding a paper jam prediction device and a paper jam prediction method. However, this disclosure is not limited to these embodiments. Any modifications to the embodiments that can be conceived by those skilled in the art, or combinations of elements from different embodiments, may also be included within the scope of one or more technical solutions of this disclosure, provided they do not depart from the spirit of this disclosure.

[0217] For example, some or all of the constituent elements of the paper jam prediction device described in the above-described embodiments may be constituted by a single system LSI (Large Scale Integration). For example, the paper jam prediction device may also be constituted by a system LSI having a sound collection unit, a prediction unit, and an output unit. Furthermore, the system LSI may not include a microphone.

[0218] A system LSI is a multifunctional LSI that integrates multiple components onto a single chip. Specifically, it is a computer system comprising a microprocessor, ROM (Read Only Memory), RAM (Random Access Memory), etc. The computer program is stored in the ROM. The system LSI performs its functions by having the microprocessor execute the computer program.

[0219] Furthermore, while this is referred to as a system LSI, it is also called IC, LSI, Super LSI, or Very Large Scale LSI, depending on the level of integration. In addition, the method of integrated circuitization is not limited to LSI; it can also be implemented using dedicated circuits or general-purpose processors. It can also utilize FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that can reconfigure the connections or settings of the internal circuit cells of the LSI.

[0220] Furthermore, if advancements in semiconductor technology or other derived technologies lead to the development of integrated circuit technologies that replace LSIs, then these technologies can certainly be used for the integration of functional blocks. This could include applications in biotechnology, among others.

[0221] Furthermore, the technical solution disclosed herein is not only a paper jam prediction device, but also a paper jam prediction method comprising characteristic structural parts included in the device as steps. Furthermore, the technical solution disclosed herein may also be a computer program that enables a computer to execute the characteristic steps included in the paper jam prediction method. Furthermore, the technical solution disclosed herein may also be a computer-readable, non-transitory recording medium containing such a computer program.

[0222] Industrial applicability

[0223] According to this disclosure, it is possible to easily predict signs of paper jams, such as paper floating, based on the output obtained by inputting information related to friction noise during paper feeding into a learned model. Since the paper jam prediction device and method of this disclosure can be applied to devices that feed paper to various processing devices for paper handling, they can be used in a wide range of fields, including household, industrial, and research applications.

[0224] Label Explanation

[0225] 10 sheets of paper

[0226] 15 staples

[0227] 20 bundles of paper

[0228] 30. Location where paper floats

[0229] 100, 100a, 100b, 100c Paper Jam Prediction Device

[0230] Information Processing Departments 110, 110a, 110b, and 110c

[0231] 112, 112a Sound Collection Unit

[0232] Identification sections 113a and 113b

[0233] 114, 114a Speculation Section

[0234] 116 Output Section

[0235] 120 Storage Department

[0236] 130 Ministry of Communications

[0237] Study Departments 140, 140a, and 140b

[0238] 150 Sensors Department

[0239] 200 Paper feeding device

[0240] 210 Conveying Department

[0241] 212, 212a, 212b paper feed rollers

[0242] 214, 214a, 214b separating rollers

[0243] 216, 216a, 216b reduction rollers

[0244] 220 Drive Unit

[0245] 230 Control Department

[0246] 240 Storage Department

[0247] 250 Ministry of Communications

[0248] 260 Supply Port

[0249] 270 Maintenance Section

[0250] 300 microphones

Claims

1. A paper jam prediction device for predicting signs of a paper jam in a paper feeding device, wherein, have: The sound collection unit collects the friction noise generated when paper is fed from the holding part that holds multiple sheets of paper into the inside of the paper feeding device. This friction noise is an inaudible sound generated near the separating roller that separates the sheets of paper one by one in the paper feeding device, due to the friction between the sheets. The inference unit, based on the output obtained by inputting information related to the aforementioned friction sound into the learned machine learning model, infers whether paper buoyancy has occurred in the aforementioned paper feeding device. as well as The output unit, when the above-mentioned prediction unit predicts that the paper has floated up, outputs a signal to the paper feeding device to stop supplying paper to the inside of the paper feeding device.

2. The paper jam prediction device as described in claim 1, wherein, The information related to the aforementioned frictional sound input into the learned model is an image of the spectrum of the aforementioned frictional sound or an image of its frequency characteristics.

3. The paper jam prediction device as described in claim 1 or 2, wherein, The aforementioned friction sound is an inaudible sound generated by the friction between the paper supplied from the aforementioned holding section and the paper held in the aforementioned holding section.

4. The paper jam prediction device as described in claim 3, wherein, The aforementioned inaudible sounds are sounds with frequencies in the ultrasonic range.

5. The paper jam prediction device as described in claim 1 or 2, wherein, The teacher data used in the learning of the above machine learning model includes: The first data consists of information related to the aforementioned friction sound and annotations indicating that the paper floats; and The second data consists of information related to the aforementioned friction sound and a note indicating that no paper floated up.

6. The paper jam prediction device as described in claim 1 or 2, wherein, The aforementioned learned models include multiple learned models corresponding to various types of paper; The aforementioned paper jam prediction device also includes an identification unit that identifies the type of paper supplied from the holding unit to the interior of the paper feeding device; Based on the type of paper identified by the identification unit, the above-mentioned prediction unit inputs information related to the friction sound into the learned model corresponding to the identified type of paper.

7. The paper jam prediction device as described in claim 6, wherein, The aforementioned identification unit identifies the type of paper based on data obtained from at least one of an image sensor, an ultrasonic sensor, an optical sensor, a weight sensor, and machine learning.

8. The paper jam prediction device as described in claim 1 or 2, wherein, The machine learning model described above is a convolutional neural network model.

9. A method for predicting paper jam signs, for predicting signs of paper jams in a paper feeding device, wherein, include: The sound collection step collects the friction noise generated when paper is fed from the holding part that holds multiple sheets of paper into the interior of the paper feeding device. This friction noise is an inaudible sound generated near the separating roller that separates the sheets of paper one by one in the paper feeding device, due to the friction between the sheets. The inference step, based on the output obtained by inputting information related to the above-mentioned friction sound into the learned machine learning model, i.e. the learned model, infers whether paper buoyancy occurs in the above-mentioned paper feeding device. as well as In the output step, if it is suspected that the paper has floated up, a signal is output to the paper feeding device to stop supplying paper to the inside of the paper feeding device.

10. A recording medium, which is a computer-readable, non-transitory recording medium, wherein, The document contains a program for causing a computer to execute the paper jam prediction method of claim 9.

Images