Fax machine, spam fax detection learning system, and spam fax detection system

Abstract

To improve the accuracy of determining an annoying fax.SOLUTION: Provided is an annoying fax determiner training system that comprises a fax device and an information processing device to train a machine learning model that determines whether the fax received by the fax device is an annoying fax. The fax device includes a display control unit that shows a divided image region determined on the basis of the features of image data of the received fax, an operation acceptance unit that accepts operation by a user who designates divided image data having the features of an annoying fax, for divided image data that corresponds to the divided image region, and a communication unit that transmits the divided image data to the information processing device. The information processing device includes a training unit that trains the machine learning model using, as training data, the divided image data, classification information determined on the basis of the features of the divided image data, and the likelihood of the divided image data inputted by the user of being an annoying fax.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a fax device, a nuisance fax determination device learning system, and a nuisance fax determination system.

Background Art

[0002] In recent years, fax devices having a nuisance fax prevention function for rejecting nuisance faxes such as unwanted advertisements have become widespread. Patent Document 1 discloses a technique for recognizing characters included in a predetermined area of image data obtained by reading a received fax document with a scanner in order to identify the transmission destination of the received fax.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, in the conventional technology, the information included in the image data of the fax document cannot be appropriately used, and the problem is that the accuracy of determining a nuisance fax is low.

[0004] An embodiment of the present invention aims to improve the accuracy of determining a nuisance fax by appropriately using the information included in the image data of the fax document in view of the above problems.

Means for Solving the Problems

[0005] To solve the above-mentioned problems, the present invention provides a spam fax detection system comprising a fax machine and an information processing device that learns a machine learning model for determining whether a fax received by the fax machine is a spam fax, wherein the fax machine includes a display control unit that displays a segmented image region determined based on the characteristics of the image data of the received fax, an operation reception unit that accepts user operations to specify segmented image data having the characteristics of a spam fax for segmented image data which is image data corresponding to the segmented image region, and a communication unit that transmits the segmented image data to the information processing device, and the information processing device includes a learning unit that learns a machine learning model using segmented image data, classification information determined based on the characteristics of the segmented image data, and the probability that the segmented image data entered by the user is a spam fax as learning data. [Effects of the Invention]

[0006] According to embodiments of the present invention, the accuracy of identifying spam faxes can be improved. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows an example of a schematic diagram of a spam fax detection learning system according to an embodiment of the present invention. [Figure 2] This figure shows an example of a schematic diagram of a spam fax detection system according to an embodiment of the present invention. [Figure 3] This figure shows an example of the hardware configuration of an information processing device according to an embodiment of the present invention. [Figure 4] This figure shows an example of the hardware configuration of a fax machine according to an embodiment of the present invention. [Figure 5] This figure shows an example of a configuration diagram of a functional block in a spam fax detection learning system according to an embodiment of the present invention. [Figure 6] This figure shows an example of a configuration diagram of a functional block in a spam fax detection system according to an embodiment of the present invention. [Figure 7]This figure shows an example of a flowchart relating to the data generation process for training data according to an embodiment of the present invention. [Figure 8] This figure shows an example of area selection processing (manual) for a fax image according to an embodiment of the present invention. [Figure 9] This figure shows an example of area selection processing (semi-automatic) for a fax image according to an embodiment of the present invention. [Figure 10] This figure shows an example of area selection processing (automatic) for a fax image according to an embodiment of the present invention. [Figure 11] This figure shows an example of a data structure for training data according to an embodiment of the present invention. [Figure 12] This figure shows an example of a sequence relating to a training data generation process (modified version) according to an embodiment of the present invention. [Figure 13] This figure shows an example of positional information of segmented image regions in image data according to an embodiment of the present invention. [Figure 14] This figure shows an example of a data structure for information regarding the position and accuracy of specified segmented image data according to an embodiment of the present invention. [Figure 15] This figure shows an example of a machine learning model for a spam fax detection device according to an embodiment of the present invention. [Figure 16] This figure shows an example of a screen displaying the results of detecting spam faxes according to an embodiment of the present invention. [Figure 17] This figure shows an example of a sequence related to the spam fax detection process according to an embodiment of the present invention. [Modes for carrying out the invention]

[0008] Hereinafter, embodiments of the fax machine, spam fax detection system, and spam fax detection system according to the present invention will be described in detail with reference to the attached drawings.

[0009] [First Embodiment] <System Overview> Figure 1 shows an example of a schematic diagram of a spam fax detection learning system according to an embodiment of the present invention. The spam fax detection learning system 1 includes a fax machine 9a and an information processing device 5a connected to a communication network 3 such as the Internet or a telephone line. In the fax machine 9a, image data obtained by scanning a received fax or printed fax is divided into regions with distinctive features such as shapes and characters. Furthermore, the divided image data (divided image data) is classified based on the characteristics of the image data, and classification information corresponding to the classified category is assigned. The user determines whether each divided image data has the characteristics of a spam fax and determines the probability that it is a spam fax. The probability is set to 1 (or 100%) when the probability of it being a spam fax is highest, and to 0 (or 0%) when the probability is lowest. The fax machine 9a transmits the divided image data with classification information and probability assigned to it to the information processing device 5a via the communication network 3 as learning data (which may also be called training data) for training the spam fax detection system. Here, all divided image data with the characteristics of a spam fax selected by the user may have a probability of 1. The information processing device 5a uses the training data (segmented image data, classification information, and accuracy) received from the fax machine 9a to train a machine learning model for a spam fax detection device, and then transmits the trained machine learning model to the fax machine 9a.

[0010] FIG. 2 is a diagram showing an example of a schematic view of a nuisance fax determination system according to an embodiment of the present invention. The nuisance fax determination system 2 includes, for example, fax devices 9b and 9c and information processing devices 5a and 5b connected to a communication network 3 such as the Internet or a telephone line. The nuisance fax determination system 2 has two forms. In the nuisance fax determination system 2a, the fax device 9b determines whether the received fax is a nuisance fax using the machine learning model of the learned nuisance fax determiner. On the other hand, in the nuisance fax determination system 2b, the fax device 9c transmits the image data of the received fax to the information processing device 5b via the communication network 3. The information processing device 5b determines whether the received image data is a nuisance fax using the machine learning model of the learned nuisance fax determiner, and transmits the determination result to the fax device 9c.

[0011] Here, it is assumed that the fax device 9b and the information processing device 5b have received the machine learning model of the learned nuisance fax determiner from the information processing device 5a in advance and stored it in the storage device. Also, the functions or processes (learning of the machine learning model and nuisance fax determination) realized by the information processing device 5a and the information processing device 5b may be realized by the same device.

[0012] Note that the system configuration shown in FIGS. 1 and 2 is an example. For example, the communication network 3 may include a connection section by wireless communication such as mobile communication or wireless LAN. Also, the number of fax devices 9 and information processing devices 5 may be any number.

[0013] <Hardware Configuration Example> FIG. 3 is a diagram showing an example of the hardware configuration of an information processing apparatus 5 (5a, 5b) according to an embodiment of the present invention. As shown in FIG. 3, the information processing apparatus 5 is constructed by a computer, and includes a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0014] Among these, the CPU 501 controls the operation of the entire information processing apparatus 5. The ROM 502 stores programs used for driving the CPU 501 such as an IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data to and from the HD 504 according to the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I / F 508 is an interface for connecting various external devices. Examples of the external devices in this case include a USB (Universal Serial Bus) memory, a printer, and the like. The network I / F 509 is an interface for performing data communication using the communication network 3. The bus line 510 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 501 shown in FIG. 3.

[0015] The keyboard 511 is a type of input means equipped with multiple keys used for inputting characters, numbers, or various instructions. The pointing device 512 is a type of input means used for selecting and executing various instructions, selecting processing targets, moving the cursor, etc. The DVD-RW drive 514 controls the reading or writing of various data to the DVD-RW 513, which is an example of a removable recording medium. Note that the DVD-RW drive 514 is not limited to DVD-RW, but may also be DVD-R, etc. The media I / F 516 controls the reading or writing (storage) of data to the recording medium 515, such as flash memory.

[0016] <Example Hardware Configuration (Image Forming Apparatus)> Figure 4 shows an example of the hardware configuration of a fax machine 9 (9a, 9b, 9c) according to an embodiment of the present invention. The fax machine 9 may also be called an image forming apparatus (or an MFP, Multifunction Peripheral / Product / Printer), and includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network interface 950.

[0017] Of these, the controller 910 includes the main components of the computer: the CPU 901, system memory (MEM-P) 902, northbridge (NB) 903, southbridge (SB) 904, ASIC (Application Specific Integrated Circuit) 906, local memory (MEM-C) 907, HDD controller 908, and HD 909, with the NB 903 and ASIC 906 connected by an AGP (Accelerated Graphics Port) bus 921.

[0018] Of these, the CPU 901 controls the entire fax machine 9. The NB 903 is a bridge for connecting the CPU 901 with the MEM-P902, SB904, and AGP bus 921, and includes a memory controller that controls reading and writing to the MEM-P902, as well as a PCI (Peripheral Component Interconnect) master and an AGP target.

[0019] MEM-P902 consists of ROM902a, which is a memory for storing programs and data that realize the various functions of the controller 910, and RAM902b, which is used for program and data deployment and drawing during memory printing. The programs stored in RAM902b may be configured to be provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0020] SB904 is a bridge for connecting NB903 to PCI devices and peripheral devices. ASIC906 is an integrated circuit (IC) for image processing applications that has hardware elements for image processing and acts as a bridge connecting the AGP bus 921, PCI bus 922, HDD908, and MEM-C907, respectively. This ASIC906 consists of a PCI target and AGP master, an arbiter (ARB) that forms the core of the ASIC906, a memory controller that controls the MEM-C907, multiple DMACs (Direct Memory Access Controllers) that perform image data rotation using hardware logic, and a PCI unit that performs data transfer via PCI bus 922 between the scanner unit 931, printer unit 932, and fax unit 933. Note that the ASIC906 may also be connected to a USB (Universal Serial Bus) interface or an IEEE1394 (Institute of Electrical and Electronics Engineers 1394) interface.

[0021] The scanner unit 931 controls the scanning function, which reads characters and images printed on paper and obtains image data. The printer unit 932 controls the printer function, which prints the acquired image data. The fax unit 933 controls the fax function, which sends and receives faxes via the network interface 509.

[0022] The short-range communication circuit 920 includes a card reader 920a for reading user authentication information stored on an IC card or the like.

[0023] The control panel 940 has a touch panel 940a and a numeric keypad 940b that accept user input. The touch panel 940a also displays application screens and other information run by the fax machine 9.

[0024] The media interface 916 controls the reading or writing (storage) of data to or from the recording medium 915, such as flash memory.

[0025] <About the features> Figure 5 shows an example of a configuration diagram of a functional block in a spam fax detection learning system according to an embodiment of the present invention. The fax machine 9a has a communication unit 10, a splitting unit 11, a classification unit 12, an operation reception unit 14, a display control unit 15, and a reading unit 16. Each of these units is a function or means realized by the CPU 901 executing instructions included in one or more programs installed on the fax machine 9a that constitutes the spam fax detection learning system 1 of Figure 1. The storage unit 13 can be realized by a storage device such as the HD 909 of the fax machine 9a.

[0026] The communication unit 10 is the communication function of the fax machine 9a, and it transmits and receives faxes and information with the information processing device 5a, etc., via the communication network 3.

[0027] The division unit 11 determines divided image regions from the fax image data, each region having a characteristic such as a graphic or character, and then acquires image data (referred to as divided image data) from each divided image region. Here, the division unit 11 may also ensure that adjacent graphics or characters within a predetermined distance are included in the same divided image region.

[0028] The classification unit 12 determines classification information for the segmented image data acquired by the segmentation unit 11 based on the features of the image (figures, characters, etc.). For example, the classification unit 12 classifies the segmented image data into three types: characters, non-characters such as figures, and a mixture of characters and non-characters, and determines classification information such as "0" for characters, "1" for non-characters, and "2" for a mixture of characters and non-characters.

[0029] The memory unit 13 stores information about the trained spam fax detection machine learning model (number of layers, number of nodes in each layer, weight values, etc.).

[0030] The operation reception unit 14 receives operations from the user via the touch panel 940a or keypad 940b of the fax machine 9a, such as specifying a segmented image area that has the characteristics of a spam fax, and inputting a value indicating the likelihood that it is a spam fax.

[0031] The display control unit 15 displays the fax image data and the result of determining whether it is a spam fax on the touch panel 940a of the fax machine 9a.

[0032] The reading unit 16 acquires image data of a fax by scanning the printed fax using the scanner unit 931 of the fax machine 9a. Alternatively, the reading unit 16 may acquire image data of a fax recorded on a recording medium 915 such as flash memory using the media interface 916 of the fax machine 9a.

[0033] The information processing device 5a includes a communication unit 20, a division unit 21, a classification unit 22, and a learning unit 24. Each of these units is a function or means realized by the CPU 501 executing instructions contained in one or more programs installed on the information processing device 5a that constitutes the spam fax detection learning system 1 shown in Figure 1. The storage unit 23 can be realized by a storage device such as the HD 504 of the information processing device 5a.

[0034] The communication unit 20 is a communication function of the information processing device 5a, and it sends and receives information with the fax machine 9a and the like via the communication network 3.

[0035] The splitting section 21 has the same function or means as the splitting section 11 of the fax machine 9a.

[0036] The classification unit 22 has the same function or means as the classification unit 12 of the fax machine 9a.

[0037] The learning unit 24 performs training on the machine learning model for the spam fax detector. Here, the machine learning model receives segmented image data acquired by the segmentation units 11 and 21 and classification information acquired by the classification units 12 and 22 as input, and outputs the probability that the fax is spam. In other words, the training data for the machine learning model consists of segmented image data, classification information, and the probability. The learning unit 24 performs training by updating the parameters (weight values) of the machine learning model so that the error between the output value output when the segmented image data and classification information are input to the machine learning model and the probability value of the training data is minimized. Details of the training method will be described later.

[0038] The memory unit 13 stores the training data for the machine learning model of the spam fax detector received from the fax machine 9a. The memory unit 23 also stores information about the machine learning model trained by the learning unit 24 (number of layers, number of nodes in each layer, weight values, etc.).

[0039] Figure 6 shows an example of a configuration diagram of a functional block in a spam fax detection system according to an embodiment of the present invention. The fax devices 9b and 9c have a communication unit 10, a splitting unit 11, a classification unit 12, an operation reception unit 14, a display control unit 15, a reading unit 16, and a determination unit 17. Each of these units is a function or means realized by the CPU 901 executing instructions included in one or more programs installed in the fax devices 9b and 9c that constitute the spam fax detection system 2 in Figure 2. The storage unit 13 can be realized by a storage device such as the HD 909 of the fax devices 9b and 9c.

[0040] The communication unit 10, division unit 11, classification unit 12, storage unit 13, operation reception unit 14, display control unit 15, and reading unit 16 are the same as those described as functions or means of the fax machine 9a in Figure 5.

[0041] The determination unit 17 obtains the probability that a fax is spam by inputting the segmented image data and classification information into a trained machine learning model. Alternatively, the determination unit 17 may use, for example, the highest probability value output for multiple segmented image data obtained from the fax image data as the probability value that the fax is spam. The determination unit 17 then determines whether or not the fax is spam based on the probability.

[0042] The information processing device 5b includes a communication unit 20, a division unit 21, a classification unit 22, and a determination unit 25. Each of these units is a function or means realized by the CPU 501 executing instructions contained in one or more programs installed on the information processing device 5b that constitute the spam fax determination system 2. The storage unit 23 can be realized by a storage device such as the HD 504 of the information processing device 5b.

[0043] The communication unit 20, the division unit 21, the classification unit 22, and the storage unit 23 are the same as those described as functions or means of the information processing device 5a in Figure 5.

[0044] The determination unit 25 has the same function or means as the determination unit 17 of the fax machines 9b and 9c.

[0045] <Training data generation process> Figure 7 shows an example flowchart relating to the training data generation process according to an embodiment of the present invention. The training data generated in this sequence is used to train the machine learning model of the spam fax detector in the spam fax detector learning system 1 shown in Figure 1. The processing of each step in Figure 7 will be described below.

[0046] Step S30: The splitting unit 11 of the fax machine 9 acquires segmented image data by dividing the image data of the fax received by the fax machine 9, or the image data obtained by the reading unit 16 of the fax machine 9a scanning a printed fax, into characteristic regions. There are three methods for specifying the regions of the segmented image data to be acquired: manual, semi-automatic, and automatic. Hereafter, the details of how to acquire segmented image data will be explained using the diagrams in Figures 8, 9, and 10, which show examples of manual, semi-automatic, and automatic region specification processing for a fax image according to an embodiment of the present invention, respectively.

[0047] In the processing screen 40 for manual area specification shown in Figure 8, first, the operation reception unit 14 of the fax machine 9a receives an operation to specify the positions of the start point 41 and end point 42 for specifying the area of ​​the divided image data by the user on the touch panel 940a of the fax machine 9a. The display control unit 15 of the fax machine 9a displays the start point 41 and end point 42 specified by the user on the touch panel 940a, and further displays the boundary line of the divided image area 43, which is shown as a rectangle with the start point 41 and end point 42 as diagonal vertices, using a dotted line or the like. The division unit 11 of the fax machine 9a acquires the image data of the divided image area 43 as divided image data. The data format of the image data and the divided image data may be, for example, BMP, JPEG, etc.

[0048] In the processing screen 50 for semi-automatic area selection shown in Figure 9, first, the operation reception unit 14 of the fax machine 9a receives an operation to specify coordinates 51 for specifying an area of ​​divided image data by the user on the touch panel 940a of the fax machine 9a. The user specifies the position by touching the area near the center of an object such as a figure included in the divided image data for which the area is to be specified on the touch panel 940a. The division unit 11 of the fax machine 9a acquires image data of a rectangular divided image area 52 that includes an object such as a figure or character containing the coordinates 51 specified by the user, and objects adjacent to that object within a predetermined distance, as divided image data. The display control unit 15 of the fax machine 9a displays the boundary lines of the divided image area 52 of the divided image data acquired by the division unit 11 on the touch panel 940a using dotted lines or the like.

[0049] In the processing screen 60 for automatic area selection shown in Figure 10, first, the division unit 11 of the fax machine 9a acquires objects such as shapes or characters contained in the image data. Furthermore, the division unit 11 determines, for example, multiple objects whose distance from each other is within a predetermined distance to form a single divided image area. Next, the display control unit 15 of the fax machine 9a displays the boundary lines of the divided image areas acquired by the division unit 11 on the touch panel 940a using dotted lines or the like, as shown in the divided image areas 61 to 65 in Figure 10. The operation reception unit 14 of the fax machine 9a accepts an operation to specify a divided image area, for example, by the user touching the display area of ​​the divided image area on the touch panel 940a of the fax machine 9a. The division unit 11 of the fax machine 9a acquires the image data of the divided image area specified by the user as divided image data.

[0050] As described above, the splitting unit 11 of the fax machine 9a acquires split image data obtained by splitting the fax image data received by the fax machine 9a. Let's return to Figure 7 for further explanation.

[0051] Step S31: The classification unit 12 of the fax machine 9a determines the classification information obtained by classifying the segmented image data acquired in step S30. As a method of classifying the segmented image data, for example, the classification unit 12 classifies the segmented image data into three types: strings, non-strings (figures, etc.), and a mixture of strings and non-strings, by recognizing the strings contained in the segmented image data using OCR. Here, OCR is an abbreviation for Optical Character Recognition. Furthermore, the classification information may be further refined depending on the ratio of strings to non-strings. Alternatively, strings may be further classified by font size, language, whether they are handwritten or not, etc., and non-string images may be further classified by whether they are simple figures composed only of straight lines or complex images such as pictures or photographs.

[0052] Step S32: The operation reception unit 14 of the fax machine 9a receives input from the user regarding the likelihood that the segmented image data acquired in step S30 is a spam fax. The likelihood is set to 1 (or 100%) when the probability of it being a spam fax is highest, and to 0 (or 0%) when the probability is lowest. Here, the user may specify only a likelihood of 0 or 1 (100%), or they may be allowed to specify any value between 0 and 1 (100%).

[0053] Through the above process, the fax machine 9a can generate training data for training a machine learning model of a spam fax detector.

[0054] Furthermore, the communication unit 10 of the fax machine 9a transmits the generated learning data to the communication unit 20 of the information processing device 5a. The storage unit 23 of the information processing device 5a stores the received learning data. Figure 11 is a diagram showing an example of the data structure of learning data according to an embodiment of the present invention. In the data structure 71 of Figure 11, one learning data is arranged in the order of "divided image data", "classification information" for the divided image data, and "probability" that it is a spam fax, and contains a total of M learning data.

[0055] Furthermore, by enabling the information processing device 5a to receive learning data from fax machines 9a connected to a general communication network 3 such as the Internet, it is possible to collect learning data that can handle a wide range of spam faxes. Alternatively, to avoid problems such as receiving inappropriate learning data from malicious users or problems related to the handling of personal information, learning data may be received only from fax machines 9a connected to a limited communication network 3 such as a company LAN. In other words, the fax machines 9a to which learning data can be sent may be determined based on a policy set by the user, taking security into consideration.

[0056] <Modification of the training data generation process> Figure 12 shows an example of a sequence relating to a modified learning data generation process according to an embodiment of the present invention. In this sequence, the process of dividing the image data into segmented image data in step S30 of the learning data generation process shown in Figure 7, and the process of acquiring classification information of the segmented image data in step S31, are performed by the information processing device 5a instead of the fax machine 9a. The processing of each step in Figure 12 will be described below.

[0057] Step S100: The communication unit 10 of the fax machine 9a transmits the image data of the fax received by the fax machine 9a, or the image data obtained by the reading unit 16 of the fax machine 9a scanning the printed fax, to the communication unit 20 of the information processing device 5a.

[0058] Step S101: In step S30 of Figure 7, the division unit 21 of the information processing device 5a acquires objects such as shapes or characters contained in the received image data using the automatic region designation method shown in Figure 10, and determines multiple objects whose distance from each other is within a predetermined distance as a single divided image region. Furthermore, the division unit 21 acquires divided image data corresponding to all the determined divided image regions, and position information of the divided image regions in the image data. Figure 13 is a diagram showing an example of position information of a divided image region in image data according to an embodiment of the present invention. The position information of the divided image region 73 in the image data 72 of Figure 13 can be represented, for example, as two coordinates: the upper left vertex 74 and the lower right vertex 75 of the divided image region 73. Here, the coordinates are represented as (X,Y), where the upper left of the image data 72 is the origin (0,0), and the point is located X to the right and Y downwards by a number of pixels. Returning to Figure 12, we will explain.

[0059] Step S102: The classification unit 22 of the information processing device 5a determines the classification information of the segmented image data using the same procedure as the processing in step S31 in Figure 7.

[0060] Step S103: The communication unit 20 of the information processing device 5a transmits the position information of all divided image regions acquired in step S101 to the communication unit 10 of the fax machine 9a.

[0061] Step S104: The display control unit 15 of the fax machine 9a overwrites the boundary lines of the divided image regions with dotted lines or the like on the image data displayed on the touch panel 940a based on the position information of the received divided image regions. Furthermore, the operation reception unit 14 of the fax machine 9a accepts an operation by the user to specify the divided image data to be used as learning data, and inputs the probability that the specified divided image data is a spam fax, in the same procedure as the processes shown in steps S30 and S32 of Figure 7.

[0062] Step S105: The communication unit 10 of the fax machine 9a transmits the segmented image data specified by the user in step S104 and the information regarding the accuracy entered by the user to the communication unit 20 of the information processing device 5a. Figure 14 is a diagram showing an example of the data structure of the position and accuracy information of the specified segmented image data according to an embodiment of the present invention. In the data structure 76 of Figure 14, the position information in the image data and the accuracy of whether it is a spam fax for the M segmented image data selected by the user are arranged in the order of "position information" and "accuracy". Let's return to Figure 12 for explanation.

[0063] Step S106: The storage unit 23 of the information processing device 5a stores the segmented image data specified by the user as training data, along with the corresponding classification information and accuracy, based on the received information.

[0064] As the above process demonstrates, in the spam fax detection learning system 1, learning data can also be created by having the information processing device 5a perform the division and classification processing of the received fax image data, rather than the fax machine 9a.

[0065] <Training a machine learning model for a spam fax detection system> Figure 15 shows an example of a machine learning model for a spam fax detector according to an embodiment of the present invention. The machine learning model 82 shown in Figure 15 takes segmented image data 80 and classification information 81 from training data generated by the procedure shown in Figure 7 or Figure 12 as input and outputs a confidence score 83, which is the probability that a fax is spam. The machine learning model 82 is composed of, for example, an input layer that accepts input values, at least one intermediate layer, and an output layer that outputs output values, and each layer has a predetermined number of nodes (elements). At each node, an output value is output for the input value to the node using a function having properties such as a sigmoid function, and further, a value obtained by multiplying the output value by a weight is input to the node of the next layer. The input layer has multiple nodes that input pixel values ​​of segmented image data 80 and one node that inputs classification information 81. The pixel values ​​are, for example, integer values ​​from 0 to 255. The number of multiple nodes that input pixel values ​​may be determined based on a predetermined maximum size of segmented image data 80. Here, if the size of the input segmented image data 80 is smaller than the maximum size, the input segmented image data 80 may be placed in the upper left corner of the maximum-size image, and the remaining pixel values ​​may be set to 0. The output layer has one node that outputs a confidence score 83. The confidence score 83 is set to 1 (or 100%) when the probability of it being a spam fax is highest, and to 0 (or 0%) when the probability is lowest.

[0066] In Figure 1 or Figure 2, the learning unit 24 of the information processing device 5a trains the machine learning model by updating the weight values ​​so that the error between the accuracy 83, which is the output value when the segmented image data 80 and classification information 81, which are the training data, are input to the machine learning model 82, and the accuracy of the training data is reduced. The training of the machine learning model can be performed using a generally accepted method. The communication unit 20 of the information processing device 5a transmits the information of the trained machine learning model 82 (number of layers, number of nodes in each layer, weight values, etc.) to the device that performs the spam fax determination (fax machine 9b, information processing device 5b). The storage unit 23 of the information processing device 5a stores the information of the trained machine learning model 82 (number of layers, number of nodes in each layer, weight values, etc.) in the storage device.

[0067] Through the above process, the spam fax detection system 1 can learn a machine learning model that implements a detection system for identifying spam faxes.

[0068] <Processing for detecting spam faxes (when detection is performed by the fax machine)> In the spam fax detection system 2a shown in Figure 2, the fax machine 9b uses a machine learning model that has been pre-trained in the spam fax detection learning system 1 shown in Figure 1 to determine whether a received fax is a spam fax or not, according to the procedure described below.

[0069] First, the division unit 11 of the fax machine 9b determines the divided image regions by dividing the received fax image data, for example, using the automatic region selection method shown in Figure 10. Furthermore, the division unit 11 acquires divided image data for all acquired divided image regions. Next, the classification unit 12 of the fax machine 9b classifies all the divided image data acquired by the division unit 11 and determines classification information using the same procedure as in step S31 in Figure 7. Finally, the determination unit 17 of the fax machine 9b obtains the probability that the fax is spam by inputting the divided image data acquired by the division unit 11 and the classification information determined by the classification unit 12 into a pre-trained machine learning model. Alternatively, the determination unit 17 may use, for example, the highest probability value among the probabilities acquired for multiple divided image data as the probability value for the received fax. The display control unit 15 of the fax machine 9b displays the spam fax determination result on the touch panel 940a of the fax machine 9b based on the information regarding the divided image regions determined by the division unit 11 and the probability value acquired by the determination unit 17. Figure 16 shows an example of a spam fax detection result screen according to an embodiment of the present invention. The detection result screen 120 in Figure 16 displays the detection result 121, the fax image 122, and the divided image regions 123 to 127.

[0070] Judgment Result 121 is the result of determining whether the received fax is a spam fax. Here, the judgment result is that it is a spam fax, and the probability of it being a spam fax is 98%.

[0071] Fax image 122 is an image of a fax that was determined to be spam or not.

[0072] The divided image regions 123 to 127 are the divided screen regions determined by the division unit 11, and the boundaries of these regions are displayed as dotted lines. In addition, the probability value of whether the corresponding divided image data is spam is displayed to the right of the divided screen region.

[0073] <Spam fax detection process (when detected by an information processing device)> In the spam fax detection system 2b shown in Figure 2, the information processing device 5b uses a machine learning model pre-trained in the spam fax detection learning system 1 shown in Figure 1 to determine whether a fax received by the fax machine 9c is a spam fax. Figure 17 shows an example of a sequence relating to the spam fax detection process according to an embodiment of the present invention. Here, two processing methods will be described: when the processing of splitting and classifying the fax image data is performed by the fax machine 9c (referred to as Case A) and when it is performed by the information processing device 5b (referred to as Case B). The processing of each step in Figure 17 will be described below.

[0074] Step S110 (executed only in case A): The splitting unit 11 of the fax machine 9c determines divided image regions by dividing the received fax image data into characteristic regions, for example, by the automatic region selection method shown in Figure 10. Furthermore, the splitting unit 11 acquires divided image data for all acquired divided image regions.

[0075] Step S111 (executed only in case A): The classification unit 12 of the fax machine 9c classifies all the divided image data acquired by the division unit 11 and determines the classification information, following the same procedure as in step S31 in Figure 7.

[0076] Step S112: The communication unit 10 of the fax machine 9c transmits image information relating to the received fax to the communication unit 20 of the information processing device 5b. Here, the image information transmitted is, in case A, the segmented image data and classification information acquired in steps S110 and S111, and in case B, the image data of the received fax.

[0077] Step S113 (executed only in case B): The division unit 21 of the information processing device 5b determines the division image region for the received fax image data and acquires the division image data using the same procedure as in step S110.

[0078] Step S114 (executed only in case B): The classification unit 22 of the information processing device 5b classifies all the divided image data acquired by the division unit 21 and determines the classification information, following the same procedure as in step S111.

[0079] Step S115: The determination unit 25 of the information processing device 5b inputs the segmented image data and classification information acquired in steps S110, S111 or steps S113, S114 into a pre-trained machine learning model to obtain the probability that it is a spam fax.

[0080] Step S116: The communication unit 20 of the information processing device 5b transmits the determination result information, including the accuracy acquired by the determination unit 25 in step S115, to the communication unit 10 of the fax machine 9c. Here, in the determination result information, in order that the correspondence between the segmented image data and the accuracy can be identified, in case A, for example, only the accuracy is placed in the determination result information and transmitted in the same order as the segmented image data is placed in the image information received in step S112. In case B, for example, the determination result information is transmitted in which the accuracy corresponding to the position information in the image data of the segmented image data acquired in step S113 is placed in the same order as the data structure 76 in Figure 14. The display control unit 15 of the fax machine 9c displays the determination result on the touch panel 940a of the fax machine 9c, for example, as shown in the determination result screen 120 in Figure 16, based on the received determination result information.

[0081] As the above process demonstrates, the spam fax detection system 2 in Figure 2 can determine whether a received fax is spam or not by using a machine learning model that has been pre-trained in the spam fax detection system 1 in Figure 1.

[0082] <Retraining based on feedback of judgment results> If the spam fax detection system 2 incorrectly identifies a fax as spam, the user can correct the detection result to create new training data. This new training data can then be used to retrain the machine learning model, thereby improving the detection accuracy. This retraining process is performed, for example, in the spam fax detection system 2 shown in Figure 2, using the following procedure.

[0083] The operation reception unit 14 of the fax machine 9b accepts the operation of selecting the divided image regions 123 to 127 whose accuracy to be corrected, and the operation of inputting the corrected accuracy value, on the judgment result screen 120 of Figure 16.

[0084] The communication unit 10 of the fax machine 9b transmits to the communication unit 20 of the information processing device 5a, which performs training on a machine learning model, training data consisting of segmented image data and classification information corresponding to the segmented image region selected by the user, and accuracy corrected by the user.

[0085] The learning unit 24 of the information processing device 5a uses the received learning data to add to the existing learning data and then performs training (retraining) of the machine learning model.

[0086] The communication unit 20 of the information processing device 5a transmits information about the machine learning model updated by retraining (number of layers, number of nodes in each layer, weight values, etc.) to the communication unit 10 of the fax machine 9b.

[0087] The memory unit 13 of the fax machine 9b stores the information of the received machine learning model.

[0088] The procedure described above is for when the determination of whether a fax is spam is performed by the fax machine 9b, but the same procedure can be used when the determination is performed by the information processing device 5b.

[0089] <Execution of processing based on the judgment result> The spam fax detection system 2 can process received faxes according to the detection result, i.e., the probability value of them being spam faxes. For example, if the probability value of a fax being spam is 0.9 (90%) or higher, the received fax file is deleted. If the probability value is between 0.5 (50%) and 0.9 (90%), the received fax file is moved to a predetermined folder. This reduces the effort required of users to delete or move spam fax files to another folder.

[0090] <Effects of the present invention> In the spam fax detection learning system 1, the fax image data is divided into regions with features such as shapes and characters to generate segmented image data. The segmented image data that the user determines to have the characteristics of a spam fax is used as training data for the machine learning model. Therefore, the training data that is determined to be a spam fax does not include image data from unnecessary regions other than those with the characteristics of a spam fax, making it possible to input appropriate training data into the machine learning model. This makes it possible to improve the accuracy of spam fax detection.

[0091] Furthermore, in the spam fax detection learning system 1, the image data obtained by dividing the fax image data is classified based on features such as shapes and characters, and the classification information corresponding to the classified category is input to the machine learning model along with the divided image data. This makes it possible to train the machine learning model to use appropriate spam fax detection methods according to the characteristics of the image, such as images with only text, images with only shapes, and images with a mixture of text and shapes, thereby improving the accuracy of spam fax detection.

[0092] Furthermore, in the embodiments of the present invention, the process of acquiring segmented image data (segmentation unit), the process of classifying segmented image data (classification unit), and the process of determining whether a fax is spam (determination unit) can be executed in either the fax machine 9 or the information processing device 5. For example, if there are a very large number of fax machines 9 and the program or machine learning model that implements the function is frequently updated, the work and workload for updates can be reduced by having the information processing device 5 execute these processes.

[0093] Although several embodiments for carrying out the present invention have been described above, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0094] For example, the functional block configuration diagrams in Figures 5 and 6 are divided according to their main functions to facilitate understanding of the processing performed by the information processing device 5 and the fax machine 9. The present invention is not limited by the way the processing units are divided or the names of those units. The processing in the information processing device 5 and the fax machine 9 can be further divided into more processing units depending on the processing content. Furthermore, each processing unit can be divided to include even more processing.

[0095] Furthermore, each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0096] Furthermore, the described apparatus represents only one of several computing environments for carrying out the embodiments disclosed herein. In one embodiment, the information processing apparatus 5 and the fax machine 9 include multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and to perform the processing disclosed herein. [Explanation of symbols]

[0097] 1. Spam Fax Detection System Learning System 2. Spam Fax Detection System 3. Communication Network 5, 5a, 5b Information Processing Devices 9, 9a, 9b, 9c Fax machines 10, 20 Communications Department 11, 21 Division 12, 22 Classification section 13, 23 Storage section 14 Operation Reception Section 15 Display Control Unit 16 Reading section 17, 25 Judgment section 24 Learning Department [Prior art documents] [Patent Documents]

[0098] [Patent Document 1] Japanese Patent Publication No. 2011-049871

Claims

1. A spam fax detection system comprising a fax machine and an information processing device that learns a machine learning model for determining whether a fax received by the fax machine is a spam fax, The fax machine, A display control unit that displays a segmented image region determined based on the characteristics of the image data received from the fax, An operation receiving unit that accepts an operation by a user to specify the divided image data having the characteristics of a spam fax, with respect to the divided image data which is the image data corresponding to the divided image region, A communication unit that transmits the segmented image data to the information processing device, It has, The aforementioned information processing device is A learning unit that trains a machine learning model using the segmented image data, classification information determined based on the characteristics of the segmented image data, and the probability that the segmented image data entered by the user is a spam fax as training data, A learning system for detecting spam faxes.

2. The fax machine, A division unit that acquires divided image data corresponding to the divided image region determined based on the distance between figures and characters contained in the image data of the received fax image data, A classification unit that determines the classification information based on information regarding whether the segmented image data contains figures and characters, The spam fax detection learning system according to claim 1, further comprising:

3. The aforementioned information processing device is A division unit that acquires divided image data corresponding to the divided image region determined based on the distance between figures and characters contained in the image data of the received fax image data, A classification unit that determines the classification information based on information regarding whether the segmented image data contains figures and characters, The spam fax detection learning system according to claim 1, further comprising:

4. The display control unit displays the accuracy obtained by inputting the segmented image data and the classification information into the machine learning model. The operation reception unit receives an operation from a user to input a value to correct the acquired accuracy. The learning unit includes the accuracy corrected by the user in the training data and trains the machine learning model. The spam fax detection system learning system according to any one of claims 1 to 3.

5. A reading unit that scans printed faxes to acquire image data, It further possesses, The spam fax detection learning system according to claim 4, wherein the image data acquired by the reading unit can be used in the same way as the image data of a fax received by the fax machine.

6. A fax machine that determines whether a fax is a spam fax, A storage unit that stores a pre-trained machine learning model for determining spam faxes, A splitting unit that acquires split image data by dividing the image data based on the characteristics of the image data of the received fax, A classification unit that determines classification information based on the characteristics of the segmented image data, A determination unit that determines whether a fax is a spam fax based on the probability that it is a spam fax obtained by inputting the segmented image data and the classification information into the machine learning model, A display control unit that displays the determination result determined by the determination unit, A fax machine having a fax function.

7. The fax machine according to claim 6, which deletes the fax file or moves it to a predetermined folder based on the accuracy value.

8. A spam fax detection system comprising a fax machine and an information processing device that determines whether a fax received by the fax machine is a spam fax, The aforementioned information processing device is A storage unit that stores a pre-trained machine learning model for determining spam faxes, A determination unit determines whether a fax is a spam fax based on the probability that it is a spam fax, obtained by inputting the segmented image data obtained by dividing the image data based on the characteristics of the image data of the fax, and the classification information determined based on the characteristics of the segmented image data, into the machine learning model. It has, The fax machine, A communication unit receives the determination result determined by the determination unit from the information processing device, A display control unit that displays the received determination result, A spam fax detection system.

9. The spam fax detection system according to claim 8, which deletes the fax file or moves it to a predetermined folder based on the accuracy value.

Images