Information processing device, information processing system, and program

Abstract

To enable a ground pattern region in an image that has a character region and a ground pattern region to be removed with good accuracy by a simpler operation than conventionally possible.SOLUTION: A user terminal 10 as an information processing device includes a control unit 11. In the control unit 11, a designation information acquisition unit 103 acquires designation information regarding the designation of a pattern region that indicates the pattern of a ground pattern region, out of an object image having a character region in which a character is formed and a ground pattern region in which a ground pattern is formed. An object pixel specification unit 105 specifies an object pixel that is the object to be removed, on the basis of the result of having analyzed the feature quantity of the pattern region specified by designation information. A ground pattern region removal unit 106 causes the state of the specified object pixel to change, so as to remove the ground pattern region from the object image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, and a program. [Background technology]

[0002] Paper sheets with characters formed on at least a portion thereof may have a background pattern formed thereon to prevent unintentional duplication, etc. In such cases, even if character recognition is attempted, there are portions where the characters and the background pattern overlap, making it difficult to ensure the intended accuracy and visibility. In response to this, methods have been proposed for removing only the background pattern formed on the paper sheet (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2019-82814 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are issues such as the need to prepare a pre-training model, the need for specialized knowledge from the user, and limitations on the number of pixels, color, shape, etc. due to the relationship between the text and the background pattern. For these reasons, there has been a demand for the development of a method that can accurately remove background patterns with just simple user operations.

[0005] An object of the present invention is to enable removal of a background pattern area from an image having a character area and a background pattern area with a simpler operation and higher accuracy than conventional methods. [Means for solving the problem]

[0006] The invention described in claim 1 includes a processor, and the processor acquires designation information relating to designation of a pattern area indicating a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed, and calculates a feature quantity of the pattern area specified by the designation information. The brightness value that indicates the brightness of each pixel is The results of the analysis A comparison of the distribution of the luminance values ​​of the pixels in the grayscaled pattern region with the distribution of the luminance values ​​of the pixels in the image, obtained as follows: Based on the above, the target pixel to be removed is identified, and the identified The image The information processing apparatus is characterized in that the tint block area is removed from the image by changing the state of the target pixel. Claim 2 The invention described in claim 1 is characterized in that the comparison between the distribution of the luminance values ​​of the pixels in the pattern region and the distribution of the luminance values ​​of the pixels in the image is performed by comparing luminance value histograms. 1 The information processing device is described in Claim 3 The invention described in a processor; The processor: In an image having a character area where characters are formed and a background pattern area where a background pattern is formed, designation information regarding designation of a pattern area that indicates the pattern of the background pattern area is acquired, and the image is specified by the designation information. of the pattern area Indicates the R (red), G (green), and B (blue) color components for each pixel, which are characteristic quantities. a distribution of RGB values ​​of pixels in the pattern area obtained as a result of the analysis of the RGB values; and comparing the distribution of the RGB values ​​of the pixels of the image. Based on The image to be removed Identifying the target pixel and changing the state of the identified target pixel of the image to remove the background pattern area from the image. The information processing device is characterized by: The invention described in claim 4 is a processor, the processor acquiring designation information relating to designation of a pattern area indicating a pattern of the background pattern area of ​​an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed, identifying target pixels to be removed based on a result of an analysis of a feature amount of the pattern area identified by the designation information, and removing the background pattern area from the image by changing a state of the identified target pixels; The information processing device is characterized in that the pattern area is designated by a user's touch operation or mouse operation on the image. Claim 5 The invention described in claim 2 is characterized in that the designation of the pattern area is performed by pinching out and in with the user's fingers on the image or by dragging a mouse. 4 The information processing device is described in Claim 6 The invention described in claim 2 is characterized in that the designation of the pattern area is performed by the user pinching out and in with the fingers on the enlarged and displayed image, or by dragging the mouse. 5The information processing device is described in Claim 7 The invention described in claim 2 is characterized in that the shape of the pattern area formed by the touch operation or the mouse operation is any one of a rectangle, a circle, an ellipse, and the shape of the outer edge of an area formed by a filling operation. 4 The information processing device is described in Claim 8 The invention described in a processor; The processor: acquiring designation information for designating a pattern area that indicates a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed, and identifying target pixels to be removed based on a result of an analysis of a feature amount of the pattern area identified by the designation information; and performing control to store the result of the analysis of the feature amount of the pattern region and the result of specifying the target pixel in a database in association with the pattern region or the image. and removing the background pattern area from the image having the same or similar pattern area by changing the state of the identified target pixel using the results of the analysis and the identification stored in the database. The information processing device is characterized in that: Claim 9 The invention described in a processor; The processor: acquires designation information relating to designation of a pattern area indicating a pattern of the background pattern area of ​​an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed, identifies target pixels to be removed based on a result of an analysis of a feature amount of the pattern area identified by the designation information, and performs control to store the result of the analysis of the feature amount of the pattern area and the result of identifying the target pixels in a database in association with the pattern area or the image; generating a machine learning model using the results of the analysis stored in the database and the identified results; and changing the state of the identified target pixel to remove the background pattern area from the image. The information processing device is characterized by: Claim 10 The invention described in claim 2 is characterized in that the processor generates proposal information regarding a proposal to a user using the machine learning model, and removes the background pattern area when a predetermined input operation is performed on the proposal information. 9 The information processing device is described in Claim 11 The invention described in claim 2 is characterized in that the processor generates, as the proposed information, information regarding estimation of at least one of the character area, the background pattern area, and the margin area in the image. 10 The information processing device is described in Claim 12 The invention described in the above is provided with a processor, and the processor controls the display of an image having a character area where characters are formed and a background pattern area where a background pattern is formed, and an input operation for specifying a pattern area showing a pattern of the background pattern area in the image is performed. as the user's touch operation or mouse operation on the image.and acquires the image from which the background pattern area has been removed based on the results of an analysis of the features of the pattern area specified by the input operation, and controls the display of the acquired image. Claim 13 The invention described in the above includes a display means for displaying an image having a character region where characters are formed and a background pattern region where a background pattern is formed, a receiving means for receiving an input operation for specifying a pattern region showing a pattern of the background pattern region in the image, and a feature quantity of the pattern region specified by the input operation. The brightness value that indicates the brightness of each pixel is The results of the analysis A comparison of the distribution of the luminance values ​​of the pixels in the grayscaled pattern region with the distribution of the luminance values ​​of the pixels in the image, obtained as follows: Based on A specifying means for specifying a target pixel to be removed, and a means for changing the state of the specified target pixel of the image to remove the target pixel from the image. and a removal unit that removes the tint block region, wherein the display unit displays the image from which the tint block region has been removed. Claim 14 The invention described in the above provides a computer with a function for acquiring designation information relating to designation of a pattern area that indicates a pattern of a background pattern area in an image having a character area where characters are formed and a background pattern area where a background pattern is formed, and a function for acquiring designation information relating to designation of a pattern area that indicates a pattern of the ... The brightness value that indicates the brightness of each pixel is The results of the analysis A comparison of the distribution of the luminance values ​​of the pixels in the grayscaled pattern region with the distribution of the luminance values ​​of the pixels in the image, obtained as follows: A function for identifying target pixels to be removed based on the identified target pixels. The image By changing the state of the target pixel, The relevant The program is a function for removing the background pattern area from the image. [Effects of the Invention]

[0007] According to the present invention of claim 1, an information processing device can be provided that enables a user to accurately remove a background pattern area simply by performing a simple input operation of specifying a pattern area, even if the user does not have specialized knowledge. Furthermore, since pixels to be removed can be identified from the results of analyzing the brightness values, which are the feature quantities of the pattern region, noise can be suppressed more effectively than by analyzing the entire image. Furthermore, since pixels to be removed can be identified from the distribution of brightness values ​​of pixels in the pattern region, noise can be suppressed more effectively than by analyzing the entire image. Furthermore, since target pixels are identified by comparing the distribution of brightness values ​​of pixels to be removed with the distribution of brightness values ​​of pixels in the image, noise can be suppressed more effectively than by analyzing the entire image. Claim 2 According to the present invention, the target pixel is identified by comparing brightness value histograms showing the distribution of brightness values ​​of the removed pixel with the distribution of brightness values ​​of the pixels of the image, thereby suppressing noise more effectively than analyzing the entire image. Claim3 According to the present invention, It is possible to provide an information processing device that enables a user without specialized knowledge to remove a background pattern area with high accuracy by simply performing a simple input operation of specifying a pattern area. Furthermore, since pixels to be removed can be identified from the results of analyzing the brightness value and RGB value, which are characteristic quantities of the pattern area, noise can be suppressed compared to analyzing the entire image. Since pixels to be removed can be identified from the distribution of RGB values ​​of pixels in the pattern area, noise is suppressed more effectively than when the entire image is analyzed. Furthermore, since the target pixel is identified by comparing the distribution of RGB values ​​of the pixel to be removed with the distribution of RGB values ​​of the pixels in the image, noise is suppressed more effectively than when the entire image is analyzed. Claim 4 According to the present invention, even if the user does not have specialized knowledge, It is possible to provide an information processing device that can accurately remove a background pattern area by simply performing a simple input operation of specifying the pattern area. By simply specifying the pattern area using touch or mouse operations, the background pattern area can be removed with high precision. Claim 5 According to the present invention, even if a user does not have specialized knowledge, the background pattern area can be accurately removed simply by specifying the pattern area by pinching out, pinching in, or dragging the mouse. Claim 6 According to the present invention, even if the background pattern area is small, the image can be enlarged and displayed, allowing pinch-out operations, pinch-in operations, and mouse drag operations to be performed without difficulty. Claim 7 According to the present invention, even if a user does not have specialized knowledge, the background pattern area can be accurately removed simply by specifying the pattern area by pinching out, pinching in, or dragging the mouse. Claim 8 According to the present invention, It is possible to provide an information processing device that enables a user without specialized knowledge to accurately remove a background pattern area by simply performing a simple input operation of specifying a pattern area. Since processing results are accumulated in a database, processing based on past processing results becomes possible. Furthermore, for the same or similar cases, efficient processing can be achieved by utilizing past processing results. Claim 9 According to the present invention, It is possible to provide an information processing device that enables a user without specialized knowledge to accurately remove a background pattern area by simply performing a simple input operation of specifying a pattern area. Furthermore, since processing results are accumulated in a database, processing based on past processing results is possible. Since machine learning models are generated after the fact and their accuracy can be gradually improved, there is no need to prepare machine learning models in advance. Claim 10 According to the present invention, the user can refer to the proposed content and proceed with the work while checking the content of the process. Claim 11 According to the present invention, the user can proceed with the work while checking the details of the process, referring to the contents proposed for the character area, the background pattern area, and the margin area. Claim 12According to the present invention, even if the user does not have specialized knowledge, Touch or mouse operation It is possible to provide an information processing device that can accurately remove a background pattern area simply by performing a simple input operation of specifying a pattern area. Claim 13 According to the present invention, an information processing system can be provided that enables a user to accurately remove a background pattern area simply by performing a simple input operation of specifying a pattern area, even if the user does not have specialized knowledge. Furthermore, since pixels to be removed can be identified from the results of analyzing brightness values, which are characteristic quantities of a pattern region, an information processing system can be provided that can suppress noise more effectively than analyzing the entire image. Furthermore, since pixels to be removed can be identified from the distribution of brightness values ​​of pixels in a pattern region, an information processing system can be provided that can suppress noise more effectively than analyzing the entire image. Furthermore, since target pixels are identified by comparing the distribution of brightness values ​​of pixels to be removed with the distribution of brightness values ​​of pixels in the image, an information processing system can be provided that can suppress noise more effectively than analyzing the entire image. Claim 14 According to the present invention, a program can be provided that enables a user to accurately remove a background pattern area simply by performing a simple input operation of specifying a pattern area, even if the user does not have specialized knowledge. Furthermore, since pixels to be removed can be identified from the results of analyzing brightness values, which are characteristic quantities of a pattern region, a program can be provided that suppresses noise more effectively than analyzing the entire image. Furthermore, since pixels to be removed can be identified from the distribution of brightness values ​​of pixels in a pattern region, a program can be provided that suppresses noise more effectively than analyzing the entire image. Furthermore, since target pixels are identified by comparing the distribution of brightness values ​​of pixels to be removed with the distribution of brightness values ​​of pixels in the image, a program can be provided that suppresses noise more effectively than analyzing the entire image. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an information processing system to which the present embodiment is applied. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of a user terminal as an information processing device to which the present embodiment is applied. [Figure 3] 10 is a diagram illustrating the functional configuration of a control unit of a user terminal when the processes of analyzing a pattern area, identifying a target image, and removing a background pattern area are performed by the user terminal. FIG. [Figure 4] 10 is a diagram illustrating the functional configuration of a control unit of a user terminal in a case where the processes of analyzing a pattern area, identifying a target image, and removing a background pattern area are performed by a management server. FIG. [Figure 5] 10 is a diagram illustrating the functional configuration of a control unit of a management server in a case where the management server performs the processes of analyzing a pattern area, identifying a target image, and removing a background pattern area. FIG. [Figure 6] 10 is a flowchart showing a flow of processing by a user terminal when processing for analyzing a pattern area, specifying a target image, and removing a background pattern area is performed by the user terminal. [Figure 7]10 is a flowchart showing the flow of processing by a user terminal when the processing of analyzing a pattern area, identifying a target image, and removing a background pattern area is performed by a management server. [Figure 8] 10 is a flowchart showing the flow of processing by the management server when the management server performs processing for analyzing a pattern area, identifying a target image, and removing a background pattern area. [Figure 9] 10A and 10B are diagrams showing a specific example of an operation for specifying a pattern area of ​​a target image. [Figure 10] 10A and 10B are diagrams showing a specific example of analysis of brightness values ​​in analysis of feature amounts of a pattern region. [Figure 11] 10A and 10B are diagrams illustrating a specific example of removal of a background pattern area based on the results of analysis of brightness values. [Figure 12] 10A and 10B are diagrams showing a specific example of analysis of a color space based on RGB values ​​in analysis of feature amounts of a pattern region. [Figure 13] 10A and 10B are diagrams showing a specific example of removal of a tint block region based on the results of analysis of a color space based on RGB values. [Figure 14] 10A and 10B are diagrams showing another specific example of an operation for designating a pattern area in a target image. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. (Configuration of information processing system) FIG. 1 is a diagram showing an example of the overall configuration of an information processing system 1 to which the present embodiment is applied. The information processing system 1 is configured by connecting a user terminal 10 and a management server 30 via a network 90. ​​The network 90 is, for example, a LAN (Local Area Network) or the Internet.

[0010] The user terminal 10 is an information processing device operated by a user, such as a personal computer, a smartphone, a tablet terminal, etc. For example, the user terminal 10 acquires information (hereinafter referred to as "designation information") regarding the designation of an area (hereinafter referred to as "pattern area") showing a pattern such as a design or design of the background pattern area (hereinafter referred to as "pattern area") in an image (hereinafter referred to as "target image") having an area where characters are formed (hereinafter referred to as "character area") and an area where a background pattern is formed (hereinafter referred to as "background pattern area").

[0011] Furthermore, for example, the user terminal 10 identifies pixels (hereinafter referred to as "target pixels") to be removed as pixels of the background pattern region from among the multiple pixels constituting the target image based on the results of analyzing the feature quantities of the pattern region identified by the acquired specification information. Furthermore, for example, the user terminal 10 removes the background pattern region from the target image by changing the state of the target pixels identified as pixels of the background pattern region. The details of these processes by the user terminal 10 are stored and managed as processing performance data in a database in the storage unit 13 (see FIG. 2) of the user terminal 10, and are used, for example, as information for generating a machine learning model.

[0012] The management server 30 is an information processing device that serves as a server for managing the entire information processing system 1. The management server 30 can perform part of the above-described processes performed by the user terminal 10. That is, the management server 30 can perform at least part of the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area. In this case, the user terminal 10, for example, acquires various information transmitted from the management server 30 and displays it on a user interface, and transmits information input to the user interface to the management server 30. Details of these processes will be described later.

[0013] The configuration of the information processing system 1 described above is an example, and it is sufficient if the information processing system 1 as a whole has the functions to realize the above-described processing. Therefore, some or all of the functions to realize the above-described processing may be shared or cooperated within the information processing system 1. That is, some or all of the functions of the user terminal 10 may be functions of the management server 30, and some or all of the functions of the management server 30 may be functions of the user terminal 10. Furthermore, some or all of the functions of the user terminal 10 and the management server 30 that constitute the information processing system 1 may be transferred to another server (not shown), etc. This promotes processing by the information processing system 1 as a whole and also enables the processes to complement each other.

[0014] (Hardware configuration of user terminal) FIG. 2 is a diagram showing the hardware configuration of a user terminal 10 as an information processing device to which this embodiment is applied. The user terminal 10 has a control unit 11, a memory 12, a storage unit 13, a communication unit 14, an operation unit 15, and a display unit 16. These units are connected to each other via a data bus, an address bus, a PCI (Peripheral Component Interconnect) bus, etc.

[0015] The control unit 11 is a processor that controls the functions of the user terminal 10 through the execution of various software such as an OS (operating system) and application software. The control unit 11 is configured, for example, by a CPU (Central Processing Unit). The memory 12 is a storage area that stores various software and data used for executing the software, and is used as a working area for calculations. The memory 12 is configured, for example, by a RAM (Random Access Memory).

[0016] The storage unit 13 is a storage area that stores input data for various software programs, output data from various software programs, etc. The storage unit 13 is configured with, for example, a hard disk drive (HDD), a solid state drive (SSD), a semiconductor memory, etc. that are used to store programs, various setting data, etc. The storage unit 13 stores, as databases that store various information, for example, a processing performance DB 901 that stores the above-mentioned processing performance data, a model DB 902 that stores generated machine learning models, etc.

[0017] The communication unit 14 transmits and receives data to and from the management server 30 and external devices via the network 90. ​​The operation unit 15 is configured with, for example, a keyboard, a mouse, mechanical buttons, and switches, and accepts input operations. The operation unit 15 also includes a touch sensor that forms a touch panel integrally with the display unit 16. The display unit 16 is configured with, for example, a liquid crystal display or an organic EL (Electro Luminescence) display used to display information, and displays images, text data, and the like.

[0018] (Management server hardware configuration) The hardware configuration of the management server 30 is similar to the hardware configuration of the user terminal 10 shown in FIG. 2, and therefore illustration and description thereof will be omitted.

[0019] (Functional configuration of the control unit of the user terminal) FIG. 3 is a diagram showing the functional configuration of the control unit 11 of the user terminal 10 when the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the user terminal 10. When the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the user terminal 10, the control unit 11 of the user terminal 10 functions as a display control unit 101, an input operation receiving unit 102, a specified information acquisition unit 103, a pattern area analysis unit 104, a target pixel identification unit 105, a background pattern area removal unit 106, a processing result management unit 107, a model generation unit 108, and a proposal information generation unit 109.

[0020] The display control unit 101, as a display unit, controls the display of various information on the display unit 16 (see FIG. 2). For example, the display control unit 101 controls the display of a user interface on the display unit 16. The user interface can be displayed by activating dedicated application software for users that is pre-installed on the user terminal 10, or by accessing a dedicated website for users. The display control unit 101 controls the display of a target image on the user interface, for example.

[0021] The input operation receiving unit 102 serves as a receiving unit and receives a user's input operation. For example, the input operation receiving unit 102 receives an input operation for designating a pattern area in a target image displayed on the display unit 16 (see FIG. 2). Examples of input operations for designating a pattern area include a user's touch operation on the target image or a mouse operation. Specifically, the pattern area is designated by forming the pattern area by pinching out and in with the user's fingers, dragging the mouse, or the like on at least a part of the background pattern area of ​​the target image.

[0022] The shape of the pattern area formed by these operations can be, for example, a rectangle, a circle, an ellipse, etc. Also, the shape of the outer edge of an area formed by filling in at least a part of the background pattern area by a touch operation, a mouse operation, etc. can be set as the shape of the pattern area. Specific examples of the shape of the formed pattern area will be described later with reference to FIGS. 9 and 14.

[0023] The designation information acquisition unit 103 acquires, as designation information, information relating to the designation of the pattern area received by the input operation reception unit 102. The designation information acquired by the designation information acquisition unit 103 includes at least the feature amount of the pattern area. The feature amount of the pattern area becomes sample data of the feature amount of the background pattern area of ​​the target image.

[0024] The pattern region analysis unit 104 analyzes the feature quantities of the pattern region identified by the specification information acquired by the specification information acquisition unit 103. Note that the feature quantities of the pattern region are not particularly limited. For example, the analysis of the feature quantities of the pattern region may include analysis of the luminance value indicating the brightness of each pixel constituting the pattern region, analysis of the color space based on the RGB values ​​indicating the color components R (red), G (green), and B (blue) of each pixel, analysis of the color space based on the HSV values ​​indicating the color elements H (hue), S (saturation), and V (value) obtained by converting the RGB values, and analysis of the color space based on the LAB values ​​indicating the color elements L (value), A (hue, saturation), and B (hue, saturation).

[0025] The target pixel specifying unit 105 specifies target pixels to be removed as pixels of the background pattern region from among the multiple pixels constituting the target image based on the analysis results of the pattern region analysis unit 104. For example, the target pixel specifying unit 105 specifies target pixels based on the distribution of luminance values ​​of pixels in the grayscaled pattern region obtained as a result of analyzing the luminance values ​​of the pattern region.

[0026] Specifically, the target pixel identifying unit 105 identifies the target pixel based on a comparison between the distribution of luminance values ​​of the pixels constituting the grayscaled pattern area and the distribution of luminance values ​​of the pixels constituting the target image. This comparison can be performed using a luminance histogram, which is a graph of the distribution of luminance values ​​of the pixels constituting the target image. Specific examples of the luminance histogram will be described later with reference to FIGS. 9 and 10.

[0027] Furthermore, for example, the target pixel identifying unit 105 identifies the target pixel based on the distribution of RGB values ​​of the multiple pixels constituting the pattern region, which is obtained as a result of analyzing the RGB values ​​of the pattern region. Specifically, the target pixel identifying unit 105 identifies the target pixel based on a comparison between the distribution of RGB values ​​of the multiple pixels constituting the pattern region and the distribution of RGB values ​​of the multiple pixels constituting the target image. This comparison can be performed using an RGB color space, which is a graph of the distribution of RGB values ​​of the multiple pixels constituting the target image. Specific examples of the RGB color space will be described later with reference to FIGS. 12 and 13.

[0028] The background pattern area removal unit 106 removes the background pattern area from the target image as a removal unit. Specifically, the background pattern area removal unit 106 removes the background pattern area from the target image by changing the state of the target pixel identified by the target pixel identification unit 105. "Changing the state of the target pixel" means, for example, changing the target pixel to white.

[0029] Furthermore, for example, the background pattern area removal unit 106 removes background pattern areas from the target image having the same or similar pattern areas by using the processing history data stored in the database. Specifically, the background pattern area removal unit 106 removes background pattern areas from the target image having the same or similar pattern areas based on the results of machine learning on the processing history data stored in the processing history DB 901 of the storage unit 13 (see FIG. 2).

[0030] Furthermore, for example, when proposal information generated by the proposal information generating unit 109 (described later) is presented to the user and a predetermined input operation is received, the background pattern area removing unit 106 is triggered to remove the background pattern area of ​​the target image. Here, the "predetermined input operation" may be, for example, an input operation for selecting proposal information presented in a selectable manner. Furthermore, a method for presenting the proposal information to the user may be, for example, a method for displaying the proposal information on a user interface.

[0031] The processing history management unit 107 stores and manages processing history data that accumulates the details of processing performed by its own device in a processing history DB 901 (see FIG. 2) of the storage unit 13. The processing history data includes information such as the results of estimating a pattern area, the results of analyzing the feature amounts of the pattern area, and the results of identifying target pixels. The processing history management unit 107 manages information such as the results of estimating a pattern area, the results of analyzing the feature amounts of the pattern area, and the results of identifying target pixels, in association with information that can uniquely identify a pattern area or information that can uniquely identify a specific image.

[0032] The model generation unit 108 generates a machine learning model using the processing result data stored in the database. The machine learning model generated by the model generation unit 108 is stored and managed in a model DB 902 (see FIG. 2) of the storage unit 13.

[0033] The proposal information generation unit 109 generates proposal information regarding proposals to the user. For example, the proposal information generation unit 109 generates proposal information regarding proposals to the user using a machine learning model generated by the model generation unit 108 and stored in the model DB 902. The proposal information generated by the proposal information generation unit 109 includes, for example, information regarding the estimation of each of a character region, a background pattern region, and a margin region in the target image.

[0034] FIG. 4 is a diagram showing the functional configuration of the control unit 11 of the user terminal 10 in the case where the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the management server 30. When the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the management server 30, the control unit 11 of the user terminal 10 functions as a display control unit 101, an input operation reception unit 102, a specified information acquisition unit 103, a transmission control unit 110, and an information acquisition unit 111. Of these, the display control unit 101, the input operation reception unit 102, and the specified information acquisition unit 103 have the same functions as those in FIG. 3 described above, and therefore, description thereof will be omitted.

[0035] When the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the management server 30, the transmission control unit 110 controls the transmission of various information to the management server 30 or to the outside via the communication unit 14 (see FIG. 2). For example, the transmission control unit 110 controls the transmission of designation information, the input of which is accepted by the input operation accepting unit 102, to the management server 30. The information acquisition unit 111 acquires various information. For example, the information acquisition unit 111 acquires data of the target image from which the background pattern area has been removed, which is transmitted from the management server 30.

[0036] (Functional configuration of the control unit of the management server) FIG. 5 is a diagram showing the functional configuration of the control unit of the management server 30 when the management server 30 performs the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area. When the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the management server 30, the control unit of the management server 30 includes an information acquisition unit 301, a pattern area analysis unit 302, a target pixel identification unit 303, a background pattern area removal unit 304, a transmission control unit 305, a processing history management unit 306, a model generation unit 307, and a proposal information generation unit 308. Among these, the functions of the pattern area analysis unit 302, the target pixel identification unit 303, the background pattern area removal unit 304, the processing history management unit 306, the model generation unit 307, and the proposal information generation unit 308 are similar to the functions of the pattern area analysis unit 104, the target pixel identification unit 105, the background pattern area removal unit 106, the processing history management unit 107, the model generation unit 108, and the proposal information generation unit 109 in FIG. 3 , and therefore, description thereof will be omitted.

[0037] The information acquisition unit 301 acquires various types of information via the communication unit. For example, the information acquisition unit 301 acquires data of a target image transmitted from the user terminal 10, specified information for which an input operation has been accepted, and the like. The transmission control unit 305 controls transmission of various types of information to the user terminal 10 or to an external device via the communication unit. For example, the transmission control unit 305 controls transmission of data of a target image from which the background pattern area has been removed to the user terminal 10.

[0038] (User terminal processing) FIG. 6 is a flowchart showing the flow of processing by the user terminal 10 when the processing of analyzing the pattern area, identifying the target image, and removing the tint block area is performed by the user terminal 10. The user terminal 10 displays a target image on the display unit 16 (see FIG. 2) (step 601). Then, when an input operation for designating a pattern area is performed on the target image displayed on the display unit 16 (YES in step 602), the user terminal 10 accepts the input operation (step 603) and acquires the input information as designation information related to the designation of the pattern area (step 604). On the other hand, when an input operation for designating a pattern area is not performed (NO in step 602), the user terminal 10 repeats step 602.

[0039] The user terminal 10 analyzes the feature amount of the pattern area specified by the specification information acquired in step 604 (step 605), and specifies target pixels based on the results of the analysis (step 606).The user terminal 10 then changes the state of the target pixels specified in step 606 to remove the background pattern area from the target image (step 607).

[0040] 7 is a flowchart showing the flow of processing in the user terminal 10 when the processes of analyzing the pattern area, identifying the target image, and removing the background pattern area are performed by the management server 30. Note that the contents of the processing from step 701 to step 704 are the same as the contents of the processing from step 601 to step 604 in FIG. 6, and therefore a description thereof will be omitted.

[0041] The user terminal 10 transmits the specification information acquired in step 704 to the management server 30 together with the data of the target image (step 705). The management server 30 analyzes the feature amount of the pattern area of ​​the target image, identifies target pixels to be removed from among the multiple pixels that make up the target image, and removes the background pattern area from the target image by changing the state of the identified target pixels.

[0042] When data of the target image from which the background pattern area has been removed is transmitted from the management server 30 (YES in step 706), the user terminal 10 acquires the transmitted data of the target image (step 707) and displays it on the display unit 16 (step 708). On the other hand, when data of the target image from which the background pattern area has been removed is not transmitted (NO in step 706), the management server 30 repeats step 706 until data of the target image from which the background pattern area has been removed is transmitted.

[0043] (Management server processing) FIG. 8 is a flowchart showing the flow of processing by the management server 30 when the management server 30 performs the processing of analyzing the pattern area, identifying the target image, and removing the background pattern area. When the management server 30 receives the target image data and the specification information from the user terminal 10 (YES in step 801), it acquires the received target image data and specification information (step 802). On the other hand, if the target image data and specification information have not been received (NO in step 801), the management server 30 repeats step 801 until the target image data and specification information are received.

[0044] The management server 30 analyzes the feature amount of the pattern area specified by the specification information acquired in step 802 (step 803), specifies target pixels based on the results of the analysis (step 804), and changes the state of the specified target pixels to remove the background pattern area from the target image (step 805).The management server 30 then transmits data of the target image from which the background pattern area has been removed to the user terminal 10 (step 806).

[0045] (Example) FIG. 9 is a diagram showing a specific example of an operation for specifying a pattern area of ​​a target image. 9 shows, as a specific example of a target image displayed on the user interface, target image D having a text area W consisting of notations such as "Pay Statement," "April Salary 200,000," and "May Salary 250,000," and a background pattern area G that forms the background of text area W. The user specifies pattern area A by performing a touch operation on a part of background pattern area G.

[0046] For example, when a pinch-out operation is performed to form a part of the background pattern area G, the part may be designated as the pattern area A. Also, when a touch operation is performed on a part of the background pattern area G, a rectangular area with the touched point as its center point is automatically formed, and the part may be designated as the pattern area A.

[0047] Furthermore, if the outer shape of the target image D is small and therefore it is difficult to specify the pattern area A, the target image may be enlarged so that the size of the target image D is large enough to allow for easy specification of the pattern area. In this way, when an input operation for specifying the pattern area A is performed, the input information is acquired as specification information, and analysis of the feature quantities of the pattern area A is started.

[0048] FIG. 10 is a diagram showing a specific example of analysis of brightness values ​​in the analysis of feature amounts of a pattern region. Among the feature quantities of pattern area A identified by the acquired specification information, in the analysis of brightness values, first, the image data of pattern area A is converted into grayscale data to generate a brightness histogram. A "brightness histogram" is a graph with brightness values ​​on the horizontal axis and the number of pixels on the vertical axis.

[0049] 10 shows a specific example of a luminance value histogram of pixels in a grayscaled image of pattern area A. As shown in FIG. 10, the pixels in grayscaled pattern area A are concentrated in the right-hand region of the graph (the region with high luminance values). Once the luminance value histogram of pattern area A is generated, it is analyzed to obtain the minimum luminance value of the pixels in pattern area A.

[0050] FIG. 11 is a diagram showing a specific example of removing a background pattern area based on the results of analyzing the brightness values. Once the brightness histogram for pattern region A is generated, a brightness histogram is similarly generated for target image D. Specifically, the data for target image D is grayscaled to generate a brightness histogram. The upper right section of FIG. 11 shows a specific example of a brightness histogram of the pixels of the grayscaled target image D. The lower right section of FIG. 11 also shows a brightness histogram for pattern region A.

[0051] Once the brightness histogram of the target image D is generated, the brightness histogram of the target image D is compared with the brightness histogram of the pattern area A to identify the target pixels. In the example of Fig. 11, in the brightness histogram of the target image D, the pixels of the background pattern area G are concentrated in the area with high brightness values ​​shown within the dashed line on the right, and the pixels of the character area W are concentrated in the area with low brightness values ​​shown within the dashed line on the left.

[0052] Here, the brightness value of the pixel in the tint block area G with the lowest brightness value is set as a threshold value T, and pixels with brightness values ​​equal to or greater than the threshold value T are identified as "target pixels" and removed. Here, the threshold value T can be determined based on, for example, the minimum brightness value in the pattern area A. Specifically, pixels with brightness values ​​equal to or greater than the threshold value T are identified as "target pixels" and removed by a process of changing the target pixels to white. In the example of FIG. 11, the removal process is performed on pixels located to the right of the threshold value T as the target pixels.

[0053] FIG. 12 is a diagram showing a specific example of analysis of a color space based on RGB values, among analyses of feature amounts of a pattern region. Among the feature quantities of pattern area A identified by the acquired specification information, the analysis of RGB values ​​involves analyzing a color space based on the RGB values ​​of the pixels in the image of pattern area A. Specifically, an RGB color space of pattern area A, such as that shown in Fig. 12, is generated and analyzed.

[0054] An "RGB color space" refers to a color space expressed based on each value of the RGB color model. As shown in FIG. 12, the pixels of pattern area A are concentrated in the right area of ​​the RGB color space. When the RGB color space of pattern area A is generated, the mean value (hereinafter also referred to as "Cmean"), maximum value (hereinafter also referred to as "Cmax"), and minimum value (hereinafter also referred to as "Cmin") of the RGB values ​​of the pixels of pattern area A are obtained. Point P in FIG. 12 indicates the mean value (Cmean) of the RGB values ​​of the pixels of pattern area A.

[0055] Fig. 13 is a diagram showing a specific example of removing a background pattern area based on the results of analyzing a color space based on RGB values. In the example of Fig. 13, target image D has character area W1, character area W2, and background pattern area G. Character area W1 is composed of black characters, and character area W2 is composed of red characters.

[0056] Once the RGB color space for pattern area A is generated, an RGB color space is similarly generated for target image D. The top center of Fig. 13 shows a specific example of the RGB color space of the pixels of target image D. Once the RGB color space for target image D is generated, the RGB color space for target image D is compared with the RGB color space for pattern area A (see Fig. 12) to identify the target pixel.

[0057] 13, in the RGB color space of target image D, the pixels of background pattern area G are concentrated in the area shown within the dashed line on the right, the pixels of character area W1 are concentrated in the area shown within the dashed line on the left, and the pixels of character area W2 are concentrated in the area shown within the dashed line in the center. Once the RGB color space of pattern area A is generated, an analysis is performed to obtain the minimum luminance value (Cmin) of the pixels in pattern area A.

[0058] Here, based on the RGB color space of the target image D and the RGB color space of the pattern region A, the target pixel is specified. For example, based on the RGB value of each pixel in the RGB color space of the target image D and the average value (Cmean), maximum value (Cmax), and minimum value (Cmin) of the RGB values of the pattern region A, the target pixel is specified. Specifically, for pixels whose RGB values are smaller than the larger value between the absolute value of the difference between the maximum value (Cmax) and the average value (Cmean) of the RGB values of the pattern region A and the absolute value of the difference between the average value (Cmean) and the minimum value (Cmin) of the RGB values of the pattern region A in terms of the distance from the average value (Cmean) of the RGB values of the pattern region A indicated by point P, they are removed by a process of changing them to white. That is, the background pattern region G is removed by a process of changing pixels that satisfy the condition of the distance from the average value (Cmean) of the RGB values of the pattern region A indicated by point P < max{|Cmax - Cmean|, |Cmean - Cmin|} to white.

[0059] FIG. 14 is a diagram showing another specific example of an operation for designating a pattern region in a target image. In FIGS. 14(A) to (D), as a specific example of the target image displayed on the user interface, there is shown a target image D having a character region W consisting of notations such as "Salary Slip", "April Salary 200,000", and "May Salary 250,000", and a background pattern region G constituting the background of the character region W.

[0060] The specific examples shown in FIGS. 14(A) and (B) are the same as the example of FIG. 9 described above, where the pattern region A is designated by a touch operation of the user on a part of the background pattern region G. Among these, in the example of FIG. 14(A), when the user performs a touch operation or a click operation on a part of the background pattern region G, a circular region centered on the touched or clicked point is automatically formed, and that region is designated as the pattern region A. Also, in the example of FIG. 14(B), when the user performs an operation of filling in a part of the background pattern region G, the filled region is designated as the pattern region A.

[0061] 14(C) and 14(D) show examples in which a pattern area A is automatically designated without a user touching a part of the background pattern area G. In the example of FIG. 14(C), when the target image D is displayed on the user interface, a corner area of ​​the background pattern area G of the target image D is automatically designated as the pattern area A. In the example of FIG. 14(D), when the target image D is displayed on the user interface, a blank area of ​​the target image D is estimated, and the blank area is automatically designated as the pattern area A. Here, the blank area is estimated using, for example, a machine learning model generated by the model generation unit 108 (see FIG. 3).

[0062] Although the present embodiment has been described above, the present invention is not limited to the above-described embodiment. Furthermore, the effects of the present invention are not limited to those described in the above-described embodiment. For example, the configuration of the information processing system 1 shown in FIG. 1 and the hardware configuration of the user terminal 10 shown in FIG. 2 are merely examples for achieving the object of the present invention and are not particularly limited. Furthermore, the functional configurations of the user terminal 10 shown in FIGS. 3 and 4 and the functional configuration of the management server 30 shown in FIG. 5 are also merely examples and are not particularly limited. It is sufficient for the information processing system 1 of FIG. 1 to be provided with the functionality to execute the above-described processing as a whole, and the functional configuration used to realize this functionality is not limited to the examples of FIGS. 3 to 5.

[0063] Furthermore, the order of the steps in the processing of the user terminal 10 shown in Figures 6 and 7 and the processing of the management server 30 shown in Figure 8 is merely illustrative and is not particularly limited. The processing is not limited to being performed in chronological order according to the illustrated order of steps, and may be performed in parallel or individually, without necessarily being performed in chronological order. Furthermore, the specific examples shown in Figures 9 to 14 are merely illustrative and are not particularly limited.

[0064] In addition, for example, in the above-described embodiment, it is assumed that the user interface for accepting input operations is displayed mainly on the screen of a smartphone or a personal computer, but this is not limiting. For example, the user interface may be displayed on the control panel of a multifunction peripheral. [Explanation of symbols]

[0065] 1...information processing system, 10...user terminal, 11...control unit, 30...management server, 90...network, 101...display control unit, 102...input operation reception unit, 103...designated information acquisition unit, 104...pattern area analysis unit, 105...target pixel identification unit, 106...background pattern area removal unit, 107...processing result management unit, 108...model generation unit, 109...proposed information generation unit, 110...transmission control unit, 111...information acquisition unit, 301...information acquisition unit, 302...pattern area analysis unit, 303...target pixel identification unit, 304...background pattern area removal unit, 305...transmission control unit, 306...processing result management unit, 307...model generation unit, 308...proposed information generation unit

Claims

1. a processor; The processor: acquiring designation information relating to designation of a pattern area indicating a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; Identifying target pixels to be removed based on a comparison between a distribution of luminance values ​​of pixels in the grayscaled pattern region, which is obtained as a result of analyzing luminance values ​​indicating brightness for each pixel, which are characteristic quantities of the pattern region identified by the designation information, and a distribution of luminance values ​​of pixels in the image; The method is characterized in that the background pattern area is removed from the image by changing the state of the target pixel of the identified image. Information processing device.

2. the comparison of the distribution of the luminance values ​​of the pixels in the pattern region with the distribution of the luminance values ​​of the pixels in the image is performed by comparing luminance value histograms. The information processing device according to claim 1 .

3. A processor is provided, The processor: acquiring designation information relating to designation of a pattern area indicating a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; identifying target pixels of the image to be removed based on a comparison between a distribution of RGB values ​​of pixels in the pattern area obtained as a result of analyzing RGB values ​​indicating the color components of R (red), G (green), and B (blue) for each pixel, which are characteristic amounts of the pattern area identified by the designation information, and a distribution of the RGB values ​​of pixels in the image; The method is characterized in that the background pattern area is removed from the image by changing the state of the identified target pixel of the image. Information processing device.

4. A processor is provided, The processor: acquiring designation information relating to designation of a pattern area indicating a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; Identifying target pixels to be removed based on the results of an analysis of the feature amounts of the pattern region identified by the designation information; removing the background pattern area from the image by changing the state of the identified target pixel; The pattern area is specified by a touch operation or a mouse operation by a user on the image. Information processing device.

5. The pattern area is specified by pinching out and in with the user's fingers on the image or by dragging a mouse. The information processing device according to claim 4 .

6. The pattern area is specified by pinching out and in with the user's fingers on the enlarged image or by dragging a mouse. The information processing device according to claim 5 .

7. The shape of the pattern area formed by the touch operation or the mouse operation is any one of a rectangle, a circle, an ellipse, and the shape of the outer edge of an area formed by a filling operation. The information processing device according to claim 4 .

8. A processor is provided, The processor: acquiring designation information relating to designation of a pattern area indicating a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; Identifying target pixels to be removed based on the results of an analysis of the feature amounts of the pattern region identified by the designation information; performing control to store the result of the analysis of the feature amount of the pattern region and the result of identifying the target pixel in a database in association with the pattern region or the image; The method is characterized in that the background pattern area is removed from the image having the same or similar pattern area by changing the state of the identified target pixel using the results of the analysis and the results of the identification stored in the database. Information processing device.

9. A processor is provided, The processor: acquiring designation information relating to designation of a pattern area indicating a pattern of the background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; Identifying target pixels to be removed based on the results of an analysis of the feature amounts of the pattern region identified by the designation information; performing control to store the result of the analysis of the feature amount of the pattern region and the result of identifying the target pixel in a database in association with the pattern region or the image; generating a machine learning model using the results of the analysis and the identified results stored in the database; The method is characterized in that the background pattern area is removed from the image by changing the state of the identified target pixel. Information processing device.

10. The processor: generating suggestion information regarding suggestions to the user using the machine learning model; The method is characterized in that the background pattern area is removed when a predetermined input operation is performed on the proposed information. The information processing device according to claim 9 .

11. The processor generates, as the suggestion information, information regarding estimation of at least one of the character area, the background pattern area, and the margin area in the image. The information processing device according to claim 10.

12. a processor; The processor: Control is performed to display an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; receiving a touch operation or a mouse operation by a user on the image as an input operation for specifying a pattern area in the image that indicates a pattern of the tint block area; acquiring the image from which the background pattern area has been removed based on the result of analyzing the feature amount of the pattern area designated by the input operation; The acquired image is displayed. Information processing device.

13. a display means for displaying an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; a receiving unit for receiving an input operation for specifying a pattern area in the image that indicates a pattern of the tint block area; a specifying means for specifying target pixels to be removed based on a comparison between a distribution of luminance values ​​of pixels in the grayscaled pattern area, which is obtained as a result of analyzing luminance values ​​indicating brightness for each pixel, which is a feature amount of the pattern area specified by the input operation, and a distribution of luminance values ​​of pixels in the image; a removal unit that removes the background pattern area from the image by changing the state of the target pixel of the identified image; and The display means displays the image from which the background pattern area has been removed. Information processing system.

14. On the computer, a function of acquiring designation information relating to designation of a pattern area indicating a pattern of a background pattern area in an image having a character area in which characters are formed and a background pattern area in which a background pattern is formed; a function of identifying target pixels to be removed based on a comparison between a distribution of luminance values ​​of pixels in the grayscaled pattern area, which is obtained as a result of analyzing luminance values ​​indicating brightness for each pixel, which is a feature amount of the pattern area identified by the designation information, and a distribution of luminance values ​​of pixels in the image; a function of removing the background pattern area from the image by changing the state of the target pixel of the identified image; A program to achieve this.

Images