Image processing program, image processing method, and image processing device

Abstract

To provide an information processing program, an image processing method, and an image processing apparatus that prevent leakage of personal information from an image including a two-dimensional code.SOLUTION: An image processing program causes a computer to perform processing of decoding a two-dimensional code included in an image, and masking a first image area in the two-dimensional code when predetermined information satisfying a rule that defines personal information is included in a decode result.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an image processing program, an image processing method, and an image processing device. [Background technology]

[0002] When a QR code (Quick Response code (registered trademark)) is added to an image represented by image data, a technique is known for transferring the image data with the QR code deleted from the image (see, for example, Patent Document 1). Also known is a technique for capturing an image displayed on a terminal, determining a mask position for the image based on a two-dimensional code added to the captured image, and performing mask processing on the mask position (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-278579 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-238107 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, two-dimensional codes, including QR codes, may contain personal information such as My Number or names. If personal information is converted into a two-dimensional code, it becomes difficult to visually identify the personal information. However, if an image containing a two-dimensional code is generated and distributed, and the two-dimensional code is decoded during the distribution process, there is a possibility that personal information may be leaked.

[0005] Therefore, in one aspect, an object of the present invention is to provide an image processing program, an image processing method, and an image processing device that prevent personal information from being leaked from an image including a two-dimensional code. [Means for solving the problem]

[0006] In one embodiment, the image processing program decodes a two-dimensional code included in an image, and if the decoded result contains predetermined information that satisfies a rule defining personal information, the program decodes a first image area of the two-dimensional code. With blurring mask If the image contains character information that satisfies the rule, a mask is created by painting over the second image area of the character information. and have the computer execute the processing. [Effects of the Invention]

[0007] It is possible to prevent the leakage of personal information from images containing two-dimensional codes. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of an image processing system. [Figure 2] Figure 2 shows an example of the hardware configuration of an image processing server. [Figure 3] FIG. 3 shows an example of the functional configuration of the image processing server. [Figure 4] Figure 4(a) is a first example of rule information, and Figure 4(b) is a second example of rule information. [Figure 5] FIG. 5 is a third example of rule information. [Figure 6] FIG. 6 is a flowchart showing an example of processing executed by the image processing server. [Figure 7] Fig. 7(a) is an example of image data before masking. Fig. 7(b) is a diagram explaining an example of decoding a two-dimensional code. Fig. 7(c) is a diagram explaining an example of identifying personal information. Fig. 7(d) is an example of image data after masking. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] As shown in FIG. 1, the image processing system ST includes a user terminal 100 and an image processing server 200. The image processing server 200 is an example of an image processing device and executes an image processing method. The user terminal 100 is operated by a user 10. The user terminal 100 includes an input device 110, a display device 120, and a control device 130. Examples of the input device 110 include a keyboard and a mouse. Examples of the display device 120 include a liquid crystal display. The control device 130 receives instructions input through the input device 110 and controls the display of the display device 120 in accordance with the received instructions. The control device 130 also controls communication with the image processing server 200 in accordance with the received instructions. Note that while FIG. 1 shows a PC (Personal Computer) as an example of the user terminal 100, the user terminal 100 may also be a smart terminal such as a smartphone or tablet terminal.

[0011] The control device 130 and the image processing server 200 are connected to each other via a communication network NW. The communication network NW includes one or both of a LAN (Local Area Network) and the Internet. The communication form of the communication network NW may be wired communication or wireless communication.

[0012] Based on the above-mentioned instruction, the control device 130 can transmit image data to the image processing server 200. For example, the control device 130 can transmit image data including a two-dimensional code. The two-dimensional code may be a matrix-format two-dimensional code such as a QR code, or a stack-format two-dimensional code such as a PDF (Portable Data File) 417.

[0013] When the image processing server 200 receives the image data transmitted from the control device 130, it performs various processes, including image processing, on the image data, as described below. For example, the image processing server 200 masks (conceals) the image area of the two-dimensional code included in the image data. After completing the processing, the image processing server 200 transmits the image data after the image processing to the control device 130. As a result, the control device 130 receives the image data after the image processing. Therefore, if the image area of the two-dimensional code is masked, even if the two-dimensional code contains personal information, the two-dimensional code cannot be decoded (decoded or restored), and leakage of personal information from the two-dimensional code can be prevented.

[0014] Next, the hardware configuration of the image processing server 200 will be described with reference to Fig. 2. The hardware configuration of the user terminal 100 is basically the same as that of the image processing server 200, and therefore detailed description thereof will be omitted.

[0015] The image processing server 200 includes a CPU (Central Processing Unit) 200A as a processor, RAM (Random Access Memory) 200B and ROM (Read Only Memory) 200C as memories, and a network I / F 200D. The image processing server 200 may also include at least one of an HDD (Hard Disk Drive) 200E, an input I / F 200F, an output I / F 200G, an input / output I / F 200H, and a drive device 200I, as necessary. The CPU 200A to the drive device 200I are interconnected by an internal bus 200J. In other words, the image processing server 200 can be realized by a computer.

[0016] An input device 710 is connected to the input I / F 200F. Examples of the input device 710 include a keyboard and a mouse. A display device 720 is connected to the output I / F 200G. Examples of the display device 720 include an LCD display. A semiconductor memory 730 is connected to the input / output I / F 200H. Examples of the semiconductor memory 730 include a USB (Universal Serial Bus) memory and a flash memory. The input / output I / F 200H reads the image processing program stored in the semiconductor memory 730. The input I / F 200F and the input / output I / F 200H each include, for example, a USB port. The output I / F 200G includes, for example, a display port.

[0017] A portable recording medium 740 is inserted into the drive device 200I. The portable recording medium 740 may be a removable disk such as a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc). The drive device 200I reads an image processing program recorded on the portable recording medium 740. The network I / F 200D includes, for example, a LAN port. The network I / F 200D is connected to a communication network NW.

[0018] The image processing program stored in the ROM 200C or HDD 200E is temporarily stored in the RAM 200B by the CPU 200A. The image processing program recorded on the portable recording medium 740 is temporarily stored in the RAM 200B by the CPU 200A. By executing the stored image processing program, the CPU 200A realizes various functions described below and executes various processes described below. The image processing program may be one that corresponds to the flowchart described below.

[0019] Next, the functional configuration of the image processing server 200 will be described with reference to Fig. 3 to Fig. 5. Note that Fig. 3 shows the main functions of the image processing server 200.

[0020] 3, the image processing server 200 includes a storage unit 210, a processing unit 220, and a communication unit 230. The storage unit 210 can be realized by one or both of the RAM 200B and the HDD 200E described above. The processing unit 220 can be realized by the CPU 200A described above. The communication unit 230 can be realized by the network I / F 200D described above. Therefore, the storage unit 210, the processing unit 220, and the communication unit 230 are connected to each other.

[0021] Here, the storage unit 210 includes an image storage unit 211 and a rule storage unit 212. On the other hand, the processing unit 220 includes a transmission / reception unit 221, an OCR (Optical character recognition) unit 222, and a decoding unit 223. The processing unit 220 also includes a first determination unit 224, a second determination unit 225, and a mask unit 226.

[0022] The image storage unit 211 stores image data. More specifically, the image storage unit 211 stores image data transmitted from the control device 130 and received by the transmission / reception unit 221. That is, when the transmission / reception unit 221 receives image data, it stores the received image data in the image storage unit 211.

[0023] The rule storage unit 212 stores rule information. The rule storage unit 212 may store one piece of rule information or may store multiple pieces of rule information. Rule information is information that defines personal information. As shown in Figures 4(a) and (b) and Figure 5, the rule information includes an item name and a rule. The type of personal information is registered in the item name. For example, as shown in Figure 4(a), a character string of My Number, which is one type of personal information, is registered in the item name. As shown in Figure 4(b), a character string of a credit card number, which is one type of personal information, may be registered in the item name, or as shown in Figure 5, a character string of a name, which is one type of personal information, may be registered in the item name.

[0024] The personal information registered in the item names is not limited to character strings such as My Number, credit card number, and name. For example, the personal information may be a character string such as a postal code, an address, or a bank account number. In other words, the personal information is not limited to a character string that can uniquely identify an individual, but may also be a character string that can be used to estimate the identity of an individual.

[0025] The rules contained in the rule information include regular expressions and calculation formulas. Personal information can be identified using regular expressions, and also using calculation formulas. Regular expressions are identifiers that determine whether or not something is personal information through pattern matching. In the case of regular expressions, for example, the glob expression format can be used. For example, in the regular expression for the item name "My Number," the character and symbol combination "￥s*" following the string "My Number" indicates whether there are zero or more spaces between the character and symbol ": (colon)." The same is true for the character and symbol combination "￥s*" following the symbol ":." Note that the character, symbol, and number combination "￥d{12}" represents a 12-digit number.

[0026] In other words, since the My Number is represented by a 12-digit number, if the string "My Number", the 12-digit number, and the symbol ":" located between them are included as predetermined information in the 2D data, it can be determined to be personal information using regular expressions. Note that a specific explanation of the My Number is defined and disclosed in the My Number System of the Ministry of Internal Affairs and Communications.

[0027] On the other hand, a formula is an identifier that determines whether or not a piece of personal information is personal information based on the numerical values that define it. For example, if the formula for the item name "My Number" is substituted with an 11-digit number other than the last (first digit) of the 12-digit number, and the calculation result matches the check digit located at the end of the 12-digit number, the 12-digit number can be determined to be a My Number. For example, for the 12-digit number "123456789107," the formula first calculates "1 x (11-5) + 2 x (10-5) + ··· + 5 x (7-5) + 6 x (6+1) + ··· 0 x (1+1)" to obtain "213." Next, "213 ÷ 11" is calculated, resulting in the remainder "4," and then "11-4" is calculated to obtain the result "7." The calculation result "7" matches the check digit "7" at the end of the 12-digit number, so the 12-digit number can be determined to be a My Number.

[0028] Although details are omitted, the item name "Credit Card Number" shown in Figure 4(b) can also basically be determined to be a credit card number using the regular expressions and calculation formulas described above. On the other hand, in the case of the item name "Name" shown in Figure 5, since names do not contain numerical values, it is sufficient to employ regular expressions alone to determine whether or not it is a name. Names may be written in kanji, hiragana, or a mixture of kanji and hiragana. Furthermore, names may be written in katakana or alphabet.

[0029] Returning to FIG. 3, as described above, the transmitting / receiving unit 221 receives image data and stores it in the image storage unit 211. After completing the various processes described below, the transmitting / receiving unit 221 transmits the image data to be processed to the control device 130. The OCR unit 222 acquires the image data stored in the image storage unit 211 and performs optical character recognition processing to generate OCR data. That is, the OCR unit 222 converts image data from which characters cannot be extracted into OCR data from which characters can be extracted. The decoding unit 223 determines whether a two-dimensional code is present in the OCR data, and if a two-dimensional code is present, decodes the two-dimensional code. This makes it possible to identify the information contained in the two-dimensional code.

[0030] The first determination unit 224 determines whether the decoded result contains predetermined information that satisfies the above-described rules. If the decoded result contains predetermined information that satisfies the rules, the second determination unit 225 determines whether the OCR data contains text information that satisfies the rules. If the decoded result contains predetermined information that satisfies the rules, it can be determined that the predetermined information contained in the decoded result is personal information. In this case, since the text information contained in the OCR data is also likely to be personal information, the second determination unit 225 further determines whether the text information is personal information using the rules. If the decoded result contains predetermined information that satisfies the rules, the mask unit 226 masks the image area of the 2D code in the image data. Furthermore, if the OCR data contains text information that satisfies the rules, the mask unit 226 further masks the image area of the text information in the image data.

[0031] Next, the details of the processing executed by the image processing server 200 will be described with reference to FIGS.

[0032] First, as shown in Fig. 6, the transmitting / receiving unit 221 receives image data (step S1). For example, when a portion of the screen displayed on the display device 120 is transmitted as image data from the control device 130, the transmitting / receiving unit 221 receives the image data via the communication unit 230. As shown in Fig. 7(a), the image data received by the transmitting / receiving unit 221 may include a two-dimensional code 11. Note that the control device 130 can generate image data corresponding to a portion of the screen by specifying a portion of the screen and using a technique known as a screenshot.

[0033] When the transmitting / receiving unit 221 receives the image data, the OCR unit 222 generates OCR data (step S2). More specifically, when the image data is received, the transmitting / receiving unit 221 stores the image data in the image storage unit 211, so the OCR unit 222 first acquires the image data from the image storage unit 211. When the image data is acquired, the OCR unit 222 performs optical character recognition processing to generate OCR data.

[0034] When the OCR unit 222 generates the OCR data, the decoding unit 223 determines whether or not a two-dimensional code is present in the OCR data (step S3). For example, in the case of a QR code, the decoding unit 223 scans the OCR data and determines whether or not the OCR data contains a predetermined area in which a square pattern image is located at any three of the four corners. This pattern image is sometimes called an extraction symbol or a position detection pattern. If the OCR data does not contain this predetermined area, the decoding unit 223 determines that a two-dimensional code is not present (step S3: NO). If the OCR data contains this predetermined area, the decoding unit 223 determines that a two-dimensional code is present (step S3: YES).

[0035] If a two-dimensional code is present, the decoding unit 223 decodes the two-dimensional code (step S4). By decoding the two-dimensional code, it is possible to identify source information that indicates the source of the two-dimensional code. After the two-dimensional code is decoded, the first determination unit 224 determines whether the decoded result contains predetermined information that satisfies the rules (step S5).

[0036] For example, as shown in FIG. 7(b), the decoded result of the two-dimensional code 11 may contain character strings, symbols, and numerical values that match the regular expression (see FIG. 4(a)) of the item name "My Number." Furthermore, the numerical values may satisfy the formula (see FIG. 4(a)). In this way, when both the regular expression and the formula are satisfied, the first judgment unit 224 judges that the decoded result contains predetermined information that satisfies the rule (step S5: YES). On the other hand, when both the regular expression and the formula are not satisfied, or when either the regular expression or the formula is satisfied, the first judgment unit 224 judges that the decoded result does not contain predetermined information that satisfies the rule (step S5: NO).

[0037] If the decoded result contains predetermined information that satisfies the rules, masking unit 226 masks the image area of the two-dimensional code in the image data (step S6). The masking may be a process that makes the image area of the two-dimensional code undecipherable. For example, the masking may be a process that blurs the image area of the two-dimensional code using mosaic or the like, or a process that fills it with a color (black, white, red, etc.). In the case of a process that blurs the image area of the two-dimensional code, the image area of the two-dimensional code is blurred, making it undecipherable, as shown in FIG. 7(d).

[0038] After the masking unit 226 masks the image area of the two-dimensional code, the second determination unit 225 determines whether the OCR data contains character information that satisfies the rules (step S7). More specifically, the second determination unit 225 determines, line by line, whether the OCR data contains character information that satisfies the rules. If the two-dimensional code contains personal information, there is a high possibility that the OCR data also contains character information that represents the personal information. Therefore, the second determination unit 225 executes the process of step S7 only when the two-dimensional code contains personal information.

[0039] For example, as shown in Fig. 7(c), some character information included in the OCR data may contain character strings and symbols that match the regular expression (see Fig. 5) for the item name "Name." If the regular expression is satisfied, the second judgment unit 225 judges that the OCR data contains character information that satisfies the rule (step S7: YES). On the other hand, if the regular expression is not satisfied, the second judgment unit 225 judges that the OCR data does not contain character information that satisfies the rule (step S7: NO).

[0040] The second determination unit 225 can also determine whether the OCR data contains character information and numerical values that satisfy the regular expression and formula (see FIG. 4(b)) for the item name "credit card number." In this embodiment, a membership number is shown in FIGS. 7(a) and 7(c), but the OCR data does not contain character information and numerical values that satisfy the regular expression and formula for the item name "credit card number." Therefore, the second determination unit 225 determines that the OCR data does not contain character information and numerical values that satisfy the rules.

[0041] If the OCR data contains character information that satisfies the rule, the masking unit 226 masks the image area of the character information in the image data (step S8). For example, the masking unit 226 can mask the image area of the character information including a specific name, excluding the character string "Name" and the symbol ":". The masking unit 226 may also mask the image area of the character information including the character string "Name", the symbol ":", and the specific name.

[0042] Furthermore, masking can be any process that makes the image area of the text information unviewable. For example, masking can be a process of blurring the image area of the text information, or a process of filling it with a color. In the case of filling the image area of the text information excluding the character string "Name" and the symbol ":" with black, as shown in Figure 7(d), the image area of the text information is filled with black, making the text information unviewable. Note that by making the mask for the image area of the 2D code different from the mask for the image area of the text information, it is possible to determine whether the masked image area is the image area of the 2D code or the image area of the text information.

[0043] Once the image area of the text information has been masked, the transmitter / receiver 221 transmits the image data with the masked image area to the control device 130 (step S9), and the process ends. When both the image area of the two-dimensional code and the image area of the text information have been masked, the control device 130 receives the image data shown in Fig. 7(d). If the control device 130 displays the received image data on the display device 120, the user 10 can check the image data through the display device 120.

[0044] Here, if there is no two-dimensional code in the OCR data in the processing of step S3, the transmitting / receiving unit 221 transmits image data in which the image area is not masked and ends the processing. The same applies if the decoded result does not contain predetermined information that satisfies the rules in the processing of step S5. On the other hand, if there is no character information that satisfies the rules in the OCR data in the processing of step S7, the transmitting / receiving unit 221 transmits image data in which the image area of the two-dimensional code is masked and ends the processing. In other words, the transmitting / receiving unit 221 transmits image data in which the image area of the two-dimensional code is masked and the image area of the character information is not masked and ends the processing.

[0045] In the above-described embodiment, the image data contains one two-dimensional code. However, if the image data contains multiple two-dimensional codes, the image processing server 200 may execute the following process. For example, the first determination unit 224 determines whether each of the multiple two-dimensional codes contains predetermined information that satisfies the rules. The mask unit 226 masks the image area of the two-dimensional code that contains the predetermined information, and excludes the image area of the two-dimensional code that does not contain the predetermined information from the masking target. In other words, the image data contains a mixture of masked and unmasked image areas of two-dimensional codes.

[0046] On the other hand, if at least one image area of the two-dimensional code is masked, the second determination unit 225 determines whether the OCR data contains character information that satisfies the rules. If character information that satisfies the rules is included, the mask unit 226 masks the image area of the character information. This process also makes it possible to prevent personal information from being leaked from an image containing a two-dimensional code.

[0047] Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims. For example, the image processing server 200 may be realized as a physical server or as a virtual server. [Explanation of symbols]

[0048] 100 user terminals 200 Image Processing Server 210 Storage section 211 Image storage unit 212 Rule Memory 220 Processing section 221 Transmitter / Receiver 222 OCR section 223 Decoding section 224 1st Judgment Department 225 Second Judgment Department 226 Mask Section

Claims

1. Decode the two-dimensional code contained in the image, If the decoded result contains predetermined information that satisfies the rules for defining personal information, a first image area of the two-dimensional code is masked by blurring; If the image contains character information that satisfies the rule, a second image region of the character information is masked by filling it in. An image processing program that causes a computer to perform the processing.

2. The rule defines the personal information using a regular expression, masking the first image region when the predetermined information satisfying the regular expression is included in the decoded result; 2. The image processing program according to claim 1, wherein:

3. The rules define the personal information using regular expressions and calculation formulas, masking the first image region when the predetermined information satisfying both the regular expression and the formula is included in the decoded result; 2. The image processing program according to claim 1, wherein:

4. The rule defines the personal information using a regular expression, If the character information satisfying the regular expression is included in the image, the second image region is further masked.

2. The image processing program according to claim 1, wherein:

5. The rules define the personal information using regular expressions and calculation formulas, If the image contains character information that satisfies both the regular expression and the formula, the second image region is further masked.

2. The image processing program according to claim 1, wherein:

6. Decode the two-dimensional code contained in the image, If the decoded result contains predetermined information that satisfies the rules for defining personal information, a first image area of the two-dimensional code is masked by blurring; If the image contains character information that satisfies the rule, a second image region of the character information is masked by filling it in. An image processing method in which processing is performed by a computer.

7. a decoding unit that decodes a two-dimensional code included in an image; a masking unit that, when predetermined information satisfying a rule defining personal information is included in the decoded result, masks a first image area of the two-dimensional code by blurring the first image area, and, when character information satisfying the rule is included in the image, masks a second image area of the character information by filling the second image area; An image processing device comprising:

Images