Information processing device and information processing program

Abstract

To provide an information processor capable of preventing missing information located below a marker when specifying areas to be processed on a document using markers, and an information processing program.SOLUTION: An image processing unit 10 includes: a registration unit 11A that is configured so as to, when a marker is placed on the surface of a document, identify a process target area and register the area information indicating the process target area using the position of the marker; an execution unit 11B that is configured so as to, when the marker is not placed on the surface of the document and the area information is registered, execute a series of predetermined processing on the process target area indicated by the relevant area information; and an identification unit 11C that is configured so as to, when the marker is not placed on the surface of the document and the area information is not registered, register a predetermined area on the surface of the document as a target, and after the area information is registered, identify the process target area indicated by the relevant area information as the area targeted for the above processing.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus and an information processing program.

Background Art

[0002] There has been the following technology related to specifying a processing target area in a document.

[0003] Patent Document 1 discloses a trimming program for a mobile phone that can be installed in a mobile phone having a camera, a processor for processing captured data, and a memory, and that processes photo data obtained by capturing, with the camera, an area on a document surrounded by a rectangular virtual frame formed by arranging four markers in order to desire storage as an image.

[0004] This trimming program for a mobile phone includes a procedure for automatically detecting the four markers from the photo data, and a procedure for automatically trimming a rectangular area defined by the four detected markers from the photo data to generate image data.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, this technology has a problem in that when information such as characters or figures is hidden under the arrangement position of each marker in a document, the information is lost.

[0007] The purpose of this disclosure is to provide an information processing device and an information processing program that can prevent the loss of information located beneath a marker when a processing area on a document is specified using a marker. [Means for solving the problem]

[0008] To achieve the above objective, the information processing apparatus according to the first embodiment includes a processor, which, when identifying a processing target area in a document using the position of a marker placed on the surface of the document, if the marker is placed on the surface of the document, identifies the processing target area using the position of the marker and registers area information indicating the processing target area; if the marker is not placed on the surface of the document and the area information is registered, it performs a predetermined process on the processing target area indicated by the area information; if the marker is not placed on the surface of the document and the area information is not registered, it registers a predetermined area on the surface of the document, and then, when the area information is registered, identifies the processing target area indicated by the area information as the area to be processed.

[0009] Furthermore, the information processing apparatus according to the second embodiment is configured such that, in the information processing apparatus according to the first embodiment, the processor determines whether or not the marker is placed on the surface of the original document when the original document and the marker are stationary.

[0010] Furthermore, in the third embodiment, the information processing device, in the second embodiment, has the processor determine whether the document and the marker are stationary using a captured image of the area where the document is to be placed.

[0011] Furthermore, the information processing device according to the fourth embodiment presents operation mode information indicating which operation mode the processor is operating in: a first operation mode for registering the area information, or a second operation mode for executing the processing.

[0012] Furthermore, the information processing device according to the fifth embodiment is an information processing device according to the fourth embodiment in which the processor presents information indicating that the marker should be separated from the original document when the first operating mode is completed.

[0013] Furthermore, the information processing device according to the sixth embodiment is an information processing device according to the first embodiment in which the marker is capable of identifying the content of the processing.

[0014] Furthermore, the information processing device according to the seventh embodiment is an information processing device according to the sixth embodiment in which the content of the processing can be identified by the external characteristics of the marker.

[0015] Furthermore, the information processing device according to the eighth embodiment is an information processing device according to the seventh embodiment in which the aforementioned external features are at least one of color, intensity, and shape.

[0016] Furthermore, the information processing device according to the ninth embodiment is configured such that, in the information processing device according to the first embodiment, the processor terminates the operation mode for registering the area information when there are multiple markers and the detection of the last marker is completed.

[0017] Furthermore, the information processing device according to the tenth embodiment is an information processing device according to the first embodiment in which the processing is performed using a document camera.

[0018] Furthermore, the information processing device according to the 11th embodiment is an information processing device according to the 10th embodiment in which the processing using the document camera is an image reading process.

[0019] Furthermore, in order to achieve the above object, when the information processing program according to the twelfth aspect identifies a processing target area in a manuscript by using the position of a marker arranged on the surface of the manuscript, if the marker is arranged on the surface of the manuscript, the processing target area is identified by using the arrangement position of the marker, and area information indicating the processing target area is registered. If the marker is not arranged on the surface of the manuscript and the area information is registered, predetermined processing is executed on the processing target area indicated by the area information. If the marker is not arranged on the surface of the manuscript and the area information is not registered, after registering a predetermined area on the surface of the manuscript, when the area information is registered, the processing target area indicated by the area information is specified as the area to be the target of the processing, and causes a computer to execute the processing.

Advantages of the Invention

[0020] According to the first aspect and the twelfth aspect, when designating a processing target area for a manuscript by using a marker, it is possible to prevent loss of information located under the marker.

[0021] According to the second aspect, it is possible to simplify the procedure until the presence or absence of the arrangement of the marker on the surface of the manuscript is identified.

[0022] According to the third aspect, when performing processing using a camera such as a drawing camera, the camera can be shared for identifying a stationary state.

[0023] According to the fourth aspect, it is possible to let the user grasp the operation mode.

[0024] According to the fifth aspect, it is possible to instruct the user of the timing of separation of the marker.

[0025] According to the sixth aspect, it is possible to set processing for a processing target area by using a marker.

[0026] According to the seventh aspect, the content of the process can be intuitively grasped.

[0027] According to the eighth aspect, the content of the process can be grasped using the applied features.

[0028] According to the ninth aspect, the operation mode of registering area information can be automatically terminated.

[0029] According to the tenth aspect, in the process by the calligraphy camera, it is possible to prevent the omission of information located under the marker.

[0030] According to the eleventh aspect, in the process related to the image reading process by the calligraphy camera, it is possible to prevent the omission of information located under the marker.

Brief Description of the Drawings

[0031] [Figure 1] It is a perspective view showing an example of the overall configuration of the image processing apparatus according to the embodiment. [Figure 2] It is a block diagram showing an example of the hardware configuration of the electrical system of the image processing apparatus according to the embodiment. [Figure 3] It is a block diagram showing an example of the functional configuration of the image processing apparatus according to the embodiment. [Figure 4] It is a plan view showing an example of the arrangement state of the marker with respect to the image processing apparatus according to the embodiment. [Figure 5] It is a schematic diagram showing an example of the configuration of the marker-related information database according to the embodiment. [Figure 6] It is a schematic diagram showing an example of the configuration of the process-related information database according to the embodiment. [Figure 7] It is a flowchart showing an example of the information processing according to the embodiment. [Figure 8] It is a front view showing an example of the preview screen according to the embodiment. [Figure 9] It is a front view showing an example of the preview screen according to the embodiment. [Figure 10]This is a front view showing an example of a preview screen according to the embodiment. [Figure 11] This figure illustrates a specific example of a situation in which the information processing according to this embodiment can be handled, and shows an example where text is obscured by a marker. [Figure 12] This figure illustrates a specific example of a situation that the information processing according to this embodiment can handle, and shows an example of an expected processing state. [Figure 13] This figure illustrates a specific example of a situation in which the information processing according to this embodiment can be handled, and shows an example where text is obscured by a marker. [Figure 14] This figure illustrates a specific example of a situation in which the information processing according to this embodiment can be handled, and shows an example of the arrangement of a document and a marker. [Figure 15] This figure illustrates a specific example of a situation in which the information processing according to this embodiment can be handled, and shows an example where text is obscured by a marker. [Figure 16] A perspective view showing an example of the overall configuration of an image processing apparatus according to another embodiment. [Modes for carrying out the invention]

[0032] The embodiments of this disclosure will be described in detail below with reference to the drawings. In this embodiment, an example of how the information processing device of the technology of this disclosure can be applied to an image processing device installed in an office and equipped with a document camera will be described. However, the application of the technology of this disclosure is not limited to offices, but can be applied to any place where an image processing device can be installed, such as schools or homes. Furthermore, the application of the technology of this disclosure is not limited to image processing devices, but can be applied to any device that performs some kind of processing on multiple documents, such as an image reading device that reads images or an image transmission device that transmits read images to another device.

[0033] First, the configuration of the image processing apparatus 10 according to this embodiment will be described with reference to Figure 1. Figure 1 is a perspective view showing an example of the overall configuration of the image processing apparatus 10 according to this embodiment.

[0034] As shown in Figure 1, the image processing apparatus 10 according to this embodiment includes a document tray 30 on which an original document is placed, a user interface unit (hereinafter referred to as "UI unit") 40 for displaying and inputting various information, a tray 50 for ejecting the paper on which the image has been formed, and a paper feed unit 60 for feeding various types of paper.

[0035] Furthermore, the image processing apparatus 10 according to this embodiment is equipped with a document camera 70 configured to capture images of the upper surface of the document table 30. The document camera 70 according to this embodiment is provided at the other end of an arm 72, one end of which is fixed to the rear side of the document table 30, and is positioned so that its field of view substantially coincides with the document placement area 32 on the document table 30.

[0036] In this embodiment, a document camera 70 that captures color moving images is used, but it is not limited to this. For example, a document camera 70 that captures monochrome or grayscale moving images may also be used.

[0037] On the other hand, the UI unit 40 according to this embodiment includes an input unit 14 having various switches, and a display unit 15 composed of a liquid crystal display or the like. The display unit 15 according to this embodiment is configured as a so-called touch panel display, with a light-transmitting touch panel provided on the surface side of the display.

[0038] In this embodiment, a digital multifunction device having image printing, image reading, and image transmission functions is used as the image processing device 10. However, this is not the only possible configuration, and other image processing devices, such as an image processing device having only an image printing function, or an image processing device having only an image printing function and an image reading function, may also be used as the image processing device 10.

[0039] Next, the electrical system configuration of the image processing device 10 according to this embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing an example of the hardware configuration of the electrical system of the image processing device 10 according to this embodiment.

[0040] As shown in Figure 2, the image processing apparatus 10 according to this embodiment includes a CPU (Central Processing Unit) 11 as a processor, a memory 12 as a temporary storage area, a non-volatile storage unit 13, and a UI unit 40 having the input unit 14 and display unit 15 described above. The image processing apparatus 10 according to this embodiment also includes a media read / write (R / W) device 16, a communication interface (I / F) unit 18, and the document camera 70 described above. The CPU 11, memory 12, storage unit 13, UI unit 40, media read / write device 16, communication I / F unit 18, and document camera 70 are connected to each other via bus B. The media read / write device 16 reads information written to the recording medium 17 and writes information to the recording medium 17.

[0041] The storage unit 13 in this embodiment is implemented by an HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. The storage unit 13, as a storage medium, stores an information processing program 13A. The information processing program 13A is stored (installed) in the storage unit 13 when the recording medium 17 on which the information processing program 13A is written is connected to a media read / write device 16, and the media read / write device 16 reads the information processing program 13A from the recording medium 17. The CPU 11 reads the information processing program 13A from the storage unit 13, loads it into memory 12, and sequentially executes the processes contained in the information processing program 13A.

[0042] Furthermore, the memory unit 13 stores a marker-related information database 13B and a processing-related information database 13C. Details of the marker-related information database 13B and the processing-related information database 13C will be described later.

[0043] Next, the functional configuration of the image processing apparatus 10 according to this embodiment will be described with reference to Figure 3. Figure 3 is a block diagram showing an example of the functional configuration of the image processing apparatus 10 according to this embodiment.

[0044] As shown in Figure 3, the image processing apparatus 10 according to this embodiment includes a registration unit 11A, an execution unit 11B, a specification unit 11C, and a presentation unit 11D. The CPU 11 of the image processing apparatus 10 executes an information processing program 13A, thereby enabling the registration unit 11A, the execution unit 11B, the specification unit 11C, and the presentation unit 11D to function.

[0045] In this embodiment, when the registration unit 11A identifies a processing target area in a document using the position of a marker placed on the surface of the document, if a marker is placed on the surface of the document, it identifies the processing target area using the position of the marker and registers area information indicating the processing target area.

[0046] Furthermore, in the case where the marker is not placed on the surface of the document and area information is registered, the execution unit 11B in this embodiment performs a predetermined process on the processing target area indicated by the area information.

[0047] Furthermore, in the case where a marker is not placed on the surface of the document and no area information has been registered, the identification unit 11C in this embodiment registers a predetermined image area on the surface of the document, and then, when area information is registered, identifies the processing target area indicated by the area information as the area to be processed. In this embodiment, the entire surface of the document is applied as the predetermined image area, but it is not limited to this. For example, the area on the document in which characters are written may be applied as the predetermined image area.

[0048] Furthermore, the identification unit 11C according to this embodiment determines whether or not the marker is placed on the surface of the document when the document and marker are stationary. In addition, the identification unit 11C according to this embodiment determines whether or not the document and marker are stationary using a captured image (in this embodiment, an image captured by the document camera 70) of the area where the document is to be placed.

[0049] On the other hand, the display unit 11D according to this embodiment displays operation mode information indicating whether it is operating in a first operation mode for registering area information or a second operation mode for executing the above processing. Furthermore, when the first operation mode is completed, the display unit 11D according to this embodiment displays information indicating that the marker should be separated from the document.

[0050] In this embodiment, the presentation by the presentation unit 11D is represented by the display unit 15 in the UI unit 40, but this is not the only example. For instance, presentation by image formation by the image processing device 10 or presentation by sound by the sound playback device may also be applied as the presentation by the presentation unit 11D.

[0051] Furthermore, in this embodiment, the marker is designed to identify the content of the above-mentioned process. In this embodiment, a machine-readable code such as a barcode, a two-dimensional code, or a pattern image described according to a predetermined rule is used as the marker. In addition, in this embodiment, the machine-readable code is a code that includes information indicating the content of the above-mentioned process. However, the marker is not limited to these forms, and a marker that does not include a code such as a machine-readable code may be used. In this case, the content of the above-mentioned process may be made identifiable by the visual characteristics of the marker. In this form, the visual characteristics may be at least one of color, intensity, and shape. Furthermore, even when a machine-readable code is used as the marker, the content of the process may be made identifiable by the visual characteristics such as the color of the marker.

[0052] Furthermore, in this embodiment, the above-mentioned process is one in which a process using the document camera 70 is applied, and in particular, a process related to image reading is applied as the process using the document camera 70, but this is not limited to this. For example, a process unrelated to the document camera 70, such as image formation processing by the image processing device 10, may also be applied as the above-mentioned process.

[0053] Next, with reference to Figure 4, a method for arranging the markers on the image processing device 10 will be described. Figure 4 is a plan view showing an example of the arrangement of markers on the image processing device 10 according to this embodiment.

[0054] As shown in Figure 4, in the image processing apparatus 10 according to this embodiment, when performing various processes using the document camera 70, the user first places the document 80 at any position in the document placement area 32 of the document table 30.

[0055] As described above, the markers 90A, 90B, ... in this embodiment use machine-readable codes. In the following description, when markers 90A, 90B, ... are not distinguished, they will be collectively referred to as "marker 90".

[0056] The markers 90 according to this embodiment are classified into several types, such as area designation markers that specify the processing target area depending on the position where they are placed, and execution processing designation markers that specify the content of the processing to be performed, and various types of markers 90 are provided in advance.

[0057] Then, as shown in Figure 4, when the user wants to specify a processing area for the document 80 placed on the document glass 30, they place the area specification markers 90A to 90D at the four corners of the processing area. In this embodiment, a rectangle is used as the shape of the processing area, and the method of specifying the processing area is to place the markers 90 at the four corners of the rectangular processing area, but this is not the only way. For example, a rectangle may be used as the shape of the processing area, and the method of specifying the processing area may be to place the markers 90 at a pair of diagonals of the rectangular processing area. Furthermore, the shape of the processing area is not limited to a rectangle; a triangle, trapezoid, polygon with pentagons or more, or a circle may also be used.

[0058] In this embodiment, the marker 90 for specifying a region also specifies the content of the processing to be performed on the specified processing target region. Examples of the content of the processing in this case include cropping, masking, and OCR (Optical Character Recognition / Reader) processing. Examples of the content of the processing specified by the marker 90 for specifying the execution processing include forwarding via email attachment or notification, and printing (image formation) processing.

[0059] In the example shown in Figure 4, markers 90A to 90D are used to specify areas for masking. In this case, the area to be processed (in the example shown in Figure 4, the area surrounding the My Number digits) 92 enclosed by these markers 90 is subjected to masking.

[0060] Furthermore, in the example shown in Figure 4, marker 90E is a marker 90 used to specify the execution process. In this case, there are no particular restrictions on the position where marker 90E can be placed; it can be placed anywhere within the document placement area 32, regardless of whether it is inside or outside the top surface of the document 80.

[0061] As described above, the image processing apparatus 10 according to this embodiment operates as follows when identifying the processing target area 92 on the original document 80 using the position of the marker 90 placed on the surface of the original document 80.

[0062] First, when a marker 90 for specifying an area is placed on the surface of the original document 80, the image processing device 10 identifies the area to be processed using the position of the marker 90 and registers area information indicating the area to be processed.

[0063] Furthermore, if the area designation marker 90 is not placed on the surface of the original document 80 and area information is registered, the image processing device 10 performs a predetermined process on the processing target area indicated by the area information.

[0064] Furthermore, if the area designation marker 90 is not placed on the surface of the document 80 and area information has not been registered, the image processing device 10 registers a predetermined area on the surface of the document 80 as the target area. After this registration, if area information is registered, the image processing device 10 identifies the area to be processed indicated by the area information as the area to be processed.

[0065] Next, with reference to Figure 5, the marker-related information database 13B according to this embodiment will be described. Figure 5 is a schematic diagram showing an example of the configuration of the marker-related information database 13B according to this embodiment.

[0066] The marker-related information database 13B according to this embodiment is a database in which information about the marker 90 described above is registered. As an example, as shown in Figure 5, the marker-related information database 13B according to this embodiment stores information such as marker type, placement method, marker ID (Identification), and processing details in association with each other.

[0067] The above marker type is information indicating the type of marker 90 described above, and the above placement method is information indicating the placement method of marker 90 described above. Furthermore, the above marker ID is information that is assigned in advance to identify the corresponding marker, and is different for each marker type and processing content, and the above processing content is information indicating the content of the processing represented by the corresponding marker 90.

[0068] In the example shown in Figure 5, for instance, marker 90, assigned the marker ID "A01," has registered information indicating that its type is area specification, its placement method is to enclose the processing target area, and its processing content is image cropping. Therefore, by referring to the marker-related information database 13B, it is possible to understand the type, placement method, and processing content of marker 90 as pre-registered in the database.

[0069] In other words, in this embodiment, the marker 90 does not contain information indicating the content of the process itself, but it does contain information indicating the marker ID. In this embodiment, the content of the corresponding process is identified by obtaining information indicating the content of the process corresponding to the marker ID from the marker-related information database 13B. However, this is not the only form, and the marker 90 may also contain information indicating the content of the process itself. In this case, the marker-related information database 13B is unnecessary.

[0070] Next, the processing-related information database 13C according to this embodiment will be described with reference to Figure 6. Figure 6 is a schematic diagram showing an example of the configuration of the processing-related information database 13C according to this embodiment.

[0071] The processing-related information database 13C according to this embodiment is a database in which information related to a process specified by the user using a marker 90 is registered. As an example, as shown in Figure 6, the processing-related information database 13C according to this embodiment stores information on processing content and setting values ​​in association with each other.

[0072] The above processing details are information indicating the content of the corresponding process, and correspond to the processing details in the marker-related information database 13B. The above setting values ​​are information set for the corresponding process.

[0073] If marker 90 is a marker for specifying an area, then four markers 90 are used as a set to specify one processing area. In this case, the setting value (corresponding to the "area information" mentioned above) is a single coordinate information that indicates the processing area specified by the placement of the set of markers 90. In this embodiment, the coordinate information used is a two-dimensional coordinate system with the upper left corner of the original document 80 as the origin, but it is not limited to this. For example, the coordinate information could be a two-dimensional coordinate system with the lower left, upper right, or lower right corner of any of the four corner points of the original document 80 as the origin. Also, in this embodiment, as shown in Figure 6, the coordinate information used is a pair of diagonal positions of the processing area, but it goes without saying that it is not limited to this.

[0074] Thus, in this embodiment, since any corner point of the original document 80 is applied as the coordinate system indicating the position of the processing area, the original document 80 can be placed at any position in the original document placement area 32.

[0075] Furthermore, if marker 90 is a marker other than one used for area designation, one setting value may be set for each marker 90. For example, if the processing involves forwarding an attachment via email, the setting value will be information indicating the forwarding destination (in this embodiment, the device name), while if the processing involves printing, no setting value is required.

[0076] In this embodiment, we describe a case where necessary settings, such as information indicating the transfer destination, are pre-registered to avoid confusion, but this is not the only case. For example, the user may be required to input the settings as appropriate.

[0077] In the example shown in Figure 6, for example, information is registered to perform a masking process on a rectangular area where the diagonal positions of a pair of corners are (X1,Y1)-(X2,Y2) for the image representing the original document 80. Also, in the example shown in Figure 6, the processing of the image representing the original document 80 is registered to send the image information representing the original document 80, after the masking process has been performed, to server A as an email attachment.

[0078] Next, the operation of the image processing apparatus 10 according to this embodiment when performing information processing will be explained with reference to Figures 7 to 10. Figure 7 is a flowchart of an example of information processing according to this embodiment.

[0079] The information processing shown in Figure 7 is executed when the CPU 11 of the image processing device 10 executes the information processing program 13A when the user instructs the user to execute the information processing via the UI unit 40. However, the method of executing the information processing is not limited to this method specified by the user. For example, the information processing may be executed when the document 80 and marker 90 are placed in the document placement area 32 of the document glass 30 after a predetermined period (10 minutes in this embodiment) has elapsed since the last time information processing was completed. Furthermore, in the following, to avoid confusion, when the user places the marker 90, the area designation marker 90 will always be used.

[0080] In step 100 of Figure 7, the CPU 11 sends an instruction to the document camera 70 to start capturing moving images. In response to this instruction, the document camera 70 starts capturing moving images.

[0081] On the other hand, after issuing an instruction to execute information processing, the user places the document 80 in the document placement area 32 and, if necessary, places the marker 90 corresponding to the desired processing.

[0082] Therefore, in step 102, the CPU 11 refers to the image at this point in time (a still image, hereinafter referred to as the "reference image") in the captured image (hereinafter simply referred to as the "captured image"), which is a color moving image obtained from the document camera 70. The CPU 11 then determines whether or not the marker 90 is included in the subject of the reference image. If the determination is negative, the system proceeds to step 108; if the determination is positive, the system proceeds to step 104. In this embodiment, the determination of whether or not the marker 90 is included in the subject is made by using a conventionally known technique to determine whether or not the subject contains an image showing the machine-readable code described above, but this is not the only method. For example, information indicating the shape of the marker 90 may be registered in advance, and the presence or absence of the marker 90 may be determined by pattern matching using this information. Alternatively, the image of marker 90 may be detected using AI (Artificial Intelligence) that employs a segmentation model such as a pre-trained semantic segmentation model or an instance segmentation model.

[0083] In step 104, the CPU 11 waits until the movement of all markers 90, which is caused by the placement of markers 90 by the user, comes to a standstill. In this embodiment, the CPU 11 determines that the movement of all markers 90 has come to a standstill when the subject in the captured image remains motionless for a predetermined period (5 seconds in this embodiment), but this is not the only way. For example, a marker 90 that indicates that it is the last marker to be placed may be prepared, and the CPU 11 may determine that the movement of all markers 90 has come to a standstill when this marker 90 is detected in the captured image and comes to a standstill.

[0084] In step 106, the CPU 11 refers to the reference image described above and detects the placement positions of all markers 90 included in the reference image. In this embodiment, the detection of the placement positions of all markers 90 is performed by identifying the positions of images showing the machine-readable codes described above that are included in the subject, but this is not the only method. For example, information indicating the shape of the markers 90 may be registered in advance, and the placement positions of all markers 90 may be detected by identifying the locations where the markers 90 exist through pattern matching using this information.

[0085] In step 108, the CPU 11 detects an image representing the original document 80 (hereinafter referred to as the "original document image") from the reference image. In this embodiment, the detection of the original document image is performed by pre-registering information on the outline shape and dimensions for each type of original document 80 supported by the image processing device 10, and then detecting the area that matches the registered outline shape and dimensions. However, this is not the only method. For example, in addition to the outline shape and dimensions, the background color of the original document 80 may also be used to detect the original document image.

[0086] In step 110, the CPU 11 determines whether the condition that the marker 90 is placed on the surface of the document 80 is met. If the determination is positive, the process proceeds to step 112.

[0087] In step 112, the CPU 11 controls the display unit 15 to display a preview screen with a predetermined configuration (or update the preview screen if one is already displayed). Figure 8 shows an example of a preview screen according to this embodiment.

[0088] As shown in Figure 8, in the preview screen according to this embodiment, the original image 15A included in the reference image is displayed along with the names individually assigned to each image of the marker 90, such as Marker A, Marker B, etc. In addition, in the preview screen according to this embodiment, an image (hereinafter referred to as the "table image") 15B is displayed in table format, showing each of the detected markers 90 and the corresponding processing content and setting values. Furthermore, in the preview screen according to this embodiment, an operation mode indication message 15D indicating the operation mode at this time (the first operation mode described above) is displayed. Note that the status indication message 15C shown in Figure 8 is not displayed in the preview screen at this point.

[0089] Here, the CPU 11 identifies the marker ID represented by the machine-readable code of the marker 90 detected in the most recent step 106, and obtains information on the marker type and processing content corresponding to the identified marker ID from the marker-related information database 13B. For sets of four markers 90 whose marker type is designated for area specification, the CPU 11 uses the placement position of each marker to identify the processing target area as described above.

[0090] In step 114, the CPU 11 registers the identified processing target area as a setting value in the corresponding storage area of ​​the processing-related information database 13C.

[0091] In step 116, the CPU 11 registers information indicating the processing details of all markers 90 obtained by the processing in step 112 into the corresponding memory area of ​​the processing-related information database 13C. At this time, for the markers 90 used to specify areas in sets of four, the CPU 11 registers the processing details in association with the setting value indicating the corresponding processing target area, which was registered in the processing in step 112.

[0092] In step 118, the CPU 11 updates the preview screen displayed on the display unit 15 to include a status notification message 15C, as shown in Figure 8 as an example, and then proceeds to step 138. As shown in Figure 8, the status notification message 15C in this embodiment displays a message indicating that the contents shown in the table image 15B have been registered in the processing-related information database 13C. The status notification message 15C in this embodiment also displays a message indicating that the marker 90 should be moved away from the document 80. Therefore, by referring to the status notification message 15C, the user can understand that information regarding the marker 90 they have placed has been registered, and will move the marker 90 away from the document glass 30.

[0093] On the other hand, if the result in step 110 is negative, the process proceeds to step 120. In step 120, the CPU 11 determines whether the conditions that the marker 90 is not placed on the surface of the document 80 and that a setting value indicating the processing target area is registered are met. If the result is positive, the process proceeds to step 122.

[0094] In step 122, the CPU 11 controls the display unit 15 to display a preview screen with a predetermined configuration (or update the preview screen if one is already displayed). Figure 9 shows an example of the preview screen according to this embodiment at this point.

[0095] As shown in Figure 9, the preview screen at this point displays the original image 15A of the original document 80 included in the reference image. The preview screen also displays a message 15E indicating that processing will be performed using the registered content, as well as an operation mode indication message 15D showing the current operation mode (the second operation mode described above).

[0096] In step 124, the CPU 11 reads all processing details and settings from the processing-related information database 13C, and in step 126, it executes processing according to the read processing details and settings.

[0097] As a result of the processing in step 126, for documents 80 in which information regarding the processing indicated by the marker 90 has been registered while the marker 90 is placed on the surface, the processing will be executed using the registered processing information while the marker 90 is separated from the document. Furthermore, as a result of the processing in step 126, the processing will also be executed using the registered processing information for documents 80 placed on the document feeder 30 without the marker 90 in place.

[0098] On the other hand, if the result in step 120 is negative, the process proceeds to step 128. In step 128, the CPU 11 determines whether the conditions that the marker 90 is not placed on the surface of the document 80 and that no setting value indicating the processing target area is registered are met. If the result is negative, the process proceeds to step 138; if the result is positive, the process proceeds to step 130.

[0099] In step 130, the CPU 11 controls the display unit 15 to display a preview screen with a predetermined configuration (or update the preview screen if one is already displayed). Figure 10 shows an example of the preview screen according to this embodiment at this point.

[0100] As shown in Figure 10, the preview screen at this point displays the original image 15A of the original document 80 included in the reference image. Also, since the processing content has not been registered, the preview screen at this point displays a message 15F indicating that only the original image should be registered and prompting the user to place a marker 90 indicating the processing content. Furthermore, the preview screen according to this embodiment displays an operation mode indication message 15D indicating the operation mode at this point (in this embodiment, the third operation mode).

[0101] In step 132, the CPU 11 registers the predetermined image region from the image information indicating the reference image in the storage unit 13.

[0102] On the other hand, upon referring to message 15F, the user understands that only the registration of the original document image of the document 80 that they placed will be performed, and then places the corresponding marker 90 to instruct the process to be executed for that document 80.

[0103] Therefore, in step 134, the CPU 11 waits, similar to the process in step 104 described above, until all movement of the markers 90, which is caused by the placement of the markers 90 by the user, comes to a standstill.

[0104] In step 136, the CPU 11 reads the image information registered in step 132 from the storage unit 13, executes the process specified by the marker 90 placed by the user on the image information, and then proceeds to step 138. When executing the process in step 136, the CPU 11 identifies the content and setting values ​​of the process specified by the marker 90 placed by the user, similar to the process in step 112 described above.

[0105] In step 138, the CPU 11 determines whether a predetermined termination timing has arrived to end the information processing. If the determination is negative, it returns to step 102; if the determination is positive, it proceeds to step 140. In this embodiment, the termination timing is set to the timing when the user uses the marker 90 to indicate that there is no next document 80, but it is not limited to this. For example, the termination timing may be set to the timing when the user gives an instruction to terminate the information processing via the UI unit 40.

[0106] In step 140, the CPU 11 sends an instruction to the document camera 70 to stop capturing video, and then terminates this information processing. In response to this instruction, the document camera 70 stops capturing video.

[0107] Next, with reference to Figures 11 to 15, specific examples of situations that the information processing according to this embodiment can handle will be described. First, with reference to Figures 11 to 13, the first example will be described. Figure 11 shows an example where text is hidden by a marker, Figure 12 shows an example of an expected processing state, and Figure 13 shows an example where text is hidden by a marker.

[0108] As shown in Figure 11, the first example assumes that the document 80 is a contract. Also, as shown in Figure 12, the first example assumes that the processing involves converting the date written in the contract into text data using OCR processing and using that date as the file name, while attaching a high-resolution scanned image 83 of the identification document 82 as an attachment to the image 81 of the document 80.

[0109] In this case, in order for marker 90 to be read correctly, marker 90 needs to be prepared to a certain size. As shown in Figure 11, in a format where the identification document 82 is placed on top of the original document 80 and scanned, if marker 90 is placed outside the identification document 82, the text written on the contract will be hidden. Note that in the example shown in Figure 11, the text is hidden by markers 90A, 90C, 90D, and 90E.

[0110] Furthermore, if there is an area to be processed using OCR near the marker 90 placed outside the identification document 82, it becomes impossible to properly position the marker 90 for OCR processing. In the example shown in Figure 11, the marker 90C is not properly positioned.

[0111] On the other hand, since identification documents 82 are often card-sized, placing a marker 90 on top of identification documents 82 can obscure the information to be scanned, as shown in Figure 13 as an example. In the upper example of Figure 13, marker 90B is the marker 90 obscuring the information to be scanned, while in the lower example of Figure 13, marker 90A is the marker 90 obscuring the information to be scanned.

[0112] Therefore, with conventional technology, the placement and scanning of the marker 90 had to be performed multiple times for each instruction given by the marker 90.

[0113] In contrast, the information processing according to this embodiment adds a process for registering instructions from the marker 90, and makes it possible to register them in multiple steps. This allows processing and transfer processes to be performed according to the registered instructions from the marker 90 when a scan is performed without a marker 90 in place.

[0114] Next, we will explain the second case with reference to Figures 14 and 15. Figure 14 shows an example of the placement of the original document and the marker, and Figure 15 shows an example where the text is hidden by the marker. In the second case, we will explain a case where the content of the processing can be identified by the visual characteristics (in this case, color) of the marker 90. However, in the examples shown in Figures 14 and 15, for convenience, the color of the marker 90 is represented by shades of grayscale.

[0115] As shown in Figure 14, in the second example, we assume a quotation document with a relatively small font size, referred to as document 80. Furthermore, in the second example, we assume that the list of items quoted in the above quotation document is scanned, converted into a PDF document, and then the quotation date, quotation number, and supplier company name are set as the file name before being sent to the purchasing manager via email.

[0116] When performing OCR processing on the document 80, if the document 80 is placed too far from the document camera 70, the characters will not be readable. Therefore, as shown in Figure 14 as an example, the document 80 must be placed close enough for OCR processing to be possible.

[0117] In this case, as shown in Figure 15 as an example, if the area to be read is specified with marker 90, depending on the size of the area to be specified, marker 90 may be placed on top of the text, thus obscuring the information to be read and making it impossible to read that information. Note that in the example shown in Figure 15, the text is hidden by markers 90A and 90D.

[0118] With conventional technology, if scanning is performed without placing the marker 90, it becomes impossible to identify the reading position (processing area) because there is no indication of the marker 90.

[0119] In contrast, the information processing according to this embodiment does not read the marker 90 and the document 80 simultaneously, but adds a process to automatically detect when the marker 90 is placed and register the instructions from the marker 90. Therefore, if scanning is performed when the marker 90 is not placed, processing and transfer processes can be performed according to the registered instructions from the marker 90.

[0120] In the above embodiments, the present invention has been described as being applied to an image processing device 10 in which the UI unit 40 is located on the front side of the device, as shown in Figure 1, as an example, but it is not limited to this. For example, the present invention may also be applied to an image processing device 10 in which the UI unit 40 is located on the rear side of the device (in the example shown in Figure 10, the arm 72 of the document camera 70), as shown in Figure 16, as an example.

[0121] Furthermore, although the above embodiment described a case in which information about all applied markers 90 is displayed in the table image 15B on the preview screen, it is not limited to this. For example, the user may choose to display only the information corresponding to the markers 90 specified for the displayed original image 15A as the table image 15B.

[0122] Furthermore, although the type of marker was not specifically mentioned in the above embodiment, various types such as magnetic type, marble type, sticky note type, and sticker type may be applied.

[0123] Furthermore, although not mentioned in the above embodiment, for example, characters indicating the processing content may be written on the marker 90.

[0124] Furthermore, although the above embodiment described a case in which the marker 90 includes only information indicating the content of the process, it is not limited to this. For example, in addition to information indicating the content of the process, the marker 90 may also include a set value.

[0125] Furthermore, although the above embodiment describes a case in which the first operation mode is terminated when conditions different from the conditions for starting the first operation mode are met, it is not limited to this. For example, the first operation mode may be terminated when there are multiple markers 90 and the detection of the last marker 90 is completed.

[0126] Furthermore, although the above embodiment describes a case in which various databases are registered in the image processing device 10, it is not limited to this. For example, various databases may be registered in an external device such as a server device configured to communicate with the image processing device 10, and the system may access the external device as appropriate.

[0127] Although embodiments have been described above, the technical scope of the present invention is not limited to the embodiments described above. Various modifications or improvements can be made to the above embodiments without departing from the spirit of the invention, and such modified or improved forms are also included in the technical scope of the present invention.

[0128] Furthermore, the above embodiments do not limit the invention claimed, and not all combinations of features described in the embodiments are necessarily essential for solving the problem of the invention. The embodiments described above include inventions at various stages, and various inventions can be extracted by combining the multiple constituent elements disclosed. Even if some constituent elements are deleted from all the constituent elements shown in the embodiments, the configuration with these deleted elements can be extracted as an invention, as long as the effect is obtained.

[0129] Furthermore, in the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) and dedicated processors (e.g., GPUs: Graphics Processing Units, ASICs: Application Specific Integrated Circuits, FPGAs: Field Programmable Gate Arrays, programmable logic devices, etc.).

[0130] Furthermore, although the above embodiment describes a case in which information processing is realized by a software configuration using a computer by executing a program, the present invention is not limited thereto. For example, information processing may also be realized by a hardware configuration or a combination of a hardware configuration and a software configuration. Also, although the document camera 70 was described as being configured to capture the upper surface of the document table 30, it is not limited to this, and the present invention may be configured to perform the processing of the invention using an image obtained by capturing an image of the document with a camera equipped on a smartphone or a laptop computer as captured data.

[0131] Furthermore, the configuration of the image processing apparatus 10 described in the above embodiment is merely an example, and it goes without saying that unnecessary parts may be deleted or new parts added without departing from the spirit of the present invention.

[0132] Furthermore, the information processing flow described in the above embodiment is merely an example, and it goes without saying that unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not depart from the spirit of the present invention.

[0133] (Note) (((1))) Equipped with a processor, The aforementioned processor, When identifying the area to be processed in a document using the position of a marker placed on the surface of the document, If the marker is placed on the surface of the original document, the processing area is identified using the position of the marker, and area information indicating the processing area is registered. If the marker is not placed on the surface of the document and the area information is registered, the processing area indicated by the area information is subjected to a predetermined process. If the marker is not placed on the surface of the document and the area information has not been registered, a predetermined area on the surface of the document is registered, and then, when the area information is registered, the area to be processed indicated by the area information is identified as the area to be processed. Information processing device. (((2))) The aforementioned processor, When the original document and the marker are stationary, the presence or absence of the marker on the surface of the original document is determined. The information processing device described in (((1))). (((3))) The aforementioned processor, Using the captured image of the area where the document is to be placed, it is determined whether the document and the marker are stationary. The information processing device described in (((2))). (((4))) The aforementioned processor, The system displays operation mode information indicating which of the following operation modes is being used: a first operation mode for registering the aforementioned area information, and a second operation mode for executing the aforementioned processing. An information processing device as described in any one of items (((1))) to (((3))). (((5))) The aforementioned processor, When the first operating mode is completed, information indicating that the marker should be separated from the document is displayed. The information processing device described in (((4))). (((6))) The marker is such that the content of the process can be identified. An information processing device as described in any one of items (((1))) through (((5))). (((7))) The marker is such that the content of the process can be identified by the external characteristics of the marker. The information processing device described in (((6))). (((8))) The aforementioned external features are at least one of color, shade, and shape. The information processing device described in (((7))). (((9))) The aforementioned processor, If there are multiple markers and the detection of the last marker is completed, the operation mode for registering the region information is terminated. An information processing device as described in any one of items (((1))) through (((8))). (((10))) The aforementioned process is a process using a document camera. An information processing device as described in any one of items (((1))) through (((9))). (((11))) The processing using the aforementioned document camera is a process related to image reading. The information processing device described in (((10))).

[0134] According to the information processing device described in (((1))), when specifying a processing area on a document using a marker, it is possible to prevent the loss of information located below the marker. According to the information processing device described in (((2))), the procedure for determining whether or not a marker is placed on the surface of the original document can be simplified. According to the information processing device described in (((3))), when processing is performed using a camera such as a document camera, the camera can be used to identify a stationary state. According to the information processing device related to (((4))), the user can be made aware of the operating mode. According to the information processing device related to (((5))), the timing of the marker's separation can be instructed to the user. According to the information processing device related to (((6))), processing for the processing target area can be set by a marker. According to the information processing device related to (((7))), the content of the processing can be intuitively grasped. According to the information processing device related to (((8))), the content of the processing can be grasped using the applied features. According to the information processing device related to (((9))), the operation mode for registering area information can be automatically terminated. According to the information processing device related to (((10))), it is possible to prevent the loss of information located below the marker during processing by a document camera. According to the information processing device related to (((11))), it is possible to prevent the loss of information located below the marker in the image reading process using a document camera. [Explanation of Symbols]

[0135] 10 Image Processing Device 11 CPU 11A Registration Section 11B Execution Unit 11C Specific part 11D Presentation section 12 memory 13 Storage section 13A Information Processing Program 13B Marker-related information database 13C Processing-related information database 14 Input section 15 Display section 15A Original image 15B Table Image 15C Situational Message 15D Operating Mode Indication Message 15E Message 15F Message 16. Media reading / writing device 17 Recording media 18 Communication I / F Section 30 Manuscript Stand 32 Original placement area 40 UI section 50 trays 60 Paper feed section 70 Document Camera 72 Arms 80 Manuscripts 90 Markers

Claims

1. Equipped with a processor, The aforementioned processor, When identifying the area to be processed in a document using the position of a marker placed on the surface of the document, If the marker is placed on the surface of the original document, the processing area is identified using the position of the marker, and area information indicating the processing area is registered. If the marker is not placed on the surface of the document and the area information is registered, the processing area indicated by the area information is subjected to a predetermined process. If the marker is not placed on the surface of the document and the area information has not been registered, a predetermined area on the surface of the document is registered, and then, when the area information is registered, the area to be processed indicated by the area information is identified as the area to be processed. Information processing device.

2. The aforementioned processor, When the original document and the marker are stationary, the presence or absence of the marker on the surface of the original document is determined. The information processing apparatus according to claim 1.

3. The aforementioned processor, Using the captured image of the area where the document is to be placed, it is determined whether the document and the marker are stationary. The information processing apparatus according to claim 2.

4. The aforementioned processor, The system provides operation mode information indicating which of the following operation modes is being used: a first operation mode for registering the aforementioned area information, and a second operation mode for executing the aforementioned processing. The information processing apparatus according to claim 1.

5. The aforementioned processor, When the first operating mode is completed, information indicating that the marker should be separated from the document is displayed. The information processing apparatus according to claim 4.

6. The marker is such that the content of the process can be identified. The information processing apparatus according to claim 1.

7. The marker is such that the content of the process can be identified by the external characteristics of the marker. The information processing apparatus according to claim 6.

8. The aforementioned external features are at least one of color, shade, and shape. The information processing apparatus according to claim 7.

9. The aforementioned processor, If there are multiple markers and the detection of the last marker is completed, the operation mode for registering the region information is terminated. The information processing apparatus according to claim 1.

10. The aforementioned process is a process using a document camera. The information processing apparatus according to claim 1.

11. The processing using the aforementioned document camera is a process related to image reading. The information processing apparatus according to claim 10.

12. When identifying the area to be processed in a document using the position of a marker placed on the surface of the document, If the marker is placed on the surface of the original document, the processing area is identified using the position of the marker, and area information indicating the processing area is registered. If the marker is not placed on the surface of the document and the area information is registered, the processing area indicated by the area information is subjected to a predetermined process. If the marker is not placed on the surface of the document and the area information has not been registered, a predetermined area on the surface of the document is registered, and then, when the area information is registered, the area to be processed indicated by the area information is identified as the area to be processed. An information processing program that causes a computer to perform a task.

Images