Information processing apparatus, information processing method, and non-transitory storage medium

Abstract

The present application provides an information processing device, an information processing method, and a non-transitory storage medium. A predetermined image correction process is automatically performed on an image, and information identifying that the predetermined image correction process has been performed is displayed while the image that has undergone the predetermined image correction process is being displayed.

Description

[0001] This application is a divisional application of the invention patent application filed on May 22, 2018, with application number 201810496402.3 and entitled "Information Processing Apparatus, Information Processing Method and Non-transitory Storage Medium". Technical Field

[0002] This disclosure relates to an information processing apparatus, an information processing method, and a non-transitory storage medium for performing image correction processing. Background Technology

[0003] Mobile terminals with advanced information processing capabilities, such as smartphones and tablet PCs, are widely popular. Each of these mobile terminals may have a camera for image capture, and the image capture function of this camera can be used to capture images of the subject. Some applications using such mobile terminals can capture images of the subject using the image capture function and then perform image correction processing on the captured images.

[0004] Japanese Patent Application Publication No. 2016-143966 discloses a mobile terminal that identifies whether a subject in an image is a document or a whiteboard and performs image correction processing on the identified subject (document or whiteboard).

[0005] Recent mobile applications can activate the camera on a mobile device and display a live video image of the subject being captured by the camera (live view video image) on the device's display until the shutter button is pressed. The user can press the shutter button while simultaneously viewing the live view video. The application then acquires the image of the subject and receives the user's selection of image correction processing. The application then applies the user-selected image correction processing to the acquired image and displays the image after correction.

[0006] Here, to reduce the complexity for users in selecting image correction processing, a predetermined image correction process (or scheduled image correction process) can be performed on the obtained image. In this case, it is usually unknown from the displayed image whether the scheduled image correction process has been performed. Summary of the Invention

[0007] According to one aspect of this disclosure, an information processing apparatus includes: an acquisition unit configured to acquire an image in response to a command from a user; an image correction unit configured to perform image correction processing on the acquired image; and a display unit configured to display the image after image correction processing by the image correction unit. The image correction unit performs a predetermined image correction process on the image acquired by the acquisition unit in response to the command. While displaying the image after the predetermined image correction process, the display unit displays information identifying the predetermined image correction process performed.

[0008] Other features of the invention will become clear from the following description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0009] Figure 1A and Figure 1B An example of the appearance of a mobile terminal is shown.

[0010] Figure 2 This is a hardware block diagram illustrating a mobile terminal.

[0011] Figure 3 This is a flowchart illustrating the process according to an embodiment.

[0012] Figure 4 This is a flowchart illustrating the process according to an embodiment.

[0013] Figure 5A and Figure 5B This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0014] Figure 6 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0015] Figure 7 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0016] Figure 8 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0017] Figure 9 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0018] Figure 10 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0019] Figure 11 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0020] Figure 12 This is a schematic diagram illustrating the screen displayed on a mobile terminal.

[0021] Figure 13 This is a schematic diagram illustrating the screen displayed on a mobile terminal. Detailed Implementation

[0022] Embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0023] First Embodiment

[0024] Figure 1A and Figure 1B An example of the appearance of a mobile terminal (e.g., an information processing device or an image processing device) 101 according to the first embodiment is shown. Figure 1A The front of the mobile terminal 101 is shown. The front has a touch panel display 102 and operation buttons 103. Figure 1B The back of mobile terminal 101 is illustrated. A camera 104 is located on the back. Mobile terminal 101 uses camera 104 to provide image capture functionality. This embodiment can use any terminal device with image capture functionality. In other words, this embodiment can use a smartphone or cellular phone with image capture functionality or a digital camera with communication functionality. Camera 104 has an autofocus mechanism (not illustrated) and is configured to automatically detect focus status based on contrast ratio and infrared transmission / reception time, allowing for the measurement of focal length or subject distance.

[0025] Figure 2 This is a hardware block diagram illustrating mobile terminal 101. However, the block diagram is only given to illustrate an example of the construction used to implement this embodiment.

[0026] Reference Figure 2 The CPU 201, RAM 202, and ROM 203 send and receive programs and data via the data bus 211. The storage unit 204, data transmission / reception unit 205, camera unit 206, display unit 207, operation unit 208, and motion sensor 210 are connected to the data bus 211. Furthermore, these components, as well as the CPU 201, RAM 202, and ROM 203, can send and receive programs and data from each other.

[0027] Storage unit 204 may be a flash memory and is configured to store image data and for performing operations. Figure 3 The procedure shown is for processing.

[0028] The data transmission / reception unit 205 may have a wireless LAN controller and is configured to transmit / receive data with external devices.

[0029] The imaging unit 206 may be a camera or an image sensor, and is configured to acquire an image of a subject, such as a document or a whiteboard, by capturing an image of the subject. The acquired image (captured image) data, along with header information including the manufacturer's name and model name of the mobile terminal, image resolution, aperture (F-number), and focal length, may be provided and then sent to the appropriate unit.

[0030] The display unit 207 may be a touch panel display and is configured to use camera functions to display a live view video image of the subject and a shutter button for image capture.

[0031] The operation unit 208 may be a touch panel or an operation button, and is configured to receive user operations and send information about the operation to the corresponding unit.

[0032] The motion sensor 210 may include a triaxial accelerometer, an electronic compass, and a triaxial angular velocity sensor, and is configured to detect the orientation or movement of the mobile terminal 101 by utilizing known techniques.

[0033] The CPU 201 can execute programs stored in the ROM 203 or the storage unit 204 to control components within the mobile terminal 101.

[0034] This embodiment is applicable to any terminal device having a display unit and an operation unit. In other words, this embodiment is applicable to smartphones and tablet terminals with touch panel displays, as well as PCs with mice and touch panels.

[0035] will describe Figure 5A and Figures 5B to 13 The user interface (UI) screen shown is displayed. These UI screens can be displayed by an application that is stored in storage unit 204 and executed by CPU 201.

[0036] Next, we will describe Figure 5A and Figure 5B . Figure 5A An example is shown of a screen 501 to be displayed on the display unit 207 when the CPU 201 executes an application stored in the storage unit 204. Screen 501 is the entire top screen of the application to be displayed on the display unit 207 of the mobile terminal.

[0037] Display field 502 displays the name of the application (“Printing Application”) installed and executed by CPU 201.

[0038] Icon 503 indicates the name of the printer connected to the mobile terminal (“Printer XYZ”).

[0039] The shooting settings button 504 is a button that the user can use to set the type of image correction processing performed first on the captured image. When the user presses button 504, the display on display unit 207 changes from... Figure 5A The displayed screen changes to an unillustrated screen. On this screen, the user can select an image correction mode to apply to the captured image from a variety of image correction modes (image correction processing) described below. For example, when the document image correction processing mode is selected and set, the image of the subject captured by the user is first subjected to document image correction processing, and then the result is displayed. According to this embodiment, button 504 is configured to allow setting the image correction mode. However, a fixed image correction mode can always be set without button 504. In other words, the document image correction processing mode can be fixed as the default image correction processing that is automatically performed on the captured image first. Alternatively, without button 504, the last image correction processing can be set to be performed. A "mode without image correction processing" can be included in the settable image correction modes. When the mode is set, no image correction processing is performed on the captured image. The following description assumes that the mode is set to perform document image correction processing on the captured image.

[0040] The camera icon 505 is the icon that a user presses to use the image capture function of the camera 104 on the mobile terminal 101. When this icon is pressed, the CPU 201 activates the camera 104 of the mobile terminal and switches it to the image capture enabled state. Figure 5B ). Starting the camera 104 includes calling and executing a control application for the camera 104 included in the mobile terminal 101.

[0041] Figure 5B An example is shown of a screen 509 that the CPU 201 will display on the display unit 207 to enable the capture of an image of the subject. Screen 509 includes (or displays) a display area 510, a shutter button 511, and an image recall icon 512, but does not include (or displays) an icon or name for a preset image correction mode (e.g., an icon or text string indicating "Document"). In other words, while displaying a live view video image from which the user can receive image capture instructions, information for identifying the type of image correction processing is not displayed, thereby preventing visually unclear display of the live view video image of the subject.

[0042] Display area 510 is an area for displaying a live view video image of the subject used for live view image capture. The live view video image is a moving image to be displayed under the control of CPU 201, and can be, for example, a live video image of the subject captured by camera 104. The user can press the shutter button while simultaneously viewing the displayed live view video image. CPU 201 is configured to capture an image of the subject in response to detecting a user pressing the shutter button while the live view video image is being displayed. In other words, CPU 201 maintains the display of the live view video image of the subject on display area 510 until the user presses the shutter button.

[0043] As described above, the shutter button 511 serves as a unit for receiving instructions from the user to capture an image of the subject while simultaneously displaying a live view video image. When the shutter button 511 is pressed, the CPU 201 uses the control application of the camera 104 to capture an image of the subject captured by the camera 104. The CPU 201 changes the display content on the display unit by setting the captured image as the target for image correction processing. Figure 6 The image shown will be referenced. Figure 6 The following figures illustrate this. Instead of immediately switching to the position shown after pressing the shutter button 511... Figure 6 The captured image can be displayed once on display area 510, and in response to user confirmation, the image displayed on display unit 207 can be changed to... Figure 6 Scene 601.

[0044] If the user presses icon 512, the CPU 201 displays a list of images stored in storage unit 204 on the display unit, and changes the display to resemble the image selected by the user from the list as the target of image correction processing. Figure 6 The image shown will be referenced. Figure 6 The description is accompanied by accompanying figures. Instead of immediately switching to [the desired view / platform] after selecting an image from the list... Figure 6 In screen 601, the selected image can be displayed once as a still image on display area 510, and in response to user confirmation, the screen can be switched to... Figure 6 The following description assumes that the image of the subject being photographed in response to an image capture command is set as the target of image correction processing.

[0045] Next, we will describe Figure 6 . Figure 6An example is shown of a screen 601 that the CPU 201 will display on the display unit 207. Screen 601 is used to receive user operations for performing keystone correction processing. On screen 601, the area to be subjected to keystone correction processing is specified. According to this embodiment, keystone correction processing is a process for geometrically correcting an image within a specified area of ​​a captured image to a rectangle or square. This image cropping process can also be referred to as a cropping process.

[0046] Figure 6 Display field 602 is also shown for displaying the name (“keystone correction”) of the screen displayed on display unit 207.

[0047] Figure 6 A display area 603 for displaying a captured image of the subject is also illustrated. Display area 603 displays a cropping frame 604 and a handler for transforming the cropping frame 604, which will be described below.

[0048] The clipping frame 604 displays the clipping processing area for trapezoidal correction processing. The processing procedures for transforming the frame 604 are set at the four corners of the frame 604 and the midpoints of each boundary of the frame 604, and will be operated by the user.

[0049] Icon 606 is a selection icon that can be used to indicate the aspect ratio of an image that has undergone keystone correction. Figure 6 This example shows the "Auto" setting. The aspect ratio can be selected from "Auto / A4 / LTR".

[0050] Icon 607 is a selection icon used to indicate moving to the next processing step. When the user presses icon 607, the displayed screen changes to... Figure 7 The image shown is 701. For the image transition, the CPU 201 performs trapezoidal correction processing on the image in the area specified in the cropping frame 604, and then performs image correction processing on the trapezoidally corrected image based on a preset image correction mode. According to this embodiment, document image correction processing is performed as described above. Document image correction processing removes shadows shown in the image to increase the image's brightness. Then, the CPU 201... Figure 7 The corrected image is displayed on screen 701.

[0051] will describe Figure 7 . Figure 7Screen 701 is a screen displayed on display unit 207 via CPU 201. Screen 701 is a screen that the user can use to confirm and receive user operations to change the type of image correction processing before the image after image correction processing is saved. Screen 701 displays the result of image correction processing based on a preset image correction mode for the image after keystone correction processing, and the name of the image correction mode (or information used to identify the type of image correction processing).

[0052] Figure 7 Display field 702 is shown as an example of displaying the name (“Image Correction Mode”) of the screen displayed on display unit 207.

[0053] Figure 7 It also includes a display area 703, which displays the result of image correction processing based on a preset image correction mode. Here, display area 703 displays the result of the document image correction processing performed.

[0054] Figure 7 Icon 704 is illustrated; it is a selection icon used to indicate the rotation (90 degrees / 180 degrees / 270 degrees) of the image displayed in the display area 703 on the display unit 207. When the user presses the selection icon, rotation angle options (90 degrees / 180 degrees / 270 degrees) are displayed (not illustrated). When the user selects any rotation angle from these options, the CPU 201 performs rotation processing for the image 703 displayed on the display unit 207 according to the specified rotation angle. Afterward, the CPU 201 stores the rotated image in the storage unit 204 and displays it on the display unit 207.

[0055] Figure 7 An icon 705 is also shown, indicating the name of the image correction process to be applied to the image displayed in display area 703. Icon 705 displays identification information for identifying the type of image correction process and the image after image correction. Here, "Document" is displayed. This refers to the live view video image of the subject being displayed. Figure 5B Simultaneously, the CPU 201 displays both the undisplayed identification information and the image after image correction processing, allowing users to easily identify the type of image correction processing performed. As described in this embodiment, by providing the name (text string) of the image correction processing type as identification information, an icon representing the type of image correction processing can be displayed. When the user presses icon 705, the CPU 201 further displays a list 804 of image correction processing on the display unit 207, such as... Figure 8 As shown.

[0056] Icon 706 is a selection icon used to indicate moving to save (the next processing step). When the user presses icon 706, CPU 201 converts the image displayed in display area 703 into a PDF file format and displays it on display unit 207. Figure 12 Screen 1201 shows a preview of the PDF file.

[0057] Next, we will describe Figure 8 . Figure 8 An example is shown of a screen 801 that the CPU 201 wants to display on the display unit 207. Screen 801 is a screen used to receive a selection of an image correction mode from the user and provide options to the user.

[0058] List 804 is a list of image correction processing options.

[0059] Option 806 indicates that the type of image correction processing to be performed on the keystone-corrected image is document image correction processing. When the user presses option 806, the CPU 201 performs document image correction processing on the keystone-corrected image and displays the result. Figure 7 The image shown is 701.

[0060] Option 807 indicates that the type of image correction processing to be performed on the keystone-corrected image is whiteboard image processing. When the user presses option 807, CPU 201 removes the shadows shown on the keystone-corrected image to increase the image brightness, and performs image correction processing to display the cluster parts with similar colors in the image with increased brightness in monochrome. Then, CPU 201 displays... Figure 9 The image shown is 901, which includes an image that has undergone image correction processing.

[0061] Option 808 further indicates that the type of image correction processing to be performed on the keystone-corrected image is whiteboard image processing. Options 808 and 807 differ in the size of the cluster portions to be displayed in monochrome. When the user presses option 808, the CPU 201 removes the shadows shown on the keystone-corrected image to increase the image brightness and performs image correction processing to display the cluster portions with similar colors in monochrome in the image with increased brightness. To detect the cluster portions, portions with a size larger than those detected based on option 807 are detected. The CPU 201 then displays as shown... Figure 10 The image shown is 1001, which includes an image that has undergone image correction processing.

[0062] Option 809 can be selected to leave the keystone-corrected image without any image correction processing. When the user presses option 809, the CPU 201 displays the image as is, including the keystone-corrected image. Figure 11 The image shown is 1101.

[0063] Option 810 can be selected to cancel the change in the type of image correction processing. When the user presses option 810, CPU 201 cancels the additional display of list 804 and displays the screen that was displayed when list 804 was invoked. For example, in the display Figure 7 When the icon 705 is pressed to call up the list 804 on screen 701, the list is displayed again in response to the pressing of option 810. Figure 7 Scene 701.

[0064] Next, we will describe Figure 9 . Figure 9 An example of a screen 901 to be displayed on display unit 207 by CPU 201 is shown. Screen 901 can be used by the user for confirmation before the image after image correction processing is saved, and can also be used to receive user operations for changing the type of image correction processing. Screen 901 displays the result of image correction processing performed on the image after keystone correction processing, based on the image correction mode of option 807, and the name of the image correction mode (or information for identifying the type of image correction processing).

[0065] Figure 9 Display field 902, which is similar to display field 702, is also shown.

[0066] Figure 9 Also illustrated is a display area 903 showing the results of image correction processing based on the image correction mode of option 807. (See reference...) Figure 9 Display area 903 displays the result of the first image correction process for the whiteboard.

[0067] Figure 9 The icon 904 is similar to the icon 704.

[0068] Figure 9 Also shown is icon 905, which is similar to icon 705, displaying the name of the image correction processing applied to the image displayed on display area 903. (See also...) Figure 9 Icon 905 displays "Whiteboard 1," indicating the first image correction process for the whiteboard. When the user presses icon 905, it corresponds to... Figure 8 Similar to the screen in the image, the CPU 201 additionally displays a list 804 of image correction processing on the display unit 207.

[0069] Figure 9An icon 906 is also illustrated as a selection icon that can be used to indicate saving (the next processing step). When the user presses icon 906, the CPU 201 converts the format of the image displayed in the display area 903 into a PDF file, and displays it on the display unit 207 as shown. Figure 12 Screen 1201 shows a preview of the PDF file.

[0070] Next, we will describe Figure 10 . Figure 10 An example is shown of a screen 1001 that the CPU 201 will display on the display unit 207. Screen 1001 can be used by the user for confirmation before the image after image correction processing is saved, and can also be used to receive user operations to change the type of image correction processing. Screen 1001 displays the result of image correction processing performed on the image after keystone correction processing based on the image correction mode of option 808 and the name of the image correction mode (or information for identifying the type of image correction processing).

[0071] Figure 10 Display field 1002, which is similar to display field 702, is also shown.

[0072] Figure 10 The display area 1003, which shows the results of image correction processing based on the image correction mode of option 808, is also illustrated. (See reference...) Figure 10 Display area 1003 displays the result of the second image correction process on the whiteboard.

[0073] Figure 10 The icon 1004 is similar to the icon 704.

[0074] Figure 10 Also shown is icon 1005, which is similar to icon 705, displaying the name of the image correction process applied to the image displayed on display area 1003. (See also...) Figure 10 Icon 1005 displays "Whiteboard 2," indicating the second image correction process for the whiteboard. When the user presses icon 1005, it interacts with... Figure 8 Similar to the screen in the image, the CPU 201 additionally displays a list 804 of image correction processing on the display unit 207.

[0075] Figure 10 An icon 1006 is also illustrated as a selection icon that can be used to indicate saving (the next processing step). When the user presses icon 1006, the CPU 201 converts the format of the image displayed in the display area 1003 into a PDF file, and displays it on the display unit 207 as shown. Figure 12 Screen 1201 shows a preview of the PDF file.

[0076] Next, we will describe Figure 11 . Figure 11 An example is shown of a screen 1101 that the CPU 201 intends to display on the display unit 207. The screen 1101 can be used by a user for confirmation before the image, after image correction processing, is saved, and can also be used to receive user operations for changing the type of image correction processing.

[0077] Figure 11 Display field 1102, which is similar to display field 702, is also shown.

[0078] Figure 11 The display area 1103, which shows an image based on the image correction mode of option 809, is also illustrated. (See reference...) Figure 11 Display area 1103 displays only the image after keystone correction, without any image correction processing corresponding to options 806 to 808.

[0079] Figure 11 The icon 1104 is similar to the icon 704.

[0080] Figure 11 An icon 1105 is also shown, indicating that image correction processing corresponding to options 806 to 808 is not performed. Therefore, icon 1105 displays "No Image Correction". When the user presses icon 1105, [the text abruptly ends here]. Figure 8 Similar to the screen in the image, the CPU 201 additionally displays a list 804 of image correction processing on the display unit 207.

[0081] Figure 11 An icon 1106 is also illustrated as a selection icon that can be used to indicate saving (the next processing step). When the user presses icon 1106, the CPU 201 converts the format of the image displayed in the display area 1103 into a PDF file, and displays it on the display unit 207 as shown. Figure 12 Screen 1201 shows a preview of the PDF file.

[0082] Next, we will describe Figure 12 . Figure 12 An example of a screen 1201 to be displayed on display unit 207 by CPU 201 is shown. Screen 1201 can be used by the user to set the output destination of a PDF file obtained through format conversion. Screen 1201 includes (or displays) display field 1202, display area 1203, icon 1204, icon 1205, and icon 1206.

[0083] Display field 1202 shows the name of the PDF file to be output ("20170329150834.pdf").

[0084] Display area 1203 displays an output preview of the PDF file. CPU 201 generates an output preview of the PDF file based on the response of the user pressing icons 706, 906, 1006, or 1106. The displayed content may correspond to the image displayed in display areas 703, 903, 1003, or 1103.

[0085] Figure 12 An example is shown of an icon 1204 that a user presses to set the storage destination for a PDF file. When the user presses the icon, the CPU 201 displays a screen (not shown) that the user can use to select the storage destination for the PDF file, and sets the storage destination selected by the user as the storage destination for the PDF file.

[0086] Icon 1205 can be pressed by the user to set the destination of the PDF file. When the user presses the icon, CPU 201 displays a screen where the user can select the application to use to send the PDF file (such as an email application or a Bluetooth (registered trademark) communication application). CPU 201 then uses the application selected by the user to set the destination of the PDF file and sends the PDF file to the destination.

[0087] Icon 1206 will be pressed by the user to set the printer used for printing PDF files. When the user presses this icon, CPU 201 displays it on display unit 207. Figure 13 The image shown is 1301.

[0088] Screen 1301 will be described. Icon 1301 can be operated by the user to set the printer used to print PDF files. The name (identification information) of the printer set on screen 1301 is displayed 1305. The screen includes (or displays) area 1306 and display area 1303 for printing preview of PDF files based on print settings. Area 1306 displays the print settings (or a portion thereof) for printing PDF files by the set printer (hereinafter, printer settings 1306). Screen 1301 also includes icon 1307 for receiving instructions to print PDF files based on user-defined print settings.

[0089] Display area 1303 can display a print preview of a PDF file. The print preview is generated based on the defined print settings.

[0090] Figure 13 Example 1304 shows information indicating the page number of the displayed image and the total number of pages. According to... Figure 13In the example shown, information 1304 means 1 / 1, which means that the first page is being displayed and the total number of pages is equal to 1.

[0091] Figure 13 Identification information 1305, representing the name of the printer used to print PDF files, is also illustrated. Information 1305 can be pressed by the user, and when the user presses the information icon, the CPU 201 displays a printer selection screen (not illustrated) and receives the printer selection from the user. The CPU 201 then sets the printer selected by the user as the printer used to print the PDF files.

[0092] Print settings 1306 displays the defined print settings. According to this embodiment, print settings 1306 displays the paper size (A4), number of copies (1), and print area (all). Print settings 1306 can be pressed by the user. When the user presses print settings 1306, CPU 201 displays a print settings change screen (not shown). Through this screen, the user can change the print settings.

[0093] The print command icon 1307 can be pressed by the user to print a PDF file using a configured printer. In response to a print command from the user based on pressing icon 1307, the CPU 201 sends the PDF file and defined print settings to the configured printer via the data transmission / reception unit 205. When the printer receives the PDF file and print settings, it prints the PDF file based on the print settings.

[0094] So far, it has been described Figure 5A and Figures 5B to 13 The screens shown are displayed by CPU 201 based on the application.

[0095] Next, we will refer to Figure 3 and Figure 4 The flowchart shown describes the application-based display by CPU 201. Figure 5A and Figures 5B to 13 The process of processing the image shown. Figure 3 The processing shown in the flowchart can be implemented by an application stored in storage unit 204 and executed by CPU 201. This flowchart is used for printing in a preset document image correction mode.

[0096] In step S301, CPU 201 begins the processing in the flowchart.

[0097] In step S302, the CPU 201 displays the top screen of the application on the display unit 207. Figure 5A(See screen 501 shown). When the CPU 201 detects that the user has pressed the shooting icon 505 on the screen, the process moves to step S303.

[0098] In step S303, CPU 201 activates camera 104 and displays it on display unit 207. Figure 5B The screen shown is 509. As described above, the screen displays a live view video image of the subject, but does not display information for identifying the type of image correction processing to be performed in step S306, which will be described below. When the CPU 201 detects that the user has pressed the shutter button 511 while screen 509 is being displayed, the process moves to step S304.

[0099] In step S304, the CPU 201 acquires an image of the subject captured by the camera 104. In other words, the CPU 201 acquires an image captured in response to an image capture command from the user. The captured image is temporarily stored in the storage unit 204.

[0100] Next, in step S305, the CPU 201 displays a screen on the display unit 207 for setting the target area to be subjected to keystone correction processing. Figure 6 If CPU 201 detects that the user has pressed icon 607, the process moves to the next step.

[0101] In step S306, CPU 201 performs trapezoidal correction processing on the image in the area specified in the cropping frame 604. CPU 201 stores the trapezoidal-corrected image in the trapezoidal-corrected image storage area within the storage unit 204.

[0102] Next, in step S307, the CPU 201 can select a preset (or predetermined) image correction mode from a plurality of predetermined image correction modes. Here it is assumed that the document image correction mode is selected.

[0103] Next, in step S308, the CPU 201 sets the name of the image correction mode selected in step S307. In this case, the CPU 201 stores the name of the image correction mode in the processing name storage area within the storage unit 204. Here, the CPU 201 stores "document" as the image correction processing name in the processing name storage area within the storage unit 204.

[0104] In step S309, CPU 201 reads the image that has undergone keystone correction processing and is stored in the keystone correction image storage area, and performs image correction processing on the read image corresponding to the image correction mode stored in the processing name storage area. Here, image correction processing corresponding to the document image correction mode is performed. Afterwards, CPU 201 stores the resulting image from the applied image correction processing in the correction image storage area within storage unit 204. In other words, the processing in steps S307 to S309 corresponds to automatically performing a predetermined image correction process without receiving a selection of image correction processing from the user. (Refer to...) Figure 4 The flowchart in the document describes the image correction process in detail.

[0105] In step S310, the CPU 201 displays the image that has undergone image correction processing and is stored in the corrected image storage area, and the name stored in the processing name storage area, respectively, on the corresponding display area and icon.

[0106] For example, in step S310 following step S309, CPU 201 displays on display unit 207. Figure 7 The screen shown is 701. In this case, the CPU 201 reads the image obtained from the applied image correction processing from the correction image storage area in the storage unit 204 and displays the image in the display area 703 on the screen 701. The CPU 201 reads the image correction mode name stored in the processing name storage area in the storage unit 204 and displays the name on the icon 705 on the display unit 207. This makes it easy for the user to understand the type of image correction processing performed on the image displayed in the display area 703. In other words, the CPU 201 automatically displays the image after image correction and the name of the image correction mode. The statement "automatic display" means displaying the name of the image correction processing associated with the display of the image after image correction (on the icon 705), without receiving additional instructions from the user for displaying the name.

[0107] For example, in step S310 following step S319, which will be described below, the CPU 201 displays the image that has undergone image correction processing in S319 in the corresponding display area, and displays the name of the image correction mode corresponding to the image correction mode on the corresponding icon. Therefore, it is possible to display... Figure 7 , Figure 9 , Figure 10 and Figure 11 The corresponding image shown.

[0108] If the CPU 201 detects in step S311 that the user has pressed the name icon (705, 905, 1005, 1105) corresponding to the target image correction process, the process moves to step S312, where list 804 is additionally displayed, similar to... Figure 8 In step S311, if the CPU 201 detects that the user has pressed the save icon (706, 906, 1006, 1106), the process moves to step S320.

[0109] If CPU 201 detects in step S312 Figure 8 If one of the multiple options in list 804 is selected, CPU 201 identifies which option was selected. If CPU 201 identifies that option 806 is pressed, the process moves to step S313. If CPU 201 identifies that option 807 is selected, the process moves to step S314. If CPU 201 identifies that option 808 is selected, the process moves to step S315. If CPU 201 identifies that option 809 is pressed, the process moves to step S313. If CPU 201 identifies that option 810 is selected, the process moves to step S311.

[0110] In step S313, CPU 201 stores "document" as a name corresponding to the image correction process to be applied below in a temporary area within storage unit 204.

[0111] In step S314, CPU 201 stores “Whiteboard 1” as a name corresponding to the image correction process to be applied below in a temporary area within storage unit 204.

[0112] In step S315, CPU 201 stores “Whiteboard 2” as a name corresponding to the image correction process to be applied below in a temporary area within storage unit 204.

[0113] In step S316, the CPU 201 stores “No image correction” as a name indicating that image correction processing should not be applied in a temporary area within the storage unit 204.

[0114] In step S317, CPU 201 determines whether the name stored in the processing name storage area within storage unit 204 matches the name stored in the temporary area within storage unit 204. If yes, CPU 201 proceeds the process to step S310. If no, CPU 201 proceeds the process to step S318.

[0115] In step S318, CPU 201 rewrites the name stored in the temporary area within storage unit 204 and stores the name in the processing name storage area within storage unit 204.

[0116] In the subsequent step S319, the CPU 201 reads the keystone-corrected image from the keystone-corrected image storage area in the storage unit 204, and performs image correction processing on the read image corresponding to the image correction mode stored in the processing name storage area. Then, the CPU 201 stores the image after image correction processing in the corrected image storage area in the storage unit 204. (Refer to...) Figure 4 The flowchart in the document describes the image correction process in detail. Then, CPU 201 causes the process to proceed to step S310.

[0117] In step S320, CPU 201 converts the image after image correction processing into a PDF file and stores the file in the file storage area within storage unit 204. While PDF format has been described as the conversion format, other file formats can be applied. The file format can be one that can be processed by a printer. Then, CPU 201 displays the image on display unit 207. Figure 12 The process moves to step S321 if the CPU 201 detects that the user has pressed icon 1206 on screen 1201.

[0118] In step S321, the CPU 201 displays on the display unit 207. Figure 13 In screen 1301, printer settings can be defined or changed for printing. If CPU 201 detects that the user has pressed the print command icon 1307, the process moves to step S322.

[0119] In step S322, CPU 201 sends the PDF file and print settings stored in the file storage area to the set printer (e.g., "Printer XYZ").

[0120] In step S323, CPU 201 terminates the processing in the flowchart.

[0121] The process allows for keystone correction and image correction of the captured images, and the processed images can be printed. Although the process doesn't display information identifying the type of image correction applied in the live view video of the subject being filmed, it does display this information after image correction processing. This prevents unclear display of the live view image of the subject and helps the user easily identify which of the various image correction processes has been applied.

[0122] Next, we will refer to Figure 4 The flowchart in the document describes the image correction process.

[0123] In step S401, CPU201 begins the processing in the flowchart.

[0124] In step S402, CPU 201 reads the name from the processing name storage area in storage unit 204.

[0125] In step S403, CPU 201 reads the keystone-corrected image from the keystone-corrected image storage area in storage unit 204.

[0126] In step S404, the CPU 201 applies image correction processing corresponding to the read name to the image after keystone correction processing, and stores the processed image in the corrected image storage area within the storage unit 204. According to this embodiment, various predetermined image correction processes corresponding to the names "Document," "Whiteboard 1," "Whiteboard 2," and "No Image Correction" are available. For example, the image correction process corresponding to the name "Document" performs shadow removal processing on the image after keystone correction processing. In addition to shadow removal processing, the image correction process corresponding to the name "Whiteboard 1" performs monochrome processing on the image after keystone correction processing to display clusters of similar colors represented by markers in the image in monochrome. In addition to the processing corresponding to the name "Whiteboard 1," the image correction process corresponding to the name "Whiteboard 2" also performs processing to expand the monochrome area on the image after keystone correction processing. The processing corresponding to the name "No Image Correction" does not perform image correction processing on the image after keystone correction processing.

[0127] In step S405, CPU 201 terminates the processing in the flowchart.

[0128] According to this disclosure, information about the predetermined image correction process performed for identification can be displayed while an image that has undergone predetermined image correction processing is being displayed.

[0129] Other embodiments

[0130] Embodiments of the invention can also be implemented by reading and executing computer-executable instructions (e.g., one or more programs) recorded on a storage medium (also more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more functions in the above embodiments, and / or by including a system or apparatus of one or more circuits (e.g., application-specific integrated circuits (ASICs)) for performing one or more functions in the above embodiments. Furthermore, embodiments of the invention can be implemented using methods by which a computer of a system or apparatus, for example, reads and executes computer-executable instructions from a storage medium to perform one or more functions in the above embodiments, and / or controls one or more circuits to perform one or more functions in the above embodiments. A computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessor unit (MPU)) and may include separate computers or a network of separate processors to read and execute computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, a hard disk, random access memory (RAM), read-only memory (ROM), the memory of a distributed computing system, an optical disc (such as a compact optical disc (CD), a digital versatile optical disc (DVD), or a Blu-ray disc (BD)). TM One or more of the following: flash memory devices and memory cards.

[0131] The embodiments of the present invention can also be implemented by providing software (programs) that perform the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads out and executes the program.

[0132] While the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims should be given the broadest interpretation to cover all such variations and equivalent structures and functions.

Claims

1. An information processing method, comprising: The step involves obtaining images captured by the camera unit. The display steps involve displaying the obtained image and cropping frame on the display unit; The transformation step involves transforming the clipping box by using the midpoints of each of the four boundaries of the clipping box. The first image correction step involves performing shape correction and brightness correction on the image cropped from the obtained image based on the transformed cropping frame. The brightness correction is used to increase the brightness of the shadows of the cropped image included in the cropped image. as well as The second image correction step involves performing a further brightness correction process based on a user operation received while the cropped image, which has already undergone shape correction and brightness correction processing, is displayed on the display unit to further increase the brightness of the shadows in the cropped image.

2. The information processing method according to claim 1, wherein, The transformation step transforms the cropping frame by further utilizing the points at the four corners of the cropping frame in the displayed image.

3. The information processing method according to claim 1, wherein, The first image correction step performs the shape correction process on the image in the region specified by the transformed cropping frame in the displayed image.

4. The information processing method according to claim 1, wherein, The first image correction step performs the shape correction process, and then performs a brightness correction process to increase the brightness of the shadows in the cropped image.

5. The information processing method according to claim 1, wherein, The shape correction process is trapezoidal correction.

6. The information processing method according to claim 5, wherein, The trapezoidal correction is a process used to perform geometric correction on the cropped image.

7. The information processing method according to claim 1, wherein, The camera unit is a camera.

8. The information processing method according to claim 1, wherein, The cropped image is the image corresponding to the document included in the obtained image.

9. The information processing method according to claim 1, further comprising: The receiving step involves receiving a setting for the aspect ratio of the image obtained after the shape correction process, while the image captured by the camera unit is displayed on the display unit.

10. The information processing method according to claim 9, wherein, The aspect ratio is the aspect ratio of the A4 paper size.

11. The information processing method according to claim 1, further comprising: Another display step involves displaying a name corresponding to the name of the brightness correction process while the image that has undergone the shape correction process and the brightness correction process is displayed on the display unit.

12. The information processing method according to claim 1, further comprising: In another display step, while the image that has undergone the shape correction processing and the brightness correction processing is displayed on the display unit, a save button is displayed, which is used to save the image that has undergone the shape correction processing and the brightness correction processing.

13. An information processing apparatus, comprising: The acquisition unit is configured to acquire images captured by the camera unit; The display unit is configured to display the acquired image and the cropping frame; Transformation unit, which is constructed to transform the clipping box by using the midpoints of the four boundaries of the clipping box. A first image correction unit is configured to perform shape correction and brightness correction processing on an image cropped from the obtained image based on a transformed cropping frame, wherein the brightness correction processing is used to increase the brightness of shadows of the cropped image included in the cropped image; as well as The second image correction unit is configured to perform further brightness correction processing based on a user operation received while the cropped image, which has already undergone shape correction and brightness correction processing by the first image correction unit, is displayed on the display unit, in order to further increase the brightness of the shadows in the cropped image.

14. The information processing apparatus according to claim 13, wherein, The first image correction unit performs the shape correction process, and then performs a brightness correction process to increase the brightness of the shadows in the cropped image.

15. The information processing apparatus according to claim 13, wherein, The display unit also displays a save button, which is used to save the image that has undergone the shape correction and brightness correction processes when the image is displayed on the display unit after the shape correction and brightness correction processes have been performed.

Images