Information processing device, information processing method, and information processing program

Abstract

This information processing device comprises a processor, wherein the processor acquires a user's operation history with regard to an image, extracts at least one extracted image from a candidate image group accessible by the user on the basis of the operation history, and sets a presentation mode for the extracted image on the basis of the operation history and the extracted image.

Description

Information processing apparatus, information processing method, and information processing program

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

[0002] Japanese Unexamined Patent Application Publication No. 2006-172097 describes that a parameter indicating a user's printing tendency and preference is calculated with reference to a user's usage history and characteristics of an analyzed image, and an image to be recommended for printing is determined from image data based on the parameter. It also describes that the parameter is updated each time the user's usage history is updated, the reliability of the parameter improves each time the user uses an image, and this enables selection of an image that matches the user's preference as a print-recommended image.

[0003] Japanese Unexamined Patent Application Publication No. 2020-098444 describes that an image and instruction history information indicating whether a print instruction has been given for the image are acquired, and conditions for a recommended image for which printing should be recommended are machine-learned based on a dataset in which the image and the instruction history information are associated with each other.

[0004] One embodiment according to the technology of the present disclosure provides an information processing apparatus, an information processing method, and an information processing program capable of appropriately presenting an image.

[0005] A first aspect of the present disclosure is an information processing apparatus including a processor, wherein the processor acquires a user's operation history related to an image, extracts at least one extracted image from a group of image candidates accessible to the user based on the operation history, and sets a presentation form of the extracted image based on the operation history and the extracted image.

[0006] The processor may perform a simulation using the extracted image based on the operation history.

[0007] The processor may acquire adjustment information for the simulation and perform the simulation based on the adjustment information.

[0008] The processor may set image processing to be applied to the extracted image based on the operation history and the extracted image, apply the image processing to the extracted image, and present the image.

[0009] The processor may extract an image from the group of image candidates that meet predetermined criteria.

[0010] The criteria may also be based on resolution.

[0011] The criteria may be determined based on the print resolution of the printer on which the extracted image is printed.

[0012] The processor may reshape the extracted image to a predetermined size and present the reshaped extracted image.

[0013] The processor may crop the extracted image to the specified size if the difference in aspect ratio between the extracted image and the specified size is less than a predetermined threshold value.

[0014] The processor may reshape the extracted image to the specified size by adding padding around the edges if the difference in aspect ratio between the extracted image and the specified size exceeds a predetermined threshold value.

[0015] The processor may identify images of interest that are presumed to be of interest to the user based on the operation history, and extract images based on the images of interest.

[0016] The processor may extract an image from the remaining image candidates after excluding images from the image candidate group that have a relevance to the image of interest less than a predetermined first criterion, and images that have a relevance greater than the first criterion but meet a predetermined second criterion or higher.

[0017] The processor may obtain user operation information regarding the extracted images presented according to the presentation format, and based on the operation information, it may redo at least one of the extraction of the extracted images and the setting of the presentation format.

[0018] The processor may, based on the operation information, decide which items to redo among the extraction of extracted images and the setting of the presentation format.

[0019] The operation history may include at least one of the following: image viewing history, saving history, editing history, and printing history.

[0020] The operation history includes time information about when the image was manipulated, and the processor may set the presentation format based on this time information.

[0021] The processor may set the timing for presenting the extracted images based on time information.

[0022] The processor may set the timing for extracting an image from the image candidate group based on time information.

[0023] A second aspect of this disclosure is an information processing method, wherein a computer performs a process to acquire a user's operation history related to an image, extract at least one extracted image from a group of image candidates accessible to the user based on the operation history, and set the presentation format of the extracted image based on the operation history and the extracted image.

[0024] A third aspect of this disclosure is an information processing program which causes a computer to perform the following processes: acquire a user's operation history related to an image; extract at least one extracted image from a group of image candidates accessible to the user based on the operation history; and set the presentation format of the extracted image based on the operation history and the extracted image.

[0025] This is a diagram illustrating an example of the schematic configuration of an information processing system. This is a block diagram illustrating an example of the hardware configuration of an information processing device. This is a block diagram illustrating an example of the functional configuration of an information processing device. This is a diagram illustrating an example of browsing history. This is a diagram illustrating an example of saved history. This is a diagram illustrating an example of editing history. This is a diagram illustrating an example of printing history. This is a diagram illustrating an example of a browsing screen. This is a diagram illustrating an example of an extracted image presentation screen. This is a diagram illustrating an example of an extracted image presentation screen. This is a diagram illustrating an example of an extracted image presentation screen. This is a diagram illustrating an example of an image formatting method. This is a diagram illustrating an example of an editing screen. This is a flowchart illustrating an example of information processing.

[0026] The following describes an example of an embodiment of the disclosed technology with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals, and redundant descriptions are omitted. Furthermore, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from actual ratios.

[0027] Referring to Figure 1, an example of the configuration of the information processing system 100, which includes the information processing device 10 according to this embodiment, will be described. The information processing system 100 is a system for presenting images to a user who uses the information processing device 10.

[0028] The information processing system 100 includes an information processing device 10, an image server 12, a camera 14, and a printer 16. The information processing device 10, the image server 12, the camera 14, and the printer 16 are connected to each other in a manner that allows them to communicate with one another via a wired or wireless network or other communication interface. The number of each device included in the information processing system 100 is not particularly limited, and each device may consist of multiple devices having similar functions.

[0029] The image server 12 is a computer that stores and manages various types of data, including images. Specifically, the image server 12 is a server computer for providing image services such as websites and SNS (Social Networking Service). The image server 12 has the function of storing images taken and created by users, as well as images posted (uploaded) by other users. The information processing device 10 can acquire various types of images by accessing the image server 12 via the network.

[0030] Camera 14 optically captures images in response to user operation. Camera 14 includes an imaging optical system that forms an image of light and an image sensor that outputs the formed optical image as an electrical image signal. As the image sensor, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor can be used. The image captured by camera 14 is transmitted to the information processing device 10. Note that camera 14 may be integrated with the information processing device 10, for example, as in a smartphone, or it may be configured as a standalone device.

[0031] The printer 16 is a device capable of printing images onto a recording medium. The printer 16 may support multiple types of recording media with different printing methods (e.g., instant photo method, dye-sublimation thermal transfer method, and inkjet method, etc.) and printing resolutions. Different types of recording media include, for example, instant film, inkjet paper, and sticker paper with different characteristics, those with the same characteristics but different sizes, and those with the same characteristics and size but different designs (base color and pattern, etc.). The information processing system 100 may be configured to print on various recording media by providing multiple printers 16 capable of printing on one type of recording medium.

[0032] The user manipulates (for example, views) various images, including images provided (downloaded) from the image server 12 and images captured by the camera 14, using the information processing device 10. At this stage, a large number of images are manipulated. The information processing device 10 extracts images based on the image manipulation history and presents them to the user, and sets the presentation format appropriately. By referring to the presented images, the user can easily decide which images to print using the printer 16. Below, an example of the configuration of the information processing device 10 according to this embodiment will be described.

[0033] Referring to Figure 2, an example of the hardware configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24, an input unit 25, and a communication interface (I / F) 26. The CPU 21, storage unit 22, memory 23, display 24, input unit 25, and communication interface 26 are connected to each other via a bus 28, such as a system bus and a control bus, enabling the exchange of various types of information.

[0034] The storage unit 22 is implemented by a storage medium such as an HDD (Hard Disk Drive), SSD (Solid State Drive), and flash memory. The storage unit 22 stores the information processing program 27 of the information processing device 10. The CPU 21 reads the information processing program 27 from the storage unit 22, expands it into the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of the processor of this disclosure.

[0035] The display 24 is, for example, a liquid crystal display and displays various information. The input unit 25 includes a pointing device such as a mouse and a keyboard, and is used to input various information to the device. The display 24 may be configured as a touch panel and used in conjunction with the input unit 25.

[0036] The communication interface 26 is an interface for communicating with other devices, including the image server 12, camera 14, and printer 16. For this communication, wired communication standards such as Ethernet (registered trademark) or FDDI (Fiber Distributed Data Interface), or wireless communication standards such as 4G, 5G, Wi-Fi (registered trademark), or Bluetooth (registered trademark) can be used. As the information processing device 10, for example, a smartphone, tablet terminal, wearable terminal, personal computer, and server computer can be appropriately applied.

[0037] Referring to Figure 3, an example of the functional configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 includes, as a functional configuration, an acquisition unit 30, an extraction unit 32, a setting unit 34, an image processing unit 36, and a control unit 38. The CPU 21 executes the information processing program 27, thereby enabling each functional unit to function.

[0038] The acquisition unit 30 acquires the user's operation history related to images. The operation history may be collected by, for example, the information processing device 10 or the image server 12. When the information processing device 10 collects the operation history, the acquisition unit 30 acquires the operation history stored in the storage unit 22 or the like. When the image server 12 collects the operation history, the acquisition unit 30 acquires the operation history stored in the image server 12.

[0039] In this disclosure, "image" includes various images, including still images or frames of moving images. For example, images include images provided by the image server 12, images (photographs) taken by the camera 14, and images generated by the information processing device 10. Images generated by the information processing device 10 include, for example, digital illustrations created using paint software, and screenshots. "Screenshot" refers to an image captured as a still image of the screen display content.

[0040] In this disclosure, “operation” includes viewing, saving, editing, and printing images. “Operation history” is a record of the results of user operations on an image. For example, operation history includes at least one of the following: image viewing history, saving history, editing history, and printing history. Operation history may also include time information on when the image was operated on. For example, time information includes at least one of the following: duration of time, time period, and frequency of time when the image was operated on.

[0041] FIG. 4 shows an example of an image browsing history. The "browsing history" is a record in which an image identification ID (image ID) is associated with the image browsing date and time, browsing time (duration), number of browsing times, and browsing medium. The number of browsing times is a counter indicating the number of times the same image has been browsed. The browsing medium includes, for example, an image providing service provided by the image server 12, and a local folder (internal storage such as the storage unit 22).

[0042] Furthermore, the browsing history related to a frame of a moving image is not limited to the frame itself, and may also include information such as the browsing time (duration) and the number of browsing times for the entire moving image. In addition, the browsing history may include identification information of frames with high browsing frequency in the entire moving image.

[0043] FIG. 5 shows an example of an image saving history. The "saving history" is a record in which an image identification ID is associated with the image saving date and time, and a saving destination. The saving destination may be, for example, internal storage such as a local folder, or external storage on a cloud, or may be a saving function (a so-called bookmark function) on an image browsing service.

[0044] FIG. 6 shows an example of an image editing history. The "editing history" is a record in which an image identification ID is associated with the image editing date and time, and editing content. The editing content includes, for example, the type of image processing and detailed content (parameters, etc.) of the image processing.

[0045] FIG. 7 shows an example of an image printing history. The "printing history" is a record in which an image identification ID is associated with the image printing date and time, the number of printed sheets, and printing conditions. The printing conditions include, for example, the type of recording medium and a printing method. The number of printed sheets may indicate the number of sheets printed in one printing operation, or may indicate a value obtained by summing the number of sheets printed in a plurality of printing operations for each image identification ID.

[0046] FIG. 8 shows an example of a screen D1 displayed on the display 24 by the control unit 38. The screen D1 is a screen for browsing and saving images, and is, for example, a viewer for images acquired from an image providing service provided by the image server 12 or a local folder. The screen D1 includes a plurality of images A1 to A6.

[0047] The screen D1 includes a menu 80 of "Browse", "Saved", "Recommended" and "History". The screen D1 is an example of a screen displayed when the "Browse" menu 80 is selected. When the "Recommended" menu 80 is selected, the screen transitions to screens D2 to D7 in FIGS. 9 to 14 (details will be described later). Note that the illustration method for screens D2 to D8 described later is the same as that for screen D1.

[0048] When the "Saved" menu 80 is selected, the screen transitions to a display screen for images saved on the screen D1, images previously saved on the image providing service provided by the image server 12, and / or images previously saved in a local folder (not shown). When the "History" menu 80 is selected, the screen transitions to a display screen for images browsed in the past, images saved in the past, and images recommended in the past (not shown).

[0049] The screen D1 includes "Popularity", "Newest" and "Random" as sorting options 82 for the images A1 to A6. The control unit 38 may accept a sorting selection by the user via the option 82, and perform control to sort and display the images A1 to A6 according to the selected option 82. "Popularity" means, for example, an order corresponding to the number of accesses in the image providing service.

[0050] The screen D1 includes buttons 84 for "Save", "Edit", "Print" and "Simulation". The user refers to the screen D1, selects the check box 86 of a desired image from among the images A1 to A6 to add a check mark, and then presses the "Save" button 84 to save the checked image. Browsing history and saving history are collected based on the operations during this process.

[0051] The extraction unit 32 extracts at least one image from a group of image candidates accessible to the user, based on the operation history. The "group of image candidates" is a collection of images that the user can view. The group of image candidates may include images that have been manipulated by the user (for example, images included in the browsing history, saving history, editing history, and printing history), but it may also include images that have not been manipulated. The details of the images included in the group of image candidates are the same as those of the images that the user has manipulated, as described above.

[0052] For example, the extraction unit 32 may pre-collect searchable images using a search engine and use these as a group of image candidates. Specifically, a certain number of images may be automatically collected from the internet via a browsing program. Alternatively, preset images may be stored in the storage unit 22, etc., and these images may be included in the group of image candidates. Furthermore, images may be aggregated from multiple different sources, such as local folders, cloud storage, and social networking services (SNS), to form a group of image candidates. Note that only sources permitted by the user may be targeted.

[0053] Specifically, the extraction unit 32 identifies images of interest that are presumed to be of interest to the user based on the operation history, and extracts images based on the images of interest. For example, the extraction unit 32 may use images that have been printed in the past as images of interest. Alternatively, the extraction unit 32 may use images that have been viewed for a predetermined time (e.g., 1 minute) or longer as images of interest. Alternatively, the extraction unit 32 may use images that have been saved but not printed as images of interest. Alternatively, the extraction unit 32 may estimate images as having a high degree of interest in order of the number of times they satisfy each of these conditions.

[0054] Furthermore, the extraction unit 32 may switch the recommendation logic depending on the user's behavior pattern. For example, a user who views many images in a short time may have different interests than a user who spends time viewing each image. Therefore, the extraction unit 32 may analyze each user's behavior pattern and change the extraction conditions.

[0055] The extraction unit 32 may use the image of interest itself as the extracted image, or it may analyze the image features of the image of interest and use other similar images as the extracted images. For example, the extraction unit 32 may analyze the image features of the image of interest, assign tag information, and use images with tag information identical or similar to the assigned tag information as the extracted images.

[0056] Furthermore, the extraction unit 32 may extract the desired image from the remaining image candidates after excluding images from the image candidate group whose relevance to the image of interest is less than a predetermined first criterion, and images whose relevance is greater than the first criterion but equals or greater than a predetermined second criterion. The "relevance" can be calculated using the similarity between the image features and the degree of overlap of tags assigned in advance based on the features. This allows for the elimination of images that are extremely biased towards the user's preferences while also excluding unrelated images, thus enabling the extraction of relevant images while maintaining diversity.

[0057] For example, when evaluating relevance based on tags, you can set weights for each tag or specify required or excluded tags. For instance, consider a case where a user frequently views images from a particular manga series. In this case, you can limit the extraction to images belonging to the broad category of tags for this manga series, while excluding images that share even the same subcategory, such as tags for the same character or scene. The broad and subcategories of tags can be determined based on the number of images to which they are assigned. Tags assigned to more images are presumed to be broad categories, while tags assigned to fewer images are presumed to be subcategories.

[0058] Furthermore, the extraction unit 32 may extract an image from the group of image candidates that meet predetermined criteria.

[0059] For example, the criterion may be a resolution criterion. For instance, the criterion may be that the image has a resolution equal to or greater than a predetermined threshold. Alternatively, the criterion may be determined based on the print resolution of the printer 16 on which the extracted image is printed. That is, the minimum resolution of the extracted image may be the print resolution of the printer 16. Alternatively, the minimum resolution of the extracted image may be a value calculated using a formula that utilizes the print resolution, such as half the print resolution. By narrowing down the extracted images based on the print resolution of the printer 16, the desired print quality can be achieved.

[0060] As another example, metadata attached to the image (e.g., Exif information) may be used as a criterion, such as whether the evaluation is above a predetermined threshold. For example, images with high scores on the camera 14's evaluation function or on the image provision service may be extracted.

[0061] Another example is using the suitability of the image's content for printing as a criterion. For instance, some recording media may not be suitable for landscape paintings or illustrations. Therefore, a criterion could be established based on the type of content suitable for each recording medium. The content can be identified through image frequency analysis, etc.

[0062] The setting unit 34 sets the presentation format of the extracted images based on the operation history and the extracted images. "Presentation format" includes the method of presenting the extracted images and the timing of the presentation of the extracted images. Examples of presentation methods include a presentation format in which multiple images are displayed on the screen, a format in which images are displayed one by one with screen switching, or a presentation format using simulation. For example, the number and arrangement of images when multiple images are displayed on the screen may also be set.

[0063] Specifically, the setting unit 34 may perform a simulation using the extracted image based on the operation history. "Simulation" refers to the process of combining the extracted image with a pre-prepared template image and presenting the generated composite image 50 to the user. As the template image, an image that mimics the state of the extracted image when printed, or the state of that printed material when used as decoration or an item, can be applied.

[0064] For example, template images could include images showing the image in a picture frame, in a photo stand, attached to a wall with tape or thumbtacks, and hanging with clips. For example, for the framed template image, variations could be provided in terms of frame type, background color, number of images that can be combined, layout of the combined images, and margin ratio. Furthermore, template images could include images showing the image in a smartphone case, clipped to a keychain, and placed in a photo album.

[0065] For example, a template image could be an image showing the extracted image printed on various recording media (e.g., instant film or photographic paper). For example, as a template image showing the image printed on instant film, variations may be provided in the size of the instant film (mini film, wide film, and square film, etc.) and the design of the instant film (base color and pattern, etc.). In this case, it is preferable to use a recording medium that can be printed by the printer 16.

[0066] The control unit 38 controls the display 24 to present the extracted images according to the set presentation mode. Figures 9 to 14 show examples of screens D2 to D7 displayed on the display 24 by the control unit 38. Screens D2 to D7 are screens for presenting extracted images. Screens D2 to D7 display at least a portion of the multiple extracted images R1 to R6 in a composite form on a template image.

[0067] In Figure 9, screen D2 displays a list of composite images 50 showing the extracted images R1 to R6 printed on instant film 51. In Figure 10, screen D3 and in Figure 11, screen D4 display composite images 50 showing the extracted images R1 to R2 printed on instant film 51 and placed in a single frame 52, but with different layouts. In Figure 12, screen D5 displays a composite image 50 showing the extracted images R1 to R3, each printed on instant film 51 with a different base design, placed in a single frame, with the number of images in the frame increasing to three.

[0068] In Figure 13, screen D6 displays a composite image 50 showing extracted images R1 to R3 printed on instant film 51 and then attached to a wall surface 53 with tape 54. Also, extracted image R1 in screen D6 is rotated horizontally. In Figure 14, screen D7 displays a composite image 50 showing extracted images R1 to R3 printed on instant film 51 of different sizes and then attached to a wall surface 53 with tape 54. Extracted image R1 is printed on mini film, extracted image R2 is printed on square film, and extracted image R3 is printed on wide film. Also, in screen D7, each instant film 51 has handwritten-style text 55 written in the frame area.

[0069] In this way, when presenting extracted images, for example, the extracted images may be rotated. Alternatively, for example, the state of each extracted image as printed on a different recording medium may be shown. That is, various types, sizes, and designs of recording media may be mixed on the presentation screen. For example, instant film and photographic paper may be mixed together. Alternatively, for example, text 55 may be added to the margins outside the extracted images (including the frame portion on the recording medium).

[0070] In any of the screens D2 to D7, the user selects the checkbox 86 for the extracted image(s) they wish to simulate in the extracted image list 88 and places a check mark on it. The control unit 38 uses the selected extracted images as the synthesis target and, if necessary, switches to a different template or changes the layout to present various usage images. Additionally, an "other preview" button 84 may be provided, and when pressed by the user, the template image may be switched.

[0071] The setting unit 34 may acquire adjustment information for the simulation and perform the simulation based on that information. The "adjustment information" may include, for example, background color, number of images, layout, and margin ratio. The setting unit 34 may also accept adjustment information input from the user via the input unit 25. For example, the type of frame in the simulation, the selection of images within the frame, the number of images within the frame, and the order in which the images within the frame are arranged may be freely set. Alternatively, for example, the size and design of the instant film in the simulation may be freely set.

[0072] The user may save the selected images by pressing the "Save" button 84. Alternatively, the user may edit and print the selected images together by pressing the "Edit and Print" button 84 (details below). The user may also print the selected images together by pressing the "Print" button 84. Operation history is collected during these operations.

[0073] Furthermore, it is preferable to use images that mimic actual products for sale as template images. That is, the picture frames and photo albums shown in the composite image 50 may be sold as actual products. This allows users to realize the simulation state by purchasing the product and printing the extracted image.

[0074] Furthermore, the "Simulation" button 84 may also be displayed on screen D1, which is displayed when the "Browse" menu 80 is selected, on the screen displayed when the "Saved" menu 80 is selected, and on the screen displayed when "History" is selected (see Figure 8). When the "Simulation" button 84 is pressed, the setting unit 34 performs the same simulation as described above. In this case, the image to be composited with the template image may be selected by the user, or the setting unit 34 may select it as appropriate.

[0075] The setting unit 34 may set the presentation format based on time information. Specifically, the setting unit 34 may set the timing for presenting extracted images based on time information. For example, if the setting unit 34 finds a tendency for operations to be more frequent during a particular time period based on the operation history, it may set the extracted images to be presented according to that time period. Alternatively, for example, the setting unit 34 may automatically make recommendations when a certain number of operations are reached.

[0076] Furthermore, the setting unit 34 may set the timing for extracting images from the image candidate group based on time information. For example, if a user continuously views images in a specific field during the night, the system may perform image extraction for that field again at night. Alternatively, for example, the system may assume a user group based on the time of day the operation is performed, setting an adult-oriented presentation format at night and a student-oriented presentation format during the day.

[0077] Alternatively, the system could be configured to display extracted images when a user takes an action (such as clicking "Like") on an image provision service. Or, for example, it could be configured to display extracted images when a predetermined amount of time has elapsed since the last display.

[0078] The setting unit 34 may set the image processing to be applied to the extracted images based on the operation history and the extracted images. That is, the setting unit 34 may estimate the user's preferred image processing tendencies (e.g., brightness correction or color filter) based on the operation history and set appropriate image processing for each extracted image in accordance with those tendencies.

[0079] The image processing unit 36 ​​performs image processing on the extracted image. "Image processing" includes, for example, adjusting pixel values ​​such as brightness and contrast, applying filters, enlarging or reducing, cropping, moving, rotating, flipping, changing resolution, and inputting text. If the extracted image has been processed, the control unit 38 presents the extracted image after image processing.

[0080] Specifically, the image processing unit 36 ​​may reshape the extracted image to a predetermined size. "Shaping" means making the image an image with predetermined width and height. That is, the aspect ratio and resolution are also adjusted to predetermined values. Examples of shaping methods include cropping a part of the image (circumscribed mode), filling in the periphery of the image (inscribed mode), and enlarging or reducing the image in the vertical and / or horizontal directions.

[0081] An example of a molding method will be described with reference to Figure 15. Figure 15 shows a molded image 90A formed in circumscribed mode and a molded image 90B formed in inscribed mode from the original extracted image 90. In circumscribed mode, the original extracted image 90 is trimmed to a specified size to obtain molded image 90A. In circumscribed mode, the entire original extracted image 90 cannot be printed, so it is preferable not to cut out important subjects (e.g., text and main subjects).

[0082] On the other hand, in inscribed mode, the original extracted image 90 is shaped to a specified size by supplementing its periphery to obtain a shaped image 90B. "Supplementation" means filling in the missing portion 92 of the specified size with an arbitrary image. For example, as shown in Figure 15, the missing portion 92 may be filled with a color extracted from the periphery of the original extracted image 90, or it may be filled in to create a gradient using that color. Alternatively, the missing portion 92 may be filled with a predetermined color or pattern image. In inscribed mode, although the missing portion 92 may be noticeable, it has the advantage of being able to print the entire original extracted image 90. As mentioned above, filling the missing portion 92 with a color extracted from the periphery of the original extracted image 90 can reduce the unnaturalness caused by the missing portion 92.

[0083] The user may choose which method to use for shaping, or the image processing unit 36 ​​may decide. For example, if the difference in aspect ratio between the extracted image and the specified size is less than a predetermined reference value, the image processing unit 36 ​​may apply an external mode, which shapes the extracted image to the specified size by cropping it. Alternatively, if the difference in aspect ratio between the extracted image and the specified size is greater than or equal to a predetermined reference value, the image processing unit 36 ​​may apply an internal mode, which shapes the extracted image to the specified size by filling in the area around it.

[0084] It is preferable that the decision to crop or supplement based on the aspect ratio is performed last, after all other image processing has been applied. In other words, various image processing may be performed before cropping or supplementation. For example, if cropping an important portion from an extracted image results in a large difference in aspect ratio, supplementation (processing in inscribed mode) may be performed on the cropped extracted image.

[0085] Figure 16 shows an example of screen D8 displayed on the display 24 by the control unit 38. Screen D8 is a screen for editing (image processing) the extracted image. The user transitions to screen D8 when the "Edit" button 84 is pressed on screen D1, or when the "Edit and Print" button 84 is pressed on screens D2 to D7.

[0086] Screen D8 contains extracted images R1 to R3, with extracted image R2 being displayed larger than the other images. This indicates that extracted image R2 is currently selected and is the target for editing. Including the unselected extracted images R1 and R3 on the same screen D8 allows for checking the harmony with other images and seamlessly changing the selection status.

[0087] Screen D8 includes buttons 84 for "Brightness, etc.," "Filter," "Resolution," "Text," "Rotation," "Circumscribed / Inscribed" (switching between circumscribed and inscribed modes), "Zoom," and "Print." When the user selects each button 84, a detailed settings window pops up or automatic corrections are applied. For example, rotation can be set to be possible in 90-degree increments or at any angle. Also, for example, an upper limit on zoom can be set, and a warning may be displayed if the resolution becomes insufficient. Also, for example, arbitrary text information can be added to the extracted image.

[0088] Furthermore, the control unit 38 may acquire user operation information regarding the extracted image presented according to the presentation mode, and may redo at least one of the extraction of the extracted image and the setting of the presentation mode based on the operation information. For example, if the user does not save or print the extracted image, a feedback loop may be configured in which the extraction unit 32 extracts a different image again, or the setting unit 34 changes the presentation mode.

[0089] The control unit 38 may, based on the operation information, decide which items to redo among the extraction of extracted images and the setting of the presentation format. For example, if the user saved the extracted images but did not print them, the extracted images may remain as they are, and only the change in the presentation format may be redone.

[0090] The control unit 38 may control the printer 16 to print the selected extracted images. For example, it may have a function to print multiple images at once when they are laid out on the simulation screen. Alternatively, images that have been printed can be moved to the "print history" and excluded from subsequent extractions to prevent unnecessary duplication.

[0091] (Example 1) This example assumes that a user views images uploaded to SNS on the display 24 of the information processing device 10. The acquisition unit 30 acquires the image viewing history. The extraction unit 32 identifies images uploaded to SNS that are not included in the viewing history as image candidates. Based on the viewing history, the extraction unit 32 considers images that the user has viewed for a long time as images of interest. From the image candidate group, the extraction unit 32 excludes images with similarity to images of interest that are too low (below the first criterion) and images with similarity that are too high (above the second criterion), and extracts the remaining image candidates as extracted images. The setting unit 34 sets a presentation format that matches the user's preferences based on the viewing history. For example, it sets a presentation format that matches the taste of the images included in the viewing history.

[0092] (Example 2) This example uses screenshots of videos provided by a video streaming service as the target for extraction. The acquisition unit 30 acquires the viewing history, including the viewing time and viewing frequency of the video. Based on the viewing history, the extraction unit 32 considers frames of scenes that the user frequently plays as images of interest. The extraction unit 32 treats each frame of the video as a group of image candidates. If the extraction unit 32 estimates that the user has a strong interest in a particular character, it extracts image candidates that include that character and are in different scenes as extracted images. Based on the viewing history, the setting unit 34 sets the timing for presenting extracted images to nighttime, as nighttime operation is more frequent.

[0093] (Example 3) This example assumes the use of printing user photos stored in cloud storage. The acquisition unit 30 acquires the storage history and editing history for the most recent month. The extraction unit 32 estimates the user's tendency to prefer vivid colors recently based on the storage history and editing history. The extraction unit 32 considers all photos on the cloud storage as a group of image candidates. The extraction unit 32 extracts photos that have sufficient resolution and vivid colors as extracted images. The setting unit 34 sets image processing to correct for high saturation suitable for photo printing based on the user's preferences estimated from the operation history, and also sets "frame preview" as the presentation format.

[0094] Next, the operation of the information processing device 10 will be explained with reference to Figure 17. In the information processing device 10, the CPU 21 executes the information processing program 27, thereby executing the information processing shown in Figure 17. This processing is executed, for example, when the user issues an instruction to start execution.

[0095] In step S10, the acquisition unit 30 acquires the user's operation history related to the image. In step S12, the extraction unit 32 extracts at least one image from a group of image candidates accessible to the user based on the operation history. In step S14, the setting unit 34 sets the presentation format of the extracted image based on the operation history and the extracted image. In step S16, the control unit 38 controls the display 24 to present the extracted image according to the set presentation format, and terminates this process.

[0096] As described above, the information processing device 10 according to this embodiment includes a processor. The processor acquires the user's operation history related to an image, extracts at least one extracted image from a group of image candidates accessible to the user based on the operation history, and sets the presentation format of the extracted image based on the operation history and the extracted image.

[0097] In other words, according to the information processing device 10 of this embodiment, an appropriate presentation format can be set according to the user's operation history and extracted images, so images can be presented appropriately to the user. Therefore, it is possible to encourage printing using the printer 16.

[0098] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0099] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing a specific process such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). The type of hardware may also be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0100] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on physically separate devices. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0101] Furthermore, although the above embodiment describes a configuration in which the information processing program 27 is pre-stored (installed) in the storage unit 22, the invention is not limited to this configuration. The information processing program 27 may be provided in the form of a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the information processing program 27 may be provided in the form of a download from an external device via a network.

[0102] The technology disclosed herein extends to all program products. A program product includes all forms of products for providing programs. For example, a program product includes programs provided via a network such as the Internet, as well as non-temporary computer-readable recording media such as CD-ROMs, DVD-ROMs, and USB memory sticks on which programs are stored.

[0103] The technology of this disclosure can also be appropriately combined with the above-described embodiments and modifications. The descriptions and illustrations shown above are detailed explanations of the parts relating to the technology of this disclosure and are merely examples of the technology of this disclosure. For example, the above descriptions of the configuration, function, operation, and effect are examples of the configuration, function, operation, and effect of the parts relating to the technology of this disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements added, or replaced from the descriptions and illustrations shown above, as long as they do not deviate from the spirit of the technology of this disclosure.

[0104] The following further notes are disclosed regarding the above embodiments. [Note 1] An information processing device comprising a processor, wherein the processor acquires a user's operation history related to an image, extracts at least one extracted image from a group of image candidates accessible to the user based on the operation history, and sets the presentation form of the extracted image based on the operation history and the extracted image. [Note 2] The information processing device according to Note 1, wherein the processor performs a simulation using the extracted image based on the operation history. [Note 3] The information processing device according to Note 2, wherein the processor acquires adjustment information for the simulation and performs the simulation based on the adjustment information. [Note 4] The information processing device according to any one of Notes 1 to 3, wherein the processor sets image processing to be applied to the extracted image based on the operation history and the extracted image, and presents the extracted image after applying the image processing. [Note 5] The information processing device according to any one of Notes 1 to 4, wherein the processor extracts the extracted image from an image candidate that satisfies predetermined criteria among the group of image candidates. [Note 6] The criterion is a criterion relating to resolution, as described in Note 5. [Note 7] The criterion is determined based on the print resolution of the printer on which the extracted image is printed, as described in Note 6. [Note 8] The processor is an information processing device according to any one of Notes 1 to 7, which shapes the extracted image to a predetermined size and presents the shaped extracted image. [Note 9] The processor is an information processing device according to Note 8, which shapes the extracted image to the predetermined size by trimming it if the difference in aspect ratio between the extracted image and the predetermined size is less than a predetermined standard value. [Note 10] The processor is an information processing device according to Note 8 or Note 9, which shapes the extracted image to the predetermined size by filling in the periphery of it if the difference in aspect ratio between the extracted image and the predetermined size is greater than or equal to a predetermined standard value. [Note 11] The information processing apparatus according to any one of Notes 1 to 10, wherein the processor identifies an image of interest that is presumed to be of interest to the user based on the operation history, and extracts the extracted image based on the image of interest.[Note 12] The information processing device according to Note 11, wherein the processor extracts the extracted image from the remaining image candidates, excluding images from the group of image candidates whose degree of relevance to the image of interest is less than a predetermined first criterion and images whose degree of relevance is greater than the first criterion and equal to or greater than a predetermined second criterion. [Note 13] The information processing device according to any one of Notes 1 to 12, wherein the processor acquires user operation information for the extracted image presented according to the presentation format, and redoes at least one of the extraction of the extracted image and the setting of the presentation format based on the operation information. [Note 14] The information processing device according to Note 13, wherein the processor determines which of the extraction of the extracted image and the setting of the presentation format to redo based on the operation information. [Note 15] The information processing device according to any one of Notes 1 to 14, wherein the operation history includes at least one of the image viewing history, saving history, editing history, and printing history. [Note 16] The information processing device according to any one of Notes 1 to 15, wherein the operation history includes time information when the image was manipulated, and the processor sets the presentation form based on the time information. [Note 17] The information processing device according to Note 16, wherein the processor sets the presentation timing of the extracted image based on the time information. [Note 18] The information processing device according to Note 16 or Note 17, wherein the processor sets the timing for extracting the extracted image from the image candidate group based on the time information. [Note 19] The information processing device according to any one of Notes 16 to 18, wherein the time information includes at least one of the length of time, time period, and frequency when the image was manipulated. [Note 20] The information processing device according to Note 3, wherein the processor accepts the input of the adjustment information by the user. [Note 21] The information processing device according to any one of Notes 1 to 20, wherein the image candidate group includes screenshots. [Appendix 22] An information processing method in which a computer performs the following processes: acquires the user's operation history regarding an image; extracts at least one extracted image from a group of image candidates accessible to the user based on the operation history; and sets the presentation format of the extracted image based on the operation history and the extracted image.[Note 23] An information processing program that causes a computer to perform the following processes: acquire the user's operation history regarding an image; extract at least one extracted image from a group of image candidates accessible to the user based on the operation history; and set the presentation format of the extracted image based on the operation history and the extracted image.

[0105] The disclosure of Japanese Patent Application No. 2025-057052, filed on 28 March 2025, is incorporated herein by reference in its entirety. All documents, patent applications and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application and technical standard were specifically and individually noted to be incorporated by reference.

Claims

1. An information processing device comprising a processor, the processor acquiring a user's operation history related to an image, extracting at least one extracted image from a group of image candidates accessible to the user based on the operation history, and setting the presentation format of the extracted image based on the operation history and the extracted image.

2. The information processing apparatus according to claim 1, wherein the processor performs a simulation using the extracted image based on the operation history.

3. The information processing apparatus according to claim 2, wherein the processor acquires adjustment information for the simulation and performs the simulation based on the adjustment information.

4. The information processing apparatus according to claim 1, wherein the processor sets image processing to be performed on the extracted image based on the operation history and the extracted image, and presents the extracted image after performing the image processing.

5. The information processing apparatus according to claim 1, wherein the processor extracts the extracted image from the group of image candidates that satisfy predetermined criteria.

6. The information processing apparatus according to claim 5, wherein the aforementioned standard is a standard relating to resolution.

7. The information processing apparatus according to claim 6, wherein the standard is determined based on the print resolution of the printer on which the extracted image is printed.

8. The information processing apparatus according to claim 1, wherein the processor formats the extracted image to a predetermined size and presents the formatted extracted image.

9. The information processing apparatus according to claim 8, wherein the processor reshapes the extracted image to the specified size by trimming it if the difference in aspect ratio between the extracted image and the specified size is less than a predetermined reference value.

10. The information processing apparatus according to claim 8, wherein the processor, when the difference in aspect ratio between the extracted image and the specified size is greater than or equal to a predetermined reference value, shapes the extracted image to the specified size by filling in the area around it.

11. The information processing apparatus according to claim 1, wherein the processor identifies an image of interest that is presumed to be of interest to the user based on the operation history, and extracts the extracted image based on the image of interest.

12. The information processing apparatus according to claim 11, wherein the processor extracts the extracted image from the remaining image candidates, excluding images from the group of image candidates whose degree of relevance to the image of interest is less than a predetermined first criterion and images whose degree of relevance is greater than the first criterion and equal to or greater than a predetermined second criterion.

13. The information processing apparatus according to claim 1, wherein the processor acquires user operation information for the extracted image presented according to the presentation format, and, based on the operation information, repeats at least one of the extraction of the extracted image and the setting of the presentation format.

14. The information processing apparatus according to claim 13, wherein the processor determines, based on the operation information, an item to be redone from the extraction of the extracted image and the setting of the presentation form.

15. The information processing apparatus according to claim 1, wherein the operation history includes at least one of the image viewing history, saving history, editing history, and printing history.

16. The information processing apparatus according to claim 1, wherein the operation history includes time information of when the image was manipulated, and the processor sets the presentation form based on the time information.

17. The information processing apparatus according to claim 16, wherein the processor sets the presentation timing of the extracted image based on the time information.

18. The information processing apparatus according to claim 16, wherein the processor sets the timing for extracting the extracted image from the image candidate group based on the time information.

19. An information processing method in which a computer performs the following processes: acquires the user's operation history related to an image; extracts at least one extracted image from a group of image candidates accessible to the user based on the operation history; and sets the presentation format of the extracted image based on the operation history and the extracted image.

20. An information processing program that causes a computer to perform the following processes: acquire a user's operation history related to an image; extract at least one extracted image from a group of image candidates accessible to the user based on the operation history; and set the presentation format of the extracted image based on the operation history and the extracted image.

Images