Information processing device, information processing method, and information processing program

Abstract

When taking a screenshot, additional information tailored to the user's profile will be added and provided. [Solution] The information processing device according to the present invention is characterized by comprising: an acquisition unit that acquires screenshots taken by a user; an estimation unit that estimates the user's profile information; a generation unit that generates additional information corresponding to the user's profile information regarding the screenshot based on the information related to the acquired screenshot and the user's profile information; an addition unit that adds the additional information to the screenshot; and a provision unit that provides the added additional information to the user along with the screenshot.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] With the spread of SNS (Social Networking Service), a technique for sharing generated screenshots with other users has been disclosed (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above prior art, screenshots are generated by users viewing distribution content, but users cannot obtain any further information from the screenshots.

[0005] The present application has been made in view of the above, and an object thereof is to add and provide additional information according to the user's profile when taking a screenshot.

Means for Solving the Problems

[0006] The information processing device according to the present application is characterized by comprising: an acquisition unit that acquires screenshots taken by a user; an estimation unit that estimates the user's profile information; a generation unit that generates additional information corresponding to the user's profile information with respect to the screenshot based on the acquired information relating to the screenshot and the user's profile information; an addition unit that adds the additional information to the screenshot; and a provision unit that provides the added additional information to the user together with the screenshot. [Effects of the Invention]

[0007] According to one embodiment, when taking a screenshot, additional information corresponding to the user's profile can be added and provided. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an explanatory diagram showing an overview of the information processing system according to the embodiment. [Figure 2] Figure 2 shows an example of the configuration of a terminal device according to this embodiment. [Figure 3] Figure 3 shows an example of the configuration of a server device according to this embodiment. [Figure 4] Figure 4 is a flowchart showing the processing procedure according to the embodiment. [Figure 5] Figure 5 shows an example of a hardware configuration. [Modes for carrying out the invention]

[0009] The following describes in detail, with reference to the drawings, embodiments for implementing the information processing device, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments"). Note that these embodiments do not limit the information processing device, information processing method, and information processing program according to the present application. Furthermore, the same parts are denoted by the same reference numerals in the following embodiments, and redundant descriptions are omitted.

[0010] [1. Overview of the Information Processing System] First, with reference to Figure 1, an overview of the information processing system according to the embodiment will be described. Figure 1 is an explanatory diagram showing an overview of the information processing system according to the embodiment. As shown in Figure 1, the information processing system 1 according to the embodiment includes a terminal device 10 and a server device 100. The terminal device 10 and the server device 100 are connected to each other via a network N, either by wired or wireless means, enabling communication between them. This allows the terminal device 10 to cooperate with the server device 100. The network N is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

[0011] Terminal device 10 is an information processing device used by user U. For example, terminal device 10 may be a smart device such as a smartphone or tablet, or a PC (Personal Computer) such as a desktop or notebook (laptop). Alternatively, terminal device 10 may be a mobile phone such as a feature phone, a PDA (Personal Digital Assistant), a game console or AV equipment with communication functions, an information appliance or digital appliance, a car navigation system, a smartwatch or head-mounted display (HDD), a wearable device such as smart glasses, or an IoT (Internet of Things) compatible house or building, car, home appliance, electronic device, etc.

[0012] In this embodiment, the terminal device 10 is a smart device such as a smartphone or tablet used by user U, and is a mobile terminal device that can communicate with any server device via wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), Bluetooth (registered trademark), or wireless LAN. The terminal device 10 also has a screen such as a liquid crystal display with touch panel functionality, and accepts various operations on displayed data such as content from user U using a finger or stylus, such as tapping, sliding, and scrolling. Operations performed on the area of ​​the screen where content is displayed may also be considered as operations on the content. Furthermore, the terminal device 10 may be an information processing device such as a desktop PC or notebook PC, not just a smart device.

[0013] The server device 100 is, for example, a computer such as a PC or blade server, or a mainframe or workstation. The server device 100 may also be implemented through cloud computing.

[0014] In this embodiment, the server device 100 is an information processing device that works in conjunction with each user U's terminal device 10 and provides each user U's terminal device 10 with API (Application Programming Interface) services for various applications (hereinafter referred to as "apps") and various data, and is implemented by a computer or cloud system.

[0015] Furthermore, the server device 100 may be an information processing device that provides some kind of online service to each user U's terminal device 10. For example, the server device 100 may provide services such as internet connection, search service, advertising distribution service, chat service, conversational service using voice, images, video, etc., SNS (Social Networking Service), e-commerce (EC), electronic payment, online games, online banking, online trading, accommodation / ticket reservations, video / music distribution, news, maps, route search, route guidance, route information, service information, and weather forecasts as online services. In practice, the server device 100 may cooperate with various servers that provide the above-mentioned online services and act as an intermediary for online services, or it may be responsible for processing online services.

[0016] The server device 100 can acquire user information about user U. For example, the server device 100 can acquire information about user U's attributes (attribute information), such as gender, age, and residential area. The server device 100 can also acquire information about user U's demographics, psychographics, geographics, behavioral attributes, etc. The server device 100 may also acquire information about the segment or persona to which user U belongs in the field of marketing, as user information. The server device 100 stores and manages information about user U's attributes (attribute information) along with identification information (user ID, etc.) that identifies user U.

[0017] In addition, the server device 100 acquires various types of history information (log data) indicating the actions of the user U from the terminal device 10 of the user U or from various servers or the like based on the user ID or the like. For example, the server device 100 acquires a location history, which is a history of the location and time of the user U, from the terminal device 10. Further, the server device 100 acquires a search history, which is a history of search queries input by the user U, from a search server (search engine). Further, the server device 100 acquires a browsing history, which is a history of the content browsed by the user U, from a content server. Further, the server device 100 acquires a purchase history (settlement history), which is a history of the user U's product purchases and settlement processes, from an e-commerce server or a settlement processing server. Further, the server device 100 may acquire a listing history or a sales history, which is a history of the user U's listings on the marketplace, from an e-commerce server or a settlement processing server. Further, the server device 100 acquires a posting history, which is a history of the user U's posts, from a posting server or an SNS server that provides a word-of-mouth posting service. Note that each of the above various servers or the like may be the server device 100 itself. That is, the server device 100 may function as each of the above various servers or the like.

[0018] In addition, the number of each device included in the information processing system 1 shown in FIG. 1 is not limited to that shown. For example, in FIG. 1, for the sake of simplification of the illustration, only one terminal device 10 is shown, but this is merely an example and is not limited, and two or more terminal devices may be provided.

[0019] [2. "Screenshot Information Enhancer" Function] When taking a screenshot of a news article, the server device 100 according to this embodiment provides a function (the "screenshot information enhancer" function) that provides additional information based on the profile and interests of the user U (the user) who will be the viewer of the screenshot. This function promotes deeper understanding by automatically adding additional information and context related to the content when the user takes a screenshot of a specific part of the article. Also, the information added varies depending on the user taking the screenshot. As a result, the screenshot becomes not just a simple image but a valuable information source. The main functions are as follows.

[0020] (1) Information addition based on user profile Function overview: Based on the user's profile information (interests, past browsing history, etc.), additional information related to the screenshot (additional information for each user) is provided. For example, when taking a screenshot of an economic-related article, market data and expert comments that the user is interested in are automatically added.

[0021] (2) Context tagging function Function overview: Keywords and topics related to the screenshot are automatically generated as tags and added to the image. In particular, keywords and topics that the user is likely to be interested in are automatically generated as tags from the user's profile information and added to the image. This makes it easy to search for related information when looking back at the screenshot later.

[0022] (3) Embedding of interactive links Function overview: Interactive links are embedded in the screenshot to enable direct access to related articles and data. The user can conduct further information exploration through the screenshot. Note that the link is not limited to text but can also be an icon or an image. Also, the destination of the link is not limited to articles or content but can also be other screenshots.

[0023] (4) Personalized summary Feature Overview: A personalized summary of the article in the screenshot is generated and added to the image. This allows users to easily grasp the key points.

[0024] (5) Display of community insights Feature Overview: It displays comments and ratings from other users who have taken screenshots of the same article, providing community insights. This promotes information sharing among users and enables the acquisition of diverse perspectives.

[0025] (6) Screenshot and archive function Feature Overview: The system automatically archives screenshots taken by users, making them accessible later. The archive also includes any added information and links, making it easy to reuse the information.

[0026] This "Screenshot Info Enhancer" feature transforms the common act of taking a screenshot into a highly informative experience. Users can not only save articles but also gain related information and refer to community opinions, deepening their understanding of the news. This feature enables online news platforms to provide more interactive and personalized services.

[0027] [2-1. Processing Flow] Next, we will explain the processing flow of the "Screenshot Info Enhancer" function.

[0028] For example, as shown in Figure 1, user U causes terminal device 10 to take a screenshot by operating terminal device 10 (step S1). For example, terminal device 10 takes a desired screenshot of the distributed content when user U operates terminal device 10. At this time, terminal device 10 may post the captured screenshot to a social networking service (SNS) or similar service specified by user U. The distributed content from which the screenshot is taken is, for example, distributed content that user U searches for, and may be news articles, book content, blog posts, content found through searches on SNS, etc., or other content.

[0029] Next, the server device 100 acquires a screenshot taken by user U's terminal device 10 via the network N (step S2). For example, the server device 100 acquires the screenshot taken in cooperation with user U's terminal device 10. Alternatively, the server device 100 may acquire screenshots posted on social media, etc. Furthermore, when user U takes a screenshot of the distributed content with terminal device 10, the application or other function may automatically send the screenshot from terminal device 10 to the server device 100, or user U may be asked whether or not to send the screenshot to the server device 100, and the screenshot will be sent to the server device 100 only if user U grants permission.

[0030] Next, the server device 100 estimates user U's profile information (step S3). For example, the server device 100 acquires user U's attribute information and history information, and estimates user U's interests and user persona (user profile) from this information. At this time, the server device 100 may estimate user U's profile information using a trained AI (Artificial Intelligence) or model.

[0031] Next, the server device 100 uses AI to generate additional information (user-specific additional information) regarding the screenshot, based on the information related to the acquired screenshot and the profile information of user U (step S4). For example, the server device 100 generates supplementary information and explanations of the article from which the screenshot was taken as additional information, according to the knowledge level of user U. The server device 100 also generates related or derived information that delves deeper into or expands upon parts of the article from which the screenshot was taken that are of particular interest to user U. Furthermore, if the server device 100 is not familiar with the media from which the original article from which the screenshot was taken was published, it generates an explanation of the characteristics (good points) of that media as additional information.

[0032] Next, the server device 100 automatically adds the generated additional information to the information related to the acquired screenshot (step S5). For example, the server device 100 stores the identification information indicating user U (user ID, etc.), the acquired screenshot, and the generated additional information in association. At this time, the server device 100 may generate new content by adding the generated additional information to the information related to the acquired screenshot.

[0033] Next, the server device 100 provides the generated additional information, along with a screenshot, to the user U's terminal device 10 via the network N (step S6). At this time, the server device 100 may also control how the screenshot and the additional information are displayed on the user U's terminal device 10.

[0034] Thus, in this embodiment, different information is added depending on the screenshot user. The server device 100 according to this embodiment generates additional information for the screenshot based on the information of the user who took the screenshot. Then, it provides the generated additional information linked to the screenshot.

[0035] The server device 100 may implement the above mechanism using an AI such as GPT (Generative Pre-trained Transformer). GPT is a text generation AI and a language model capable of generating text using natural language processing.

[0036] [3. Example of terminal device configuration] Next, the configuration of the terminal device 10 will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the terminal device 10 according to the embodiment. As shown in Figure 2, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40.

[0037] (Communications Section 11) The communication unit 11 is connected to the network N by wire or wireless connection and transmits and receives information to and from the server device 100 via the network N. For example, the communication unit 11 is implemented by a NIC (Network Interface Card), an antenna, and a communication circuit according to the communication method.

[0038] (Display section 12) The display unit 12 is a display device that displays various information such as location information. For example, the display unit 12 may be a liquid crystal display (LCD) or an organic electro-luminescent display (OLED). The display unit 12 may also be a touch panel display, but is not limited to this.

[0039] (Input section 13) The input unit 13 is an input device that receives various operations from the user U. For example, the input unit 13 has buttons for inputting characters, numbers, etc. The input unit 13 may also be an input / output port (I / O port) or a USB (Universal Serial Bus) port. If the display unit 12 is a touch panel display, a part of the display unit 12 functions as the input unit 13. The input unit 13 may also be a microphone that receives voice input from the user U. The microphone may be wireless.

[0040] (Positioning unit 14) The positioning unit 14 receives signals (radio waves) transmitted from GPS (Global Positioning System) satellites and, based on the received signals, acquires position information (e.g., latitude and longitude) indicating the current position of the terminal device 10. In other words, the positioning unit 14 determines the position of the terminal device 10. Note that GPS is just one example of a GNSS (Global Navigation Satellite System).

[0041] Furthermore, the positioning unit 14 can determine its position using various methods other than GPS. For example, the positioning unit 14 may use various communication functions of the terminal device 10 to determine its position as an auxiliary positioning means for position correction, etc., as described below.

[0042] (Wi-Fi positioning) For example, the positioning unit 14 determines the location of the terminal device 10 by utilizing the Wi-Fi® communication function of the terminal device 10 and the communication network provided by each telecommunications company. Specifically, the positioning unit 14 determines the location of the terminal device 10 by performing Wi-Fi communication, etc., and determining the distance to nearby base stations and access points.

[0043] (Beacon positioning) Furthermore, the positioning unit 14 may determine the location using the Bluetooth® function of the terminal device 10. For example, the positioning unit 14 determines the location of the terminal device 10 by connecting to a beacon transmitter connected via the Bluetooth® function.

[0044] (Geomagnetic positioning) Furthermore, the positioning unit 14 determines the position of the terminal device 10 based on the geomagnetic pattern of the structure, which has been measured in advance, and the geomagnetic sensor provided by the terminal device 10.

[0045] (RFID positioning) Furthermore, if, for example, the terminal device 10 is equipped with an RFID (Radio Frequency Identification) tag function equivalent to that of a contactless IC card used at a train station ticket gate or in a store, or if it is equipped with a function to read RFID tags, the location where it was used will be recorded along with the information on the payment or other transactions made by the terminal device 10. The positioning unit 14 may determine the location of the terminal device 10 by acquiring such information. Alternatively, the location may be determined by an optical sensor or infrared sensor equipped in the terminal device 10.

[0046] The positioning unit 14 may, if necessary, determine the position of the terminal device 10 using one or a combination of the positioning means described above.

[0047] (Sensor unit 20) The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. The connection can be wired or wireless. For example, the sensors may be detection devices other than the terminal device 10, such as wearable devices or wireless devices. In the example shown in Figure 2, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, a barometric pressure sensor 23, a temperature sensor 24, a sound sensor 25, a light sensor 26, a magnetic sensor 27, and an image sensor (camera) 28.

[0048] The sensors 21-28 described above are merely examples and not limiting. In other words, the sensor unit 20 may be configured to include some of the sensors 21-28, or it may include other sensors such as humidity sensors in addition to or instead of the sensors 21-28.

[0049] The acceleration sensor 21 is, for example, a 3-axis acceleration sensor and detects the physical movement of the terminal device 10, such as its direction of movement, velocity, and acceleration. The gyro sensor 22 detects the physical movement of the terminal device 10, such as its tilt in the three axes, based on its angular velocity. The barometric pressure sensor 23 detects the atmospheric pressure around the terminal device 10, for example.

[0050] Since the terminal device 10 is equipped with the acceleration sensor 21, gyroscope 22, barometric pressure sensor 23, etc., it becomes possible to determine the position of the terminal device 10 using technologies such as pedestrian dead-reckoning (PDR) that utilize these sensors 21 to 23. This makes it possible to obtain indoor location information that is difficult to obtain with positioning systems such as GPS.

[0051] For example, a pedometer using an accelerometer 21 can calculate the number of steps, walking speed, and distance walked. Additionally, a gyroscope 22 can be used to determine the user U's direction of movement, gaze direction, and body tilt. Furthermore, the barometric pressure detected by the barometric pressure sensor 23 can be used to determine the altitude and floor number of the user U's terminal device 10.

[0052] The temperature sensor 24 detects, for example, the ambient temperature around the terminal device 10. The sound sensor 25 detects, for example, the ambient sound around the terminal device 10. The light sensor 26 detects the ambient illumination around the terminal device 10. The magnetic sensor 27 detects, for example, the Earth's magnetic field around the terminal device 10. The image sensor 28 captures an image of the area around the terminal device 10.

[0053] The aforementioned pressure sensor 23, temperature sensor 24, sound sensor 25, light sensor 26, and image sensor 28 can detect the surrounding environment and conditions of the terminal device 10 by detecting atmospheric pressure, temperature, sound, and illuminance, respectively, and by capturing images of the surroundings. Furthermore, it becomes possible to improve the accuracy of the location information of the terminal device 10 based on the surrounding environment and conditions.

[0054] (Control Unit 30) The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit) or MPU (Micro Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), input / output ports, and various circuits. Alternatively, the control unit 30 may be composed of hardware such as an integrated circuit (ASIC) or FPGA (Field Programmable Gate Array). The control unit 30 includes a transmission unit 31, a reception unit 32, and a processing unit 33.

[0055] (Transmitter 31) The transmission unit 31 can transmit various information, such as information input by the user U using the input unit 13, various information detected by sensors 21-28 mounted on or connected to the terminal device 10, and location information of the terminal device 10 determined by the positioning unit 14, to the server device 100 via the communication unit 11.

[0056] (Receiving unit 32) The receiving unit 32 can receive various information provided by the server device 100, as well as requests for various information from the server device 100, via the communication unit 11.

[0057] (Processing 33) The processing unit 33 controls the entire terminal device 10, including the display unit 12. For example, the processing unit 33 can output and display various information transmitted by the transmission unit 31 and various information received from the server device 100 by the reception unit 32 to the display unit 12.

[0058] (Storage unit 40) The storage unit 40 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), and optical discs. Various programs and various data are stored in this storage unit 40.

[0059] [4. Example of Server Device Configuration] Next, the configuration of the server device 100 according to the embodiment will be described using Figure 3. Figure 3 is a diagram showing an example of the configuration of the server device 100 according to the embodiment. As shown in Figure 3, the server device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0060] (Communications Department 110) The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to the network N by wire or wireless connection.

[0061] (Storage unit 120) The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as HDDs, SSDs, and optical discs. For example, the storage unit 120 stores information transmitted and received via the communication unit 110. The storage unit 120 may also store user U's attribute information and history information (log data) along with identification information (user ID, etc.) indicating user U.

[0062] (Control unit 130) The control unit 130 is a controller, and is realized by executing various programs (corresponding to an example of an information processing program) stored in the internal storage device of the server device 100 using a storage area such as RAM as a working area, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array). In the example shown in Figure 3, the control unit 130 has an acquisition unit 131, an estimation unit 132, a generation unit 133, an addition unit 134, and a provision unit 135.

[0063] (Acquisition part 131) The acquisition unit 131 acquires the search query entered by the user U. For example, when the user U enters a search query into a search engine or the like and performs a keyword search, the acquisition unit 131 acquires the search query via the communication unit 110. In other words, the acquisition unit 131 acquires the keyword entered by the user U into the search box of a search engine, website, or application via the communication unit 110.

[0064] Furthermore, the acquisition unit 131 acquires user information about user U via the communication unit 110. For example, the acquisition unit 131 acquires identification information (such as user ID), location information, and attribute information of user U from user U's terminal device 10. The acquisition unit 131 may also acquire identification information and attribute information of user U when user U is registered. The acquisition unit 131 then stores the user information in the storage unit 120.

[0065] Furthermore, the acquisition unit 131 acquires various historical information (log data) indicating the user U's actions via the communication unit 110. For example, the acquisition unit 131 acquires various historical information indicating the user U's actions from the user U's terminal device 10, or from various servers based on the user ID, etc. The acquisition unit 131 then stores the various historical information in the storage unit 120.

[0066] Furthermore, the acquisition unit 131 acquires screenshots taken by user U via the communication unit 110. For example, the acquisition unit 131 acquires screenshots taken by user U from user U's terminal device 10 or from an external server to which user U has uploaded screenshots. The acquisition unit 131 then stores the screenshots in the storage unit 120. The acquisition unit 131 may also automatically archive the acquired screenshots.

[0067] (Estimation part 132) The estimation unit 132 estimates user U's profile information. For example, the estimation unit 132 estimates user U's interests and user persona (user profile) from user U's attribute information and history information.

[0068] (Generation unit 133) The generation unit 133 generates additional information (user-specific additional information) regarding the screenshot, based on the information related to the acquired screenshot and the profile information of user U. For example, the generation unit 133 automatically generates keywords or topics related to the screenshot as tags. In particular, the generation unit 133 automatically generates keywords or topics that user U is interested in or is likely to be interested in, based on the user U's profile information, as tags. The generation unit 133 also generates a personalized summary of the content of the screenshot according to the user U's profile information.

[0069] In this case, the generation unit 133 may generate a prompt (instruction text) requesting the generation of additional information corresponding to user U's profile information regarding the screenshot, input the screenshot, user U's profile information, and the prompt into the AI, and obtain additional information corresponding to user U's profile information. Alternatively, the generation unit 133 may search for additional information corresponding to user U's profile information via the communication unit 110. For example, the generation unit 133 may estimate the necessary information according to user U's profile information and search for the necessary information based on that estimation.

[0070] (Additional part 134) The addition unit 134 adds additional information to the screenshot. For example, the addition unit 134 adds generated tags to the screenshot image. The addition unit 134 also embeds interactive links in the screenshot, allowing direct access to relevant content or data according to user U's profile information. The addition unit 134 may also embed interactive links in the tags, allowing access to other screenshots with the same tags. The addition unit 134 also adds personalized summaries to the screenshot image.

[0071] (Provider 135) The provision unit 135 provides user U with the added information along with screenshots. The provision unit 135 also provides comments or ratings from other users U who have taken screenshots of the same content, along with their screenshots, thereby providing user U with community insights through the screenshots. The provision unit 135 also automatically archives the additional information along with the screenshots, making it accessible later.

[0072] [5. Processing Procedure] Next, the processing procedure by the server device 100 according to the embodiment will be described using Figure 4. Figure 4 is a flowchart of the processing procedure according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 130 of the server device 100.

[0073] For example, as shown in Figure 4, a screenshot taken by user U is acquired via the communication unit 110 of the server device 100 (step S101).

[0074] Next, the estimation unit 132 of the server device 100 estimates user U's profile information, such as user U's interests and user persona (user profile), from user U's attribute information and history information (step S102).

[0075] Next, the generation unit 133 of the server device 100 generates additional information (user-specific additional information) regarding the screenshot, based on the information related to the acquired screenshot and the profile information of user U (step S103).

[0076] Next, the generation unit 133 of the server device 100 automatically generates additional information, such as keywords or topics related to the screenshot, as tags (step S104).

[0077] Next, the generation unit 133 of the server device 100 generates a personalized summary of the target content of the screenshot as additional information, according to the user U's profile information (step S105).

[0078] Next, the add-on unit 134 of the server device 100 adds the generated additional information to the screenshot (step S106).

[0079] Next, the add-on 134 of the server device 100 embeds an interactive link in the screenshot, allowing direct access to relevant content or data corresponding to the user U's profile information (step S107).

[0080] Next, the provision unit 135 of the server device 100 provides the added information to user U along with a screenshot (step S108).

[0081] Next, the provision unit 135 of the server device 100 provides comments or ratings from other users U who have taken screenshots of the same content, along with the screenshots, thereby providing users U with community insights through the screenshots (step S109).

[0082] Next, the provision unit 135 of the server device 100 automatically archives the additional information along with screenshots, making it accessible later (step S110).

[0083] [6. Variant Example] The terminal device 10 and server device 100 described above may be implemented in various other forms besides those of the embodiment described above. Therefore, the following describes modifications of the embodiment.

[0084] In the above embodiment, some or all of the processing performed by the server device 100 may actually be performed by the terminal device 10 (or an application running on the terminal device 10). For example, the terminal device 10 may perform all processing in a standalone manner. In this case, the terminal device 10 is assumed to have the same functions as the server device 100 in the above embodiment. Furthermore, in the above embodiment, since the terminal device 10 is in cooperation with the server device 100, from the perspective of the user U, it appears as if the processing of the server device 100 is also being performed by the terminal device 10. In other words, from another perspective, it can be said that the terminal device 10 is equipped with the server device 100.

[0085] Furthermore, in the above embodiment, the server device 100 may be a container engine that constructs an execution environment (container) for virtually running applications on a host computer. For example, the server device 100 may construct a container for executing various functions on the user U's terminal device 10.

[0086] Furthermore, in the above embodiment, the server device 100 may vectorize the text of the screenshot and the text of the additional information using image recognition and natural language processing (NLP), and determine whether they match based on the cosine similarity of each vector. The server device 100 may then adopt the additional information if there is a match, and discard it if there is no match.

[0087] Furthermore, in the above embodiment, the server device 100 may provide additional information based on the profile and interests of user U when user U views a screenshot taken by another user. That is, the server device 100 may also provide additional information based on user U's profile and interests for screenshots other than those taken by user U. Conversely, when another user views a screenshot taken by user U, the server device 100 may provide additional information based on the profile and interests of the other user viewing the screenshot.

[0088] Furthermore, in the above embodiments, the additional information is not limited to text, but may also be images such as photographs, illustrations, or videos, or audio. For example, the additional information may be an explanation of the contents of a screenshot using images or audio.

[0089] [7. Effects] As described above, the information processing device (terminal device 10 and server device 100) according to the present invention is characterized by comprising: an acquisition unit that acquires screenshots taken by user U; an estimation unit 132 that estimates user U's profile information; a generation unit 133 that generates additional information corresponding to user U's profile information regarding the screenshot based on the acquired screenshot information and user U's profile information; an addition unit 134 that adds the additional information to the screenshot; and a provision unit 135 that provides the added additional information to user U along with the screenshot.

[0090] This allows additional information tailored to the user's profile to be added when taking a screenshot. For example, when taking a screenshot of an economics article, market data and expert comments of interest to the user can be automatically added.

[0091] The generation unit 133 automatically generates keywords or topics related to the screenshot as tags. The addition unit 134 adds the generated tags to the screenshot image.

[0092] This makes it easy to search for relevant information when reviewing screenshots later.

[0093] The add-on 134 embeds interactive links in the screenshot, allowing direct access to relevant content or data corresponding to the user U's profile information.

[0094] This allows users to explore further information through screenshots.

[0095] The generation unit 133 generates a personalized summary of the content of the screenshot according to the user U's profile information. The addition unit 134 adds the personalized summary to the screenshot image.

[0096] This allows users to easily grasp key points according to their own interests and knowledge levels.

[0097] The provision unit 135 provides comments or ratings from other users U who have taken screenshots of the same content, along with the screenshots, thereby providing users U with community insights through the screenshots.

[0098] This facilitates information sharing among users through screenshots, making it possible to gain diverse perspectives.

[0099] The data provisioning unit 135 automatically archives additional information along with screenshots, making it accessible later.

[0100] This allows you to access archived screenshots from the past later. Additionally, the archives will include extra information and links, making it easier to reuse the information.

[0101] By any or a combination of the above-described processes, the information processing device according to the present invention can provide additional information tailored to the user's profile when taking a screenshot.

[0102] [8. Hardware Configuration] Furthermore, the terminal device 10 and server device 100 according to the above-described embodiment are realized by a computer 1000 having a configuration such as that shown in Figure 5. The following explanation will use the server device 100 as an example. Figure 5 is a diagram showing an example of the hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a primary storage device 1040, a secondary storage device 1050, an output interface 1060, an input interface 1070, and a network interface 1080 are connected by a bus 1090.

[0103] The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, as well as programs read from the input device 1020, and executes various processes. The arithmetic unit 1030 can be implemented using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array).

[0104] The primary storage device 1040 is a memory device, such as RAM (Random Access Memory), that temporarily stores data used by the arithmetic unit 1030 for various calculations. The secondary storage device 1050 is a storage device where data used by the arithmetic unit 1030 for various calculations and various databases are registered, and can be implemented using ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. The secondary storage device 1050 may be internal storage or external storage. The secondary storage device 1050 may also be a removable storage medium such as USB (Universal Serial Bus) memory or SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), NAS (Network Attached Storage), file server, etc.

[0105] The output I / F 1060 is an interface for transmitting information to be output to output devices 1010, such as displays, projectors, and printers, and is implemented using connectors of standards such as USB (Universal Serial Bus), DVI (Digital Visual Interface), and HDMI (High Definition Multimedia Interface). The input I / F 1070 is an interface for receiving information from various input devices 1020, such as mice, keyboards, keypads, buttons, and scanners, and is implemented using, for example, USB.

[0106] Furthermore, the output interface 1060 and input interface 1070 may be wirelessly connected to the output device 1010 and input device 1020, respectively. In other words, the output device 1010 and input device 1020 may be wireless devices.

[0107] Furthermore, the output device 1010 and the input device 1020 may be integrated as a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated as an input / output I / F.

[0108] The input device 1020 may also be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0109] The network interface 1080 receives data from other devices via network N and sends it to the computing unit 1030, and also transmits data generated by the computing unit 1030 to other devices via network N.

[0110] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output interface 1060 and the input interface 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0111] For example, when computer 1000 functions as a server device 100, the arithmetic unit 1030 of computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the primary storage device 1040. Alternatively, the arithmetic unit 1030 of computer 1000 may load a program obtained from another device via the network interface 1080 onto the primary storage device 1040 and execute the loaded program. Furthermore, the arithmetic unit 1030 of computer 1000 may cooperate with other devices via the network interface 1080 and call and use program functions, data, etc., from other programs on other devices.

[0112] [9. Other] Although embodiments of the present invention have been described above, the present invention is not limited by the content of these embodiments. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above.

[0113] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0114] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0115] For example, the server device 100 described above may be implemented using multiple server computers, and the configuration can be flexibly changed, such as by calling external platforms via APIs (Application Programming Interfaces) or network computing depending on the function.

[0116] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0117] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit," etc. For example, the acquisition unit can be replaced with acquisition means or acquisition circuit. [Explanation of Symbols]

[0118] 1. Information Processing System 10 Terminal devices 100 Server Devices 110 Communications Department 120 Storage section 130 Control Unit 131 Acquisition Department 132 Estimation Department 133 Generation part 134 Addition part 135 Provision Department

Claims

1. A unit that acquires screenshots taken by the user, An estimation unit that estimates the user's profile information, A generation unit generates additional information relating to the screenshot that corresponds to the user's profile information, based on the information relating to the acquired screenshot and the user's profile information. An addition unit that adds the additional information to the aforementioned screenshot, A provisioning unit that provides the user with the added information along with the screenshot, An information processing device characterized by comprising:

2. The generation unit automatically generates keywords or topics related to the screenshot as tags, The addition unit adds the generated tag to the screenshot image. The information processing apparatus according to feature 1.

3. The aforementioned addition embeds interactive links in the screenshot, allowing direct access to relevant content or data corresponding to the user's profile information. The information processing apparatus according to feature 1.

4. The generation unit generates a personalized summary of the target content of the screenshot according to the user's profile information. The addition unit adds the personalized summary to the screenshot image. The information processing apparatus according to feature 1.

5. The aforementioned provision unit provides comments or ratings from other users who have taken screenshots of the same content, along with the screenshots, and provides community insights to the user through the screenshots. The information processing apparatus according to feature 1.

6. The providing unit automatically archives the additional information along with the screenshots, making it accessible later. The information processing apparatus according to feature 1.

7. An information processing method performed by an information processing device, The process of obtaining screenshots taken by the user, An estimation step for estimating the user's profile information, A generation step that generates additional information corresponding to the user's profile information with respect to the screenshot, based on the information related to the acquired screenshot and the user's profile information, An additional step of adding the additional information to the aforementioned screenshot, A provision step of providing the user with the added information along with the screenshot, An information processing method characterized by including

8. The procedure for obtaining screenshots taken by users, An estimation procedure for estimating the user's profile information, A generation procedure for generating additional information corresponding to the user's profile information with respect to the screenshot, based on the information related to the acquired screenshot and the user's profile information, The procedure for adding the additional information to the aforementioned screenshot, A provision procedure for providing the user with the added information along with the screenshot, An information processing program characterized by causing a computer to execute it.

Images