Information processing apparatus, information processing method, and information processing program

Abstract

To allow a server to acquire a screenshot and perform an action resulting from the screenshot.SOLUTION: An information processing apparatus includes: an acquisition unit which acquires a screenshot from a user; a generation unit which merges a selected comment selected by the user from among candidate comments related to a target of the screenshot, with the screenshot to generate one piece of content; and a providing unit which provides the generated content to another user.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] With the spread of social networking services (SNS), a technique for sharing generated screenshots with other users has been disclosed (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-033386 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned conventional techniques, screenshots are shared directly between users, and there is little room for the server side to be involved.

[0005] The present application has been made in view of the above, and aims to acquire a screenshot on the server side and perform an action triggered by the screenshot. [Means for solving the problem]

[0006] The information processing device of the present application is characterized by comprising an acquisition unit that acquires a screenshot from a user, a generation unit that merges a selected comment selected by the user from candidate comments related to the subject of the screenshot with the screenshot to generate one piece of content, and a provision unit that provides the generated content to other users. [Effects of the Invention]

[0007] According to one aspect of the embodiment, a screenshot can be taken on the server side and an action can be performed based on the screenshot. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing an overview of an information processing system according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing an overview of comment display caused by a screenshot. [Figure 3] FIG. 3 is an explanatory diagram showing an overview of inquiries to experts caused by screenshots. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of a terminal device according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of the configuration of a server device according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing a first processing procedure according to the embodiment. [Figure 7] FIG. 7 is a flowchart showing a second processing procedure according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an information processing device, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to these embodiments. Furthermore, the same components in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] [1. Overview of the information processing system] First, an overview of an information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram showing an overview of an information processing system according to an embodiment. As shown in Fig. 1, an information processing system 1 according to an embodiment includes a terminal device 10 and a server device 100. These various devices are connected to each other via a network N in a wired or wireless manner so as to be able to communicate with each other. This enables the terminal device 10 to cooperate with the server device 100. The network N is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, etc.

[0011] Terminal device 10 is an information processing device used by a user U. For example, terminal device 10 may be a smart device such as a smartphone or tablet terminal, a mobile phone such as a feature phone (Gala-ke or Gala-ho), a personal computer (PC), a personal digital assistant (PDA), a game console or AV device with communication functions, an information appliance or digital appliance, a car navigation system, a wearable device such as a smart watch, a head-mounted display, or smart glasses. Terminal device 10 may also be a house or building, a car, a home appliance, an electronic device, or the like that is compatible with the Internet of Things (IOT).

[0012] In this embodiment, the terminal device 10 is a smart device such as a smartphone or tablet used by a user U, and is a portable terminal device capable of communicating with any server device via a wireless communication network such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation: fifth generation mobile communication system), Bluetooth (registered trademark), or wireless LAN. The terminal device 10 has a screen such as a liquid crystal display with a touch panel function, and accepts various operations on displayed data such as content, such as tapping, sliding, and scrolling, performed by the user U with a finger or a stylus. Note that an operation performed on an area of ​​the screen where content is displayed may be considered an operation on the content. The terminal device 10 may be not only a smart device, but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC.

[0013] The server device 100 is, for example, a computer such as a PC or a blade server, or a mainframe or a workstation, etc. The server device 100 may be realized by cloud computing.

[0014] In this embodiment, the server device 100 is an information processing device that works in conjunction with the terminal device 10 of each user U and provides API (Application Programming Interface) services for various applications (hereinafter referred to as apps) and various data to the terminal device 10 of each user U, and is realized by a computer, a cloud system, etc.

[0015] The server device 100 may also be an information processing device that provides some kind of online service to the terminal device 10 of each user U. For example, the server device 100 may provide the following online services: internet connection, search service, social networking service (SNS), electronic commerce (EC), electronic payment, online games, online banking, online trading, hotel and ticket reservations, video and music distribution, news, maps, route search, route guidance, line information, operation information, and weather forecast. In practice, the server device 100 may cooperate with various servers that provide the above-mentioned online services and act as an intermediary for the online services or may be responsible for processing the online services.

[0016] The server device 100 can acquire user information about the user U. For example, the server device 100 acquires, as the user information, information (attribute information) about the attributes of the user U, such as the gender, age, and residential area of ​​the user U. The server device 100 can also acquire information about the attributes of the user U, such as demographic attributes, psychographic attributes, geographic attributes, and behavioral attributes. The server device 100 may also acquire, as the user information, a segment to which the user U belongs in the marketing field or a persona (personality). The server device 100 then stores and manages the information (attribute information) about the attributes of the user U together with identification information (such as a user ID) that identifies the user U.

[0017] The server device 100 also acquires various types of history information (log data) indicating the behavior of the user U from the terminal device 10 of the user U or from various servers based on the user ID, etc. For example, the server device 100 acquires a location history, which is a history of the user U's location and date and time, from the terminal device 10. The server device 100 also acquires a search history, which is a history of search queries entered by the user U, from a search server (search engine). The server device 100 also acquires a browsing history, which is a history of content viewed by the user U, from a content server. The server device 100 also acquires a purchase history (payment history), which is a history of the user U's product purchases and payment processes, from an e-commerce server or a payment processing server. The server device 100 may also acquire a listing history and a sales history, which are a history of the user U's listings on the marketplace, from the e-commerce server or the payment processing server. The server device 100 also acquires a posting history, which is a history of the user U's posts, from a posting server or SNS server that provides a word-of-mouth posting service. The various servers and the like described above may be the server device 100 itself. That is, the server device 100 may function as the various servers and the like described above.

[0018] Furthermore, the number of devices included in the information processing system 1 shown in Fig. 1 is not limited to that shown in the figure. For example, in Fig. 1, for the sake of simplicity, only one terminal device 10 is shown, but this is merely an example and is not limiting, and two or more devices may be included.

[0019] [2. Use of screenshots] In recent years, as a means of sharing online information among young people, screenshots, which place less of a burden on the other party's packet communication, have become popular, replacing the issuance of links (presentation of link destinations) using share buttons and the like. However, currently, screenshots are shared between user terminals, and the server side has little opportunity to intervene. In this embodiment, the server device 100 intervenes in the sharing of screenshots between terminal devices 10 and takes action in response to the screenshots.

[0020] [2-1. Group formation due to screenshots] The server device 100 groups users who have taken screenshots of the same news article and forms a closed community (such as an open chat) that only users in the same group can participate in. In other words, the server device 100 may group users based on the commonality or similarity of the news articles they have taken screenshots of, and form a closed community that only users in the same group can participate in.

[0021] The server device 100 may also provide a service that allows users in the same group to post comments on the contents of the screenshot (the news article) and a service that allows users in the same group to exchange information.

[0022] For example, as shown in Fig. 1, the server device 100 acquires (collects) screenshots of news articles on a news site / app from the terminal device 10 of the user U via the network N (step S1). At this time, the server device 100 acquires screenshots of parts of the news articles that interest the user U (or that are related to the user U's doubts, worries, or questions). At the time the screenshot is taken, it is assumed that the user U is quite interested in the content of that part (the focused part) of the news article.

[0023] In practice, when user U takes a screenshot of a news article on terminal device 10, the screenshot may be automatically sent from terminal device 10 to server device 100 using a function of an app or the like, or user U may be asked whether or not to send the screenshot to server device 100, and the screenshot may be sent to server device 100 if user U gives permission.

[0024] Next, the server device 100 determines whether the screenshots of the news articles acquired from the terminal devices 10 of the users are the same (step S2).

[0025] Next, the server device 100 groups users who are determined to have the same screenshot of the news article (step S3).

[0026] Next, the server device 100 forms a closed community (open chat, etc.) in which only users in the same group can participate (step S4).

[0027] Next, the server device 100 provides an information sharing service within the closed community (step S5).

[0028] [2-2. Comment display due to screenshot] Next, comment display caused by a screenshot will be described with reference to Fig. 2. Fig. 2 is an explanatory diagram showing an overview of comment display caused by a screenshot.

[0029] For example, as shown in FIG. 2, the server device 100 acquires a screenshot of a news article on a news site / app that allows comments to be posted from the terminal device 10 of the user U via the network N (step S11).

[0030] Note that news sites / apps are just one example. In reality, it could be social networking sites, Q&A sites, life advice sites, specialized field sites, video / music distribution sites, novel / illustration posting sites, or any other site / app that allows users to post comments.

[0031] Next, the server device 100 extracts comments related to the content of the news article from among the comments posted to the news article on the news site / app as candidate comments (step S12). That is, the server device 100 excludes comments unrelated to the content of the news article (and comments with little relevance) from among the comments posted to the news article.

[0032] At this time, when the server device 100 acquires a screenshot of a news article, it may identify the news site / app that published the news article. That is, the server device 100 may identify the news site / app that is open (running) on ​​the terminal device 10 of the user U when the screenshot is taken. When the server device 100 provides news articles to the terminal device 10 of the user U, it is easy to identify the source of the news article.

[0033] The server device 100 may also use AI (Artificial Intelligence) to learn the news site / app and the form or format of the news article as learning data, and use this AI to estimate the news site / app that published the news article. Alternatively, the server device 100 may obtain, via the network N, from the terminal device 10 of the user U, a screenshot of the news article along with identification information indicating the news site / app that published the article.

[0034] In addition, the server device 100 may select other users (experts, experienced people, etc.) who are expected to be able to make appropriate comments on the news article as respondents, send screenshots to the selected respondents, and request them to post comments, and the comments obtained may be used as candidate comments.

[0035] Next, the server device 100 provides (presents) the candidate comments to the terminal device 10 of the user U via the network N (step S13). At this time, the user U selects an appropriate comment (such as a comment that matches the user U's intention) from the candidate comments. For example, the user U selects a comment related to his / her interests, concerns, worries, questions, etc.

[0036] Next, the server device 100 generates content that can be commented on by merging the screenshot of the news article with a comment (selected comment) selected by the user U from the candidate comments (step S14). Note that there may be multiple selected comments. The server device 100 may merge the screenshot of the news article with the selected comment to create a single image. This allows comments from experts and other users to be added to the screenshot.

[0037] At this time, the server device 100 may display the comment in the content at a position related to the screenshot of the news article. For example, the server device 100 generates content in which the comment is placed at a position of a sentence or image (such as an object indicated by the comment) corresponding to the comment in the screenshot of the news article.

[0038] In addition, the server device 100 may estimate which parts of the screenshotted news article are likely to be of particular interest to the user U (such as parts that mention specific themes or keywords), and may highlight those parts in the content and provide information about those parts.

[0039] Next, the server device 100 provides the generated content to other users designated by the user U (step S15). This allows the user U to share the generated content with desired users. At this time, the server device 100 may provide the content to users in the same group as the user U.

[0040] [2-3. Inquiry to an expert regarding screenshots] Next, an inquiry to an expert caused by a screenshot will be described with reference to Fig. 3. Fig. 3 is an explanatory diagram showing an overview of an inquiry to an expert caused by a screenshot.

[0041] For example, as shown in FIG. 3, the server device 100 acquires a screenshot of a news article from a news site / app that allows comments to be posted and a question text from the terminal device 10 of the user U via the network N (step S21).

[0042] Next, the server device 100 estimates the intention of the question of the user U from the acquired screenshot and question text (step S22). That is, the server device 100 estimates which part of the news article of the screenshot that the user U was interested in and what intention the user had in asking the question.

[0043] At this time, the server device 100 may further clarify the user U's points of interest and sympathy in the news article from the user U's past searches, articles viewed, article ratings, etc., and infer the intention of the user U's question.

[0044] Furthermore, the server device 100 may infer the intention of the question using a generative AI such as a Generative Pretrained Transformer (GPT). For example, the server device 100 may input a prompt including the acquired screenshot and question text into the GPT, and obtain the intention of the question inferred by the GPT.

[0045] Next, the server device 100 selects an appropriate expert as a respondent based on the content of the screenshot and the intent of the question (or the question sentence) (step S23). That is, the server device 100 matches experts in the specialized fields (areas) related to the screenshotted portion. Note that multiple experts may be selected. For example, if the screenshotted portion (or the portion thereof that is expected to be of particular interest to the user U) relates to multiple specialized fields, the server device 100 may match experts for each specialized field.

[0046] Furthermore, the server device 100 may determine whether to ask an expert or to provide an answer using GPT based on the content of the screenshot and the intent of the question (or the question text). If the server device 100 determines to provide an answer using GPT, it may input a prompt including the content of the screenshot and the intent of the question (or the question text) into GPT and obtain an answer generated by GPT.

[0047] Furthermore, the server device 100 may have a prediction AI learn pairs of the content of the screenshot, the intent of the question, and the expert as learning data, and use this prediction AI to select an expert based on the content of the screenshot and the intent of the question.

[0048] Furthermore, experts may be associated in advance with the content (field, theme, etc.) of the screenshot. Furthermore, the server device 100 may select an expert in a field corresponding to a pair of a screenshot and a question sentence. Furthermore, the server device 100 may select an expert in a field corresponding to a portion (part) screenshotted by a plurality of users (or a predetermined number of users or more).

[0049] Next, the server device 100 provides the screenshot and the question text to the selected expert via the network N, and obtains an answer from the expert (step S24).

[0050] At this time, the server device 100 may simultaneously send the content and link of the original text of the screenshot (such as a news article) to the expert along with the screenshot and the question text.

[0051] Furthermore, the server device 100 may pay a reward (points, etc.) to the expert who provides the answer. When the expert provides an answer, the expert receives a reward (points, etc.).

[0052] Next, the server device 100 provides the answer to the terminal device 10 of the user U via the network N (step S25). At this time, the server device 100 may present to the user U, together with the answer, information regarding the name, field of expertise, rank of expertise (trust rank of the answer), etc. of the expert who provided the answer. The expert who provided the answer can appeal to the user U about the high level of his or her expertise and can increase his or her recognition as an expert.

[0053] Next, the server device 100 merges the screenshot and the answer to create an article (step S26). That is, the server device 100 generates content by merging the screenshot and the answer. At this time, the server device 100 may provide the generated content to other users designated by the user U. When the content is made available to an unspecified number of users, the expert who provided the answer can appeal to users who view the content about his or her high level of expertise and can increase his or her recognition as an expert.

[0054] Furthermore, when generating the content, the server device 100 may generate the content using the same method as the comment display described in 2-2 above. In this case, the answer from the expert is treated as a posted comment.

[0055] [2-3-1. Supplementary Information] The server device 100 may store sets of screenshots, question sentences, and answers as history information (log data). At this time, the history information may be stored for a fixed period of time, and after the fixed period has elapsed, the oldest history information may be deleted.

[0056] Furthermore, when acquiring a screenshot and a question text, the server device 100 may refer to history information of previously acquired screenshots and question texts, and determine whether the acquired screenshot and question text are the same as those previously acquired. If the acquired screenshot and question text are the same as those previously acquired, the server device 100 may extract an answer to the same screenshot and question text from the past history information and provide the extracted answer to the terminal device 10 of the user U via the network N, in order to avoid repeatedly asking the same question to the expert.

[0057] In addition, the server device 100 may estimate the suitability of user U from a combination of the screenshot, the source of the screenshot (such as a news article), and the question text, and if it estimates that user U is not suitable, it may not allow the user U to make an inquiry to an expert.

[0058] Furthermore, the server device 100 may infer from the screenshot and the question text whether the user U has a serious intention to ask a question. If the server device 100 infers that the user U does not have a serious intention to ask a question, but is asking a question with the intention (malicious) of mischief, teasing, trolling, or the like, the server device 100 may notify the terminal device 10 of the user U of a warning against the act via the network N, discard the screenshot and the question text, and end the process.

[0059] The server device 100 may also determine whether the screenshotted portion and the content of the question match (whether the degree of match exceeds a threshold), and if they match (the degree of match exceeds a threshold), allow the user to make an inquiry to an expert. The server device 100 may use image recognition and natural language processing (NLP) to determine whether the screenshotted portion and the content of the question match. For example, the server device 100 may use image recognition and natural language processing to vectorize the text of the screenshotted portion and the question, and determine whether they match based on the cosine similarity of the vectors.

[0060] Furthermore, the server device 100 may determine whether the number of users whose screenshots have the same content exceeds a threshold (e.g., 30 or more in the most recent past), and if the threshold is exceeded, may permit an inquiry to an expert. Alternatively, the server device 100 may determine whether the number of times that a screenshotted portion (part) is the same exceeds a threshold, and if the threshold is exceeded, may permit an inquiry to an expert.

[0061] In addition, the server device 100 may determine whether there are multiple experts (two or more) whose expertise matches the area of ​​expertise related to the screenshotted portion, and if there are multiple experts (two or more), allow the user to inquire about the experts.

[0062] Furthermore, the server device 100 may determine whether or not it has been within 24 hours since the original (news article, etc.) of the screenshot was made public, and if it has been within 24 hours since it was made public, may permit an inquiry to the expert.

[0063] In addition, if the server device 100 is unable to obtain screenshots of news articles from the terminal device 10 due to regulations, specifications, etc., some (or all) of the above-mentioned processing of the server device 100 that directly uses screenshots may be performed (acted on behalf of) the terminal device 10 of the user U by launching an app or the like that has the relevant function.

[0064] [3. Example of terminal device configuration] Next, the configuration of the terminal device 10 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the terminal device 10 according to the embodiment. As shown in Fig. 4, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40.

[0065] (Communications Department 11) The communication unit 11 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the server device 100 via the network N. For example, the communication unit 11 is realized by a NIC (Network Interface Card), an antenna, etc.

[0066] (Display section 12) Display unit 12 is a display device that displays various information such as position information. For example, display unit 12 is a liquid crystal display (LCD) or an organic electro-luminescent display (OLED). Display unit 12 is also a touch panel display, but is not limited to this.

[0067] (Input section 13) The input unit 13 is an input device that accepts various operations from the user U. For example, the input unit 13 has buttons for inputting characters, numbers, etc. The input unit 13 may be an input / output port (I / O port), a USB (Universal Serial Bus) port, etc. If the display unit 12 is a touch panel display, a part of the display unit 12 functions as the input unit 13. The input unit 13 may be a microphone that accepts voice input from the user U. The microphone may be wireless.

[0068] (Positioning unit 14) The positioning unit 14 receives signals (radio waves) transmitted from satellites of a GPS (Global Positioning System), and acquires position information (e.g., latitude and longitude) indicating the current position of the terminal device 10, which is the device itself, based on the received signals. That is, the positioning unit 14 positions the position of the terminal device 10. Note that GPS is merely an example of a GNSS (Global Navigation Satellite System).

[0069] The positioning unit 14 can also measure the position using various methods other than GPS. For example, the positioning unit 14 may measure the position by using various communication functions of the terminal device 10 as an auxiliary positioning means for position correction, etc., as described below.

[0070] (Wi-Fi positioning) For example, the positioning unit 14 uses a Wi-Fi (registered trademark) communication function of the terminal device 10 or a communication network provided by each communication company to measure the position of the terminal device 10. Specifically, the positioning unit 14 performs Wi-Fi communication or the like and measures the distance to a nearby base station or access point, thereby measuring the position of the terminal device 10.

[0071] (Beacon positioning) The positioning unit 14 may also measure the position by using a Bluetooth (registered trademark) function of the terminal device 10. For example, the positioning unit 14 measures the position of the terminal device 10 by connecting to a beacon transmitter connected by the Bluetooth (registered trademark) function.

[0072] (geomagnetic positioning) The positioning unit 14 also measures the position of the terminal device 10 based on a geomagnetic pattern of a structure that has been measured in advance and a geomagnetic sensor that the terminal device 10 has.

[0073] (RFID positioning) Furthermore, for example, if the terminal device 10 has a function of an RFID (Radio Frequency Identification) tag equivalent to a contactless IC card used at station ticket gates, in stores, etc., or has a function of reading an RFID tag, the location where the terminal device 10 was used is recorded together with information on the payment or the like made by the terminal device 10. The positioning unit 14 may obtain such information to determine the location of the terminal device 10. Alternatively, the location may be determined by an optical sensor, an infrared sensor, or the like provided in the terminal device 10.

[0074] The positioning unit 14 may measure the position of the terminal device 10 using one or a combination of the above-mentioned positioning means, as needed.

[0075] (Sensor unit 20) The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. The connection may be wired or wireless. For example, the sensors may be detection devices other than the terminal device 10, such as wearable devices or wireless devices. In the example shown in FIG. 4, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, a barometric pressure sensor 23, a temperature sensor 24, a sound sensor 25, a light sensor 26, a magnetic sensor 27, and an image sensor (camera) 28.

[0076] The above-described sensors 21 to 28 are merely examples and are not intended to be limiting. That is, the sensor unit 20 may be configured to include some of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or instead of the sensors 21 to 28.

[0077] The acceleration sensor 21 is, for example, a three-axis acceleration sensor, and detects physical movements of the terminal device 10, such as the direction of movement, speed, and acceleration of the terminal device 10. The gyro sensor 22 detects physical movements of the terminal device 10, such as tilt in three axial directions, based on the angular velocity of the terminal device 10. The air pressure sensor 23 detects, for example, the air pressure around the terminal device 10.

[0078] Since the terminal device 10 includes the acceleration sensor 21, the gyro sensor 22, the atmospheric pressure sensor 23, etc., it is possible to measure the position of the terminal device 10 using a technique such as Pedestrian Dead-Reckoning (PDR) that uses these sensors 21 to 23. This makes it possible to obtain indoor position information that is difficult to obtain using a positioning system such as GPS.

[0079] For example, the number of steps, walking speed, and distance walked can be calculated using a pedometer that uses the acceleration sensor 21. In addition, the direction of travel, line of sight, and body tilt of the user U can be determined using the gyro sensor 22. In addition, the altitude and floor on which the terminal device 10 of the user U is located can be determined from the air pressure detected by the air pressure sensor 23.

[0080] The temperature sensor 24 detects, for example, the temperature around the terminal device 10. The sound sensor 25 detects, for example, the sound around the terminal device 10. The light sensor 26 detects the illuminance around the terminal device 10. The magnetic sensor 27 detects, for example, the geomagnetism around the terminal device 10. The image sensor 28 captures an image around the terminal device 10.

[0081] The above-mentioned air pressure sensor 23, temperature sensor 24, sound sensor 25, light sensor 26, and image sensor 28 can detect the air pressure, temperature, sound, and illuminance, respectively, and capture images of the surroundings, thereby detecting the environment and situation around the terminal device 10. Furthermore, the accuracy of the location information of the terminal device 10 can be improved based on the environment and situation around the terminal device 10.

[0082] (control unit 30) The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM, input / output ports, etc., and various other circuits. The control unit 30 may also be configured with hardware such as an integrated circuit, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 includes a transmitting unit 31, a receiving unit 32, and a processing unit 33.

[0083] (Transmitter 31) The transmission unit 31 can transmit, for example, various information input by the user U using the input unit 13, various information detected by each sensor 21 to 28 mounted on or connected to the terminal device 10, and location information of the terminal device 10 measured by the positioning unit 14 to the server device 100 via the communication unit 11.

[0084] (Receiving unit 32) The receiving unit 32 can receive various types of information provided by the server device 100 and requests for various types of information from the server device 100 via the communication unit 11.

[0085] (Processing unit 33) The processing unit 33 controls the entire terminal device 10, including the display unit 12. For example, the processing unit 33 can output various information transmitted by the transmitting unit 31 and various information received from the server device 100 by the receiving unit 32 to the display unit 12 for display.

[0086] (Storage unit 40) The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disk, etc. The storage unit 40 stores various programs, various data, etc.

[0087] [4. Server device configuration example] Next, the configuration of the server device 100 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the configuration of the server device 100 according to the embodiment. As shown in Fig. 5, the server device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0088] (Communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC), etc. The communication unit 110 is connected to a network N by wire or wirelessly.

[0089] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD, an SSD, an optical disk, etc. The storage unit 120 may store attribute information and history information (log data) of the user U together with identification information (such as a user ID) indicating the user U.

[0090] (control unit 130) The control unit 130 is a controller, and is realized by, for example, a central processing unit (CPU), a micro processing unit (MPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like, by executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the server device 100 using a storage area such as a RAM as a working area. In the example shown in FIG. 5, the control unit 130 has an acquisition unit 131, a determination unit 132, an extraction unit 133, a grouping unit 134, a formation unit 135, an estimation unit 136, a selection unit 137, a generation unit 138, and a provision unit 139.

[0091] (Acquisition part 131) The acquisition unit 131 acquires a search query input by a user U. For example, when the user U inputs a search query into a search engine or the like to perform a keyword search, the acquisition unit 131 acquires the search query via the communication unit 110. That is, the acquisition unit 131 acquires, via the communication unit 110, the keywords input by the user U into the search box of a search engine, website, or app.

[0092] Furthermore, the acquisition unit 131 acquires user information about the user U via the communication unit 110. For example, the acquisition unit 131 acquires identification information (such as a user ID) indicating the user U, location information of the user U, attribute information of the user U, etc. from the terminal device 10 of the user U. Furthermore, the acquisition unit 131 may acquire the identification information indicating the user U, attribute information of the user U, etc. when the user U is registered. Then, the acquisition unit 131 stores the user information in the storage unit 120.

[0093] Furthermore, the acquisition unit 131 acquires various types of history information (log data) indicating the behavior of the user U via the communication unit 110. For example, the acquisition unit 131 acquires various types of history information indicating the behavior of the user U from the terminal device 10 of the user U or from various servers based on the user ID or the like. Then, the acquisition unit 131 stores the various types of history information in the storage unit 120.

[0094] Furthermore, the acquisition unit 131 acquires a screenshot from the user U via the communication unit 110. The acquisition unit 131 also acquires, via the communication unit 110, a comment selected by the user U from among the candidate comments as a selected comment.

[0095] Alternatively, the acquisition unit 131 acquires a screenshot and a question sentence from the user U via the communication unit 110. The acquisition unit 131 also acquires an answer to the question sentence from an expert.

[0096] (Judgment unit 132) The determination unit 132 determines whether the screenshots acquired from multiple users U are of the same subject.

[0097] Furthermore, the determination unit 132 refers to past history information and determines whether the acquired screenshot and question text are the same as screenshots and question texts acquired in the past.

[0098] The determination unit 132 also determines whether the portion of the screenshot taken by the user U matches the content of the question text, and if so, permits the user to make an inquiry to an expert. The determination unit 132 also determines whether the number of users U whose screenshots have the same content exceeds a threshold, and if so, permits the user to make an inquiry to an expert. The determination unit 132 also determines whether there are multiple experts whose area of ​​expertise is the portion of the screenshot taken by the user U, and if there are multiple experts, permits the user to make an inquiry to an expert.

[0099] (Extraction part 133) If the acquired screenshot and question text are the same as a screenshot and question text acquired in the past, the extraction unit 133 extracts an answer to the acquired screenshot and question text from the past history information.

[0100] Furthermore, the extraction unit 133 extracts comments related to the target of the screenshot as candidate comments from among the comments posted for the target of the screenshot.

[0101] (Grouping unit 134) The grouping unit 134 groups users U who have taken screenshots of the same object.

[0102] (Formation part 135) The forming unit 135 forms a closed community in which users in the same group can participate.

[0103] (Estimation part 136) The estimation unit 136 estimates a portion of the screenshot that is likely to be of particular interest to the user U. The estimation unit 136 also estimates the intention of the question from the screenshot and the question text.

[0104] (Selection Department 137) The selection unit 137 selects an expert to answer the question based on the screenshot and the intent of the question posed by the user U. Note that if the portion of the screenshot taken by the user U relates to multiple specialized fields, the selection unit 137 selects an expert for each specialized field.

[0105] (Generation unit 138) The generation unit 138 generates one piece of content by merging a selected comment selected by the user U from among candidate comments related to the subject of the screenshot with the screenshot. At this time, the generation unit 138 generates content in which the selected comment is placed at a relevant position on the screenshot. The generation unit 138 also generates content that highlights a portion of the screenshot that is thought to be of particular interest to the user U.

[0106] Furthermore, the generating unit 138 generates content that is an article by merging the screenshots and the answers.

[0107] (Providing Department 139) The providing unit 139 provides the screenshot and the question text to the expert to make an inquiry. The providing unit 139 also provides the original text or a link to the subject of the screenshot together with the screenshot and the question text to the expert to make an inquiry.

[0108] Furthermore, the providing unit 139 provides the answers from the experts to the user U. Furthermore, the providing unit 139 provides the generated content to other users. Furthermore, the providing unit 139 provides the candidate comments to the user U.

[0109] Furthermore, the providing unit 139 provides an information sharing service within the formed closed community.

[0110] [5. Processing Procedure] Next, a first processing procedure performed by the server device 100 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the first processing procedure according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 130 of the server device 100.

[0111] For example, as shown in FIG. 6, the acquisition unit 131 of the server device 100 acquires a screenshot of a news article or the like taken by the user U from the terminal device 10 of the user U via the communication unit 110 (step S101).

[0112] Next, the determination unit 132 of the server device 100 determines whether the screenshots acquired from the multiple users U are of the same subject (step S102).

[0113] Next, if the subject of the screenshots acquired from multiple users U is the same (step S102; Yes), the grouping unit 134 of the server device 100 groups the users U who took screenshots of the same subject (step S103).

[0114] Next, the forming unit 135 of the server device 100 forms a closed community in which users in the same group can participate (step S104).

[0115] Next, the providing unit 139 of the server device 100 provides an information sharing service within the formed closed community (step S105).

[0116] It should be noted that if the screenshots acquired from multiple users U are not of the same subject (step S102; No), the grouping unit 134, forming unit 135, and providing unit 139 of the server device 100 do not perform the processes of steps S103 to S105 described above.

[0117] Next, the extraction unit 133 of the server device 100 extracts comments related to the target of the screenshot as candidate comments from among the comments posted for the target of the screenshot (step S106). At this time, the provision unit 139 provides the candidate comments to the terminal device 10 of the user U via the communication unit 110. The acquisition unit 131 acquires, from the terminal device 10 of the user U via the communication unit 110, the comment selected by the user U from the candidate comments as a selected comment.

[0118] Next, the generating unit 138 of the server device 100 merges the selected comment selected by the user U from among the candidate comments related to the subject of the screenshot with the screenshot to generate one piece of content (step S107).

[0119] Next, the providing unit 139 of the server device 100 provides the generated content to the terminal devices 10 of the other users via the communication unit 110 (step S108). That is, the generated content is shared among the users.

[0120] Next, a second processing procedure by the server device 100 according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the second processing procedure according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 130 of the server device 100.

[0121] For example, as shown in FIG. 7, the acquisition unit 131 of the server device 100 acquires, via the communication unit 110, a screenshot of a news article or the like taken by the user U from the terminal device 10 of the user U, and a question text regarding the content of the screenshot (step S201).

[0122] Next, the determining unit 132 of the server device 100 refers to the past history information and determines whether the acquired screenshot and question text are the same as screenshots and question texts acquired in the past (step S202).

[0123] Next, if the acquired screenshot and question text are the same as those acquired in the past (step S202; Yes), the extraction unit 133 of the server device 100 extracts an answer to the acquired screenshot and question text from the past history information (step S203).

[0124] Next, if the acquired screenshot and question text are not the same as those acquired in the past (step S202; No), the estimation unit 136 of the server device 100 estimates the intention of the question of the user U from the screenshot and question text (step S204).

[0125] Next, the selection unit 137 of the server device 100 selects an expert who will answer the question based on the screenshot and the intent of the question posed by the user U (step S205).

[0126] Next, the providing unit 139 of the server device 100 makes an inquiry by providing the screenshot and the question text to the expert via the communication unit 110 (step S206). At this time, the providing unit 139 may make an inquiry by simultaneously providing the original text or a link to the subject of the screenshot along with the screenshot and the question text to the expert via the communication unit 110. Furthermore, the acquiring unit 131 acquires an answer to the question text from the expert via the communication unit 110.

[0127] Next, the providing unit 139 of the server device 100 provides the answer from the expert to the user U (step S207).

[0128] Next, the generating unit 138 of the server device 100 generates content in the form of an article by merging the screenshot and the answer (step S208).

[0129] Next, the providing unit 139 of the server device 100 provides the generated content to the terminal devices 10 of the other users via the communication unit 110 (step S209). That is, the generated content is shared among the users.

[0130] [6. Modifications] The terminal device 10 and the server device 100 described above may be implemented in various different forms other than the above embodiment. Therefore, modifications of the embodiment will be described below.

[0131] In the above embodiment, some or all of the processing executed by the server device 100 may actually be executed by the terminal device 10 (or an application running on the terminal device 10). For example, the processing may be completed in a stand-alone manner (by the terminal device 10 alone). In this case, the terminal device 10 is assumed to have the functions of the server device 100 in the above embodiment. Furthermore, in the above embodiment, the terminal device 10 cooperates with the server device 100, and therefore, from the perspective of the user U, it appears that the processing of the server device 100 is also being executed by the terminal device 10. In other words, from another perspective, the terminal device 10 can also be said to be equipped with the server device 100.

[0132] In the above embodiment, the server device 100 may use image recognition and natural language processing (NLP) to vectorize the text of the screenshot and the text of the comment (candidate comment or selected comment), and determine whether they match based on the cosine similarity of the vectors. If they match, the server device 100 may adopt the comment, and if they do not match, the server device 100 may discard the comment.

[0133] In the above embodiment, the server device 100 may use image recognition and natural language processing to vectorize the text of the screenshot and the text of the expert's answer, and determine whether they match based on the cosine similarity of the vectors. If they match, the server device 100 may adopt the answer, and if they do not match, the server device 100 may discard the answer.

[0134] In the above embodiment, the server device 100 may use image recognition and natural language processing to convert the question text and the expert's answer text into vectors, and determine whether they match based on the cosine similarity of the vectors. If they match, the server device 100 may adopt the answer, and if they do not match, the server device 100 may discard the answer.

[0135] [7. Effects] As described above, the information processing device (terminal device 10 and server device 100) according to the first invention of the present application comprises an acquisition unit 131 that acquires a screenshot from a user U, a generation unit 138 that merges a selected comment selected by the user U from among candidate comments related to the subject of the screenshot into the screenshot to generate one piece of content, and a provision unit 139 that provides the generated content to other users.

[0136] The information processing device according to the first aspect of the present invention further includes an extraction unit 133 that extracts comments related to the target of the screenshot as candidate comments from among comments posted on the target of the screenshot. The provision unit 139 provides the candidate comments to the user U. The acquisition unit 131 acquires a comment selected by the user U from the candidate comments as a selected comment.

[0137] The information processing device according to the first aspect of the present invention further includes a determination unit 132 that determines whether the subject of screenshots acquired from multiple users U is the same, a grouping unit 134 that groups users U that have taken screenshots of the same subject, and a formation unit 135 that forms a closed community in which users in the same group can participate. The provision unit 139 provides an information sharing service within the closed community.

[0138] Furthermore, the generating unit 138 generates content in which the selected comment is placed at a relevant position on the screenshot.

[0139] The information processing device according to the first aspect of the present invention further includes an estimation unit 136 that estimates a portion of the screenshot that is likely to be of particular interest to the user U. The generation unit 138 generates content that emphasizes the portion of the screenshot that is likely to be of particular interest to the user U.

[0140] From another perspective, an information processing device (terminal device 10 and server device 100) according to a second aspect of the present invention includes an acquisition unit 131 that acquires a screenshot and a question sentence from a user U, an estimation unit 136 that estimates the intention of the question of the user U from the screenshot and the question sentence, a selection unit 137 that selects an expert to answer the question from the screenshot and the intention of the question of the user U, and a provision unit 139 that provides the screenshot and the question sentence to the expert to make an inquiry. The acquisition unit 131 acquires an answer from the expert. The provision unit 139 provides the answer from the expert to the user U.

[0141] Moreover, the information processing device according to the second aspect of the present invention further includes a generating unit 138 that generates content that has been turned into an article by merging the screenshot and the answer.

[0142] Furthermore, if the portion of the screenshot taken by the user U relates to a plurality of specialized fields, the selection unit 137 selects an expert for each specialized field.

[0143] In addition, the providing unit 139 simultaneously provides the original text or a link of the subject of the screenshot to the expert along with the screenshot and the question text, and makes an inquiry.

[0144] In addition, the information processing device according to the second invention of the present application further includes a determination unit 132 that refers to past history information and determines whether the acquired screenshot and question text are the same as screenshots and question text acquired in the past, and an extraction unit 133 that extracts an answer to the acquired screenshot and question text from the past history information if they are the same.

[0145] In addition, the information processing device according to the second invention of the present application further includes a judgment unit 132 that judges whether the part of the screenshot taken by the user U matches the content of the question text, and if they match, allows the user to make an inquiry to an expert.

[0146] In addition, the information processing device according to the second invention of the present application further includes a judgment unit 132 that determines whether the number of users U whose screenshots have the same content exceeds a threshold, and if the threshold is exceeded, allows the user to inquire with an expert.

[0147] In addition, the information processing device according to the second invention of the present application further includes a judgment unit 132 that judges whether there are multiple experts whose area of ​​expertise is the part that user U has screenshotted, and if there are multiple experts, allows the user U to make an inquiry to the experts.

[0148] By performing any one or a combination of the above-described processes, the information processing device according to the present application can acquire a screenshot on the server side and perform an action triggered by the screenshot.

[0149] [8. Hardware Configuration] The terminal device 10 and the server device 100 according to the above-described embodiments are realized by a computer 1000 having a configuration as shown in Fig. 8, for example. The following description will be given taking the server device 100 as an example. Fig. 8 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I / F (Interface) 1060, an input I / F 1070, and a network I / F 1080 are connected via a bus 1090.

[0150] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic device 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

[0151] The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The secondary storage device 1050 may be an internal storage device or an external storage device. The secondary storage device 1050 may also be a removable storage medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

[0152] The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a display, a projector, a printer, etc., and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface), etc. The input I / F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, a keyboard, a keypad, a button, a scanner, etc., and is realized by a USB, etc.

[0153] Furthermore, the output I / F 1060 and the input I / F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

[0154] The output device 1010 and the input device 1020 may be integrated into one device, such as a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated into one device as an input / output I / F.

[0155] The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0156] The network I / F 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0157] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0158] For example, when the computer 1000 functions as the server device 100, the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040 to realize the functions of the control unit 130. The arithmetic unit 1030 of the computer 1000 may also load a program acquired from another device via the network I / F 1080 onto the primary storage device 1040 and execute the loaded program. The arithmetic unit 1030 of the computer 1000 may also cooperate with the other device via the network I / F 1080 to call and use the functions and data of a program from another program of the other device.

[0159] [9. Other] Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the scope of so-called equivalents. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

[0160] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0161] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0162] For example, the above-mentioned server device 100 may be realized by multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

[0163] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0164] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, an acquisition unit can be read as an acquisition means or an acquisition circuit. [Explanation of symbols]

[0165] 1. Information Processing Systems 10 Terminal Equipment 100 Server device 110 Communications Department 120 Storage section 130 control section 131 Acquisition Department 132 Judgment section 133 Extraction part 134 Grouping Section 135 Formation part 136 Estimation Department 137 Selection Department 138 Generation part 139 Provision Department

Claims

1. an acquisition unit that acquires a screenshot from a user; a generating unit that generates one piece of content by merging a selected comment selected by the user from candidate comments related to the subject of the screenshot with the screenshot; a providing unit that provides the generated content to other users; An information processing device comprising:

2. An extraction unit is further provided that extracts comments related to the subject of the screenshot as candidate comments from among comments posted to the subject of the screenshot, the providing unit provides the candidate comments to the user; The acquisition unit acquires a comment selected by the user from the candidate comments as a selected comment.

2. The information processing apparatus according to claim 1, wherein:

3. a determination unit that determines whether the screenshots acquired from a plurality of users are of the same subject; a grouping unit for grouping users who have taken screenshots of the same object; A formation department that creates a closed community in which users in the same group can participate; Furthermore, The providing unit provides an information sharing service within a closed community.

2. The information processing apparatus according to claim 1, wherein:

4. The generating unit generates content in which the selected comment is placed at a relevant position on the screenshot.

2. The information processing apparatus according to claim 1, wherein:

5. An estimation unit is further provided for estimating a portion of the screenshot that is likely to be of particular interest to the user, The generating unit generates content that highlights a portion of the screenshot that is likely to be of particular interest to the user.

2. The information processing apparatus according to claim 1, wherein:

6. An information processing method executed by an information processing device, an acquisition step of acquiring a screenshot from a user; a generating step of merging a selected comment selected by the user from candidate comments related to the subject of the screenshot with the screenshot to generate one piece of content; a providing step of providing the generated content to other users; An information processing method comprising:

7. A procedure for obtaining a screenshot from a user; a generating step of generating one piece of content by merging a selected comment selected by the user from candidate comments related to the subject of the screenshot with the screenshot; a provision step of providing the generated content to other users; An information processing program characterized by causing a computer to execute the above.

Images