Terminal device, information processing method, and information processing program

Abstract

To provide a terminal device, an information processing method, and an information processing program capable of providing more suitable information to a user.SOLUTION: A terminal device includes an image information acquisition unit, a learning unit, a provision information acquisition unit, a selection unit, and a provision unit. The image information acquisition unit acquires information on a captured image. The learning unit learns a feature of the information on the captured image acquired by the image information acquisition unit. The provision information acquisition unit acquires provision information including a plurality of pieces of proposal candidate information that are pieces of information on proposal candidates to a user. The selection unit selects, as proposal information, proposal candidate information according to the feature learned by the learning unit from among the plurality of pieces of proposal candidate information included in the provision information acquired by the provision information acquisition unit. The provision unit provides the proposal information selected by the selection unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a terminal device, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, a technique for providing a user with information about locations such as stores and facilities has been known. For example, Patent Document 1 discloses an information transmission device that acquires position information detected by a terminal device possessed by a user and transmits information about locations such as stores and facilities near the position of the terminal device to the terminal device based on the acquired position information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above prior art, although it is possible to provide a user with information about locations such as stores and facilities near the position of the terminal device, there is room for improvement in providing more suitable information for the user.

[0005] The present application has been made in view of the above, and an object thereof is to provide a terminal device, an information processing method, and an information processing program that can provide more suitable information for a user.

Means for Solving the Problems

[0006] The terminal device according to this application comprises an image information acquisition unit, a learning unit, a provision information acquisition unit, a selection unit, and a provision unit. The image information acquisition unit acquires information from captured images. The learning unit learns the characteristics of the information from captured images acquired by the image information acquisition unit. The provision information acquisition unit acquires provision information which includes multiple candidate proposal information, which is information of proposed candidates to the user. The selection unit selects from the multiple candidate proposal information included in the provision information acquired by the provision information acquisition unit as proposed information, according to the characteristics learned by the learning unit. The provision unit provides the proposed information selected by the selection unit. [Effects of the Invention]

[0007] According to one embodiment, the effect is to provide users with information that is more suitable for them. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram illustrating the information processing method performed by the terminal device according to this embodiment. [Figure 2] Figure 2 shows an example of the configuration of an information processing system that includes a terminal device according to the embodiment. [Figure 3] Figure 3 shows an example of the configuration of a terminal device according to this embodiment. [Figure 4] Figure 4 shows an example of the configuration of the acquisition unit of the terminal device according to the embodiment. [Figure 5] Figure 5 is a flowchart showing an example of information processing by the processing unit of the terminal device according to this embodiment. [Figure 6] Figure 6 is a flowchart showing an example of the suggested information provision process by the processing unit of the terminal device according to this embodiment. [Figure 7] Figure 7 is a hardware configuration diagram showing an example of a computer that implements the functions of the terminal device according to this embodiment. [Modes for carrying out the invention]

[0009] The following describes in detail, with reference to the drawings, the forms for implementing the terminal device, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments"). Note that these embodiments do not limit the terminal device, information processing method, and information processing program according to the present application. Furthermore, each embodiment can be appropriately combined as long as the processing content is not inconsistent. Also, the same parts are denoted by the same reference numerals in each of the following embodiments, and redundant explanations are omitted.

[0010] [1. An example of information processing] First, with reference to Figure 1, the information processing method performed by the terminal device according to this embodiment will be described. Figure 1 is a diagram illustrating the information processing method performed by the terminal device according to this embodiment.

[0011] The terminal device 1 shown in Figure 1 is, for example, a terminal device that can utilize an AI (Artificial Intelligence) assistant function that supports interactive voice operation. By interacting with terminal device 1, user U can control surrounding devices and obtain various information.

[0012] Furthermore, when user U makes a speech to terminal device 1 to obtain information from information provider device 2, terminal device 1 transmits information corresponding to the speech to information provider device 2. Terminal device 1 obtains content (for example, store information, facility information, event information, traffic information, weather forecast, trading partners, news, traffic information, weather, and various other information such as music) provided from information provider device 2 via network N (see Figure 2) corresponding to the information corresponding to the speech, and can display the obtained content on the display unit or output it from the speaker. An example of information processing performed by terminal device 1 is described below.

[0013] First, terminal device 1 acquires information about the captured image (step S1). For example, if information about the captured image is stored in the storage unit of terminal device 1, the processing unit of terminal device 1 acquires the information about the captured image stored in the storage unit of terminal device 1.

[0014] The information of the captured image stored in the storage unit of the terminal device 1 is, for example, the information of the captured image by the imaging unit of the terminal device 1, the information of the captured image (hereinafter sometimes referred to as a captured image) obtained by the screen shot processing by the processing unit of the terminal device 1, and the like.

[0015] Subsequently, the terminal device 1 pre-learns (step S2) before providing the information for the user U by using the features of the information of the captured image acquired in step S1. In the process of step S2, the terminal device 1 determines, for example, for each captured image, the object included in the captured image, the imaging position of the captured image, and the acquisition type of the captured image based on the information of the captured image, and learns the features of the information of the captured image based on the determined results.

[0016] The terminal device 1 determines, for example, a person (for example, the user U, the family of the user U, the friend of the user U, etc.) or an object (for example, food, store, facility, etc.) included in the captured image as an object included in the captured image.

[0017] The information of the captured image includes the image information of the captured image and the additional information of the captured image. The image information is the information of the captured image itself and includes the information of each pixel. The additional information is, for example, Exif (Exchangeable image file format) information and includes the information of the imaging position of the captured image, the information of the imaging date and time of the captured image, the information of the imaging direction of the captured image, and the information of the manufacturer and model of the imaging device.

[0018] The terminal device 1 determines the imaging position of the captured image and the acquisition type of the captured image based on the additional information of the captured image. The terminal device 1 determines whether the captured image for which the information was acquired in step S1 is a captured image obtained by the imaging unit of the terminal device 1 or a captured image obtained by the processing unit of the terminal device 1, etc. as the acquisition type of the captured image based on the information of the manufacturer and model of the imaging device included in the additional information of the captured image.

[0019] Then, based on the object included in the captured image, the imaging position of the captured image, the acquisition type of the captured image, etc. determined, the terminal device 1 learns the characteristics of the information of the captured image. For example, the terminal device 1 learns one or both of the behavior pattern of the user U and the attributes of the user U as the characteristics of the information of the captured image.

[0020] The behavior pattern of the user U is, for example, the places where the user U often goes (e.g., stores, facilities, or regions), the commuting pattern of the user U (such as the commuting route, time zone, day of the week, etc.), the companions when the user U is acting (the attributes of the companions (e.g., children or spouse), the companionship time, day of the week, etc.), etc., but is not limited thereto. The attributes of the user U are, for example, the gender, age, family, interests (e.g., favorite food) of the user U, etc., but are not limited thereto.

[0021] The terminal device 1, for example, has first determination information indicating determination conditions for each behavior pattern, and based on the first determination information, estimates the behavior pattern that satisfies the determination conditions among the plurality of behavior patterns as the behavior pattern of the user U, thereby learning the characteristics of the information of the captured image.

[0022] Also, the terminal device 1, for example, has second determination information indicating determination conditions for each attribute, and based on the second determination information, estimates the attribute that satisfies the determination conditions among the plurality of attributes as the attribute of the user U, thereby learning the characteristics of the information of the captured image.

[0023] The characteristics of the information of the captured image may be the object (e.g., person or object) itself included in the captured image. For example, when the captured image includes an image of a pancake, the characteristics of the information of the captured image may be a pancake. Also, when the captured image includes an image of a specific store, the characteristics of the information of the captured image may be the specific store.

[0024] Subsequently, for example, when the user U wants to know necessary information, the user U makes a speech for acquiring the information to be known (step S3). The terminal device 1 receives the speech from the user U as a search request, and specifies a search target corresponding to such a search request (step S4).

[0025] Terminal device 1 has a speech recognition model and a keyword estimation model. The speech recognition model and keyword estimation model are, for example, models pre-generated by machine learning. Terminal device 1 uses the speech recognition model to perform speech recognition on utterances from user U and uses the keyword estimation model to estimate search keywords.

[0026] For example, terminal device 1 inputs the speech signal from user U into a speech recognition model and obtains the text information output from the speech recognition model as the speech recognition result. Terminal device 1 also inputs the speech recognition result into a keyword estimation model and obtains the keyword output from the keyword estimation model as the search keyword.

[0027] For example, if user U says "gasoline," terminal device 1 will identify "gas station" as the search target. Also, if user U says "onigiri" (rice ball), terminal device 1 will identify "convenience store" as the search target.

[0028] Next, terminal device 1 detects its current location (step S5). For example, the processing unit of terminal device 1 causes the location detection unit of terminal device 1 to detect the current location of terminal device 1. The location detection unit of terminal device 1 receives multiple positioning signals transmitted from multiple positioning satellites in a GNSS (Global Navigation Satellite System), for example, and detects the location of terminal device 1 based on the received multiple positioning signals. The processing unit of terminal device 1 acquires the information of the current location detected by the location detection unit of terminal device 1.

[0029] Next, terminal device 1 reduces the accuracy of the current location detected in step S5 (step S6). For example, suppose the current location detected in step S5 is the latitude and longitude of the current location, and the latitude and longitude are shown to m decimal places, where m is an integer. In this case, terminal device 1 reduces the accuracy of the current location detected in step S5 by reducing the number of decimal places in the value representing the latitude and longitude of the current location, which is the current location detected in step S5.

[0030] For example, terminal device 1 reduces the accuracy of the current location detected in step S5 by setting the value indicating the latitude and longitude of the current location to a value with k decimal places (m>k). k is an integer. For example, terminal device 1 can convert the current location detected in step S5 to a value in the order of kilometers by setting k=2.

[0031] Thus, since terminal device 1 transmits reduced-accuracy current location information to information provider device 2, user U can avoid the possibility of their specific current location being determined by information provider device 2. Therefore, terminal device 1 can meet the needs of user U, who does not want their specific current location to be known. In the following, the current location with reduced accuracy in step S6 may be referred to as low-accuracy location information.

[0032] Furthermore, terminal device 1 can also change the degree to which the accuracy of the current location is reduced, for example, based on the context of user U. The context of user U is, for example, the movement state of user U, as will be described later. The movement state of user U includes the direction of movement of user U, the speed of movement of user U, and the current location of user U.

[0033] In step S6, the terminal device 1 can, for example, reduce the accuracy of the current location as the user U's movement speed increases, or reduce the accuracy of the current location to a degree that corresponds to the direction of the user U's movement. Furthermore, in step S6, the terminal device 1 can reduce the accuracy of the current location to a higher degree when the user U's means of transportation is a train compared to when the user U's means of transportation is a car.

[0034] Furthermore, terminal device 1 can also change the degree to which the accuracy of the current location is reduced based on the search target identified in step S4. For example, terminal device 1 reduces the accuracy of the current location detected in step S5 by a degree corresponding to the search target identified in step S4.

[0035] For example, terminal device 1 can change the degree of degradation of the current location accuracy depending on the type of search target. For instance, it can change the degree of degradation depending on whether the search target is a "gas station" or a "convenience store".

[0036] Terminal device 1 has information relating the search target and the degree of degradation for each search target, and reduces the accuracy of the current location based on the degree of degradation associated with the search target identified in step S4.

[0037] Next, terminal device 1 transmits an information transmission request to information providing device 2, which includes information indicating the low-precision location information that represents the current location whose accuracy was reduced in step S6 (step S7).

[0038] Next, when the information providing device 2 receives an information transmission request from the terminal device 1, it transmits information to the terminal device 1 that includes multiple candidate proposals, which are information of proposed candidates for user U, based on the low-precision location information included in the received information transmission request (step S8).

[0039] The information provider 2, for example, determines information about locations (e.g., stores or facilities) within a predetermined range centered on the location indicated by the low-precision location information included in the information transmission request as candidate information to be proposed to user U. The predetermined range is, for example, a range of several kilometers or a range of more than ten kilometers centered on the low-precision location information, but is not limited to such examples.

[0040] Potential proposals for user U include a variety of locations, such as gas stations, convenience stores, restaurants, parking lots, parks, home improvement stores, and mixed-use commercial facilities.

[0041] Information provider 2 transmits information to terminal device 1, which includes multiple candidate proposals for the selected user U. The information provided includes, for example, location lists for each location type. For example, it includes various location lists such as a list of gas stations, a list of convenience stores, and a list of restaurants.

[0042] Next, when terminal device 1 receives information transmitted from information providing device 2, it selects a location list from among the multiple location lists included in the information provided that corresponds to the search target identified in step S4 (step S9).

[0043] For example, terminal device 1 selects a list of locations from among the multiple location lists included in the provided information that contains information corresponding to the search target identified in step S4. For example, if the search target identified in step S4 is of the store type "gas station", terminal device 1 selects a list of stores of the store type "gas station". Also, if the search target identified in step S4 is of the store type "restaurant", terminal device 1 selects a list of stores of the store type "restaurant".

[0044] Next, terminal device 1 selects information about locations that meet specific conditions from the location list selected in step S9 as proposed information (step S10). Locations that meet specific conditions are, for example, locations corresponding to the context of user U, or locations corresponding to the characteristics of the captured image information learned in step S2.

[0045] The context of user U includes, for example, user U's movement status, the congestion status of the estimated user U's travel route, and the weather conditions of the estimated user U's travel route. User U's movement status includes, as mentioned above, user U's direction of movement, user U's speed of movement, user U's current location, user U's means of transportation, and user U's companions.

[0046] The movement status of user U is determined, for example, based on the detection results from sensors such as a position detection unit installed in terminal device 1. For example, terminal device 1 determines the direction of movement of user U, the speed of movement of user U, the means of movement of user U, and the presence of user U's companions based on the change in user U's position detected by the position detection unit.

[0047] User U's means of transportation may be, for example, walking, cycling, or driving a car, and terminal device 1 determines User U's means of transportation based, for example, on User U's speed of movement. Furthermore, User U's companion may be, for example, User U's child, and terminal device 1 determines, for example, that User U's companion is User U's child if User U's speed of movement is slower than usual.

[0048] Furthermore, the terminal device 1 can obtain, for example, information from an external device regarding the congestion status and weather conditions of the estimated travel route of user U, based on the current location of user U detected by the location detection unit. The terminal device 1 can also determine, based on user U's travel status, whether user U is commuting, shopping, traveling, etc., as context for user U.

[0049] Terminal device 1 can, for example, select information about a location that is appropriate to the context from among the information about multiple locations shown in the location list, which includes information corresponding to the search target identified in step S4, as information to be suggested. For example, terminal device 1 selects information about a location that is within a context-appropriate range from the user U's current location as information to be suggested.

[0050] For example, terminal device 1 can define a context-appropriate range that extends from the user U's current location to a range further than the range in the opposite direction of the user U's movement. This allows terminal device 1 to select location information within a range appropriate to the user U's context as information to be proposed.

[0051] Furthermore, terminal device 1 can expand the range according to the context as the user U's movement speed increases. This allows terminal device 1 to select location information within a range appropriate to the user U's context as the information to be suggested.

[0052] Furthermore, if user U's mode of transportation is a car, terminal device 1 can also define the area to the left of user U's direction of travel as a context-appropriate range. This allows terminal device 1 to select information about places that are easily accessible to user U as the information to be suggested.

[0053] Furthermore, if user U's mode of transportation is a train, terminal device 1 can determine the direction of the train's movement from user U's direction of travel and define a predetermined range centered on a station in that direction as a context-appropriate range.

[0054] Furthermore, terminal device 1 can also select information about different types of locations as suggested information, depending on the context. For example, terminal device 1 selects information about different types of locations as suggested information, depending on one or more of the following: user U's current location, speed of movement, direction of movement, and means of transportation. For example, terminal device 1 selects information about different types of locations as suggested information depending on whether user U is walking, traveling by car, or taking a train. Contextually appropriate types of locations include, for example, places easily accessible on foot from user U's current location, places easily accessible by car from user U's current location, and places easily accessible by train from user U's current location.

[0055] Furthermore, if the search target is a restaurant and a list of restaurants is selected in step S9, terminal device 1 will, for example, select information on restaurants that offer food and beverage delivery as suggested information if the user U's current location is on a high floor of an apartment building.

[0056] Furthermore, if the search target is restaurants and a list of shoe shops is selected in step S9, and the user U's mode of transportation is walking, terminal device 1 will, for example, select information on sneaker stores as suggested information.

[0057] Furthermore, terminal device 1 can select location information from the location list selected in step S9 that corresponds to the characteristics of the captured image information learned in step S2 as information to be proposed.

[0058] For example, if the terminal device 1 learns in step S2 that the features of the captured image information are stores or facilities, and the captured image information is captured image information, then in step S9, the terminal device 1 can select the information of stores or facilities learned as features of the captured image information from the location list selected, as the information to be proposed.

[0059] If the information in the captured image is the same as the information in the captured image, it is presumed that the stores and facilities included in the captured image are likely to be the stores and facilities that user U wants to visit, and therefore terminal device 1 can select an appropriate target for suggestion.

[0060] Furthermore, if the captured image information of terminal device 1 is information of an image captured by the imaging unit of terminal device 1, and the characteristics of the captured image information learned in step S2 are stores or facilities, terminal device 1 will select stores or facilities different from those stores or facilities and which are presumed not to have been visited by user U as proposed targets. Stores and facilities being characteristics of the captured image information is just one example of a location corresponding to the characteristics of the captured image information.

[0061] Furthermore, if the characteristics of the information in the captured image learned in step S2 correspond to the behavioral pattern of user U, terminal device 1 can select location information corresponding to user U's behavioral pattern as the information to be proposed. The behavioral pattern is defined, for example, by the content of the activity, the time and day of the week of the activity, the area of ​​the activity, and the companions.

[0062] For example, suppose the learned behavioral pattern of user U is eating lunch with children, and the current situation (time of day, day of the week, location, etc.) is the same as or similar to the situation of eating lunch with children (time of day, day of the week, location, etc.). In this case, if the location list selected in step S9 is a list of restaurants, terminal device 1 will select from among the multiple restaurants in the list that are suitable for customers with children to visit as suggested restaurants.

[0063] Furthermore, if the terminal device 1 is at the same time as or approaching the time when user U will perform a specific action (for example, going to the toilet, smoking a cigarette, drinking water, etc.), it selects information about a location where the specific action can be performed from among the information of multiple locations shown in the location list selected in step S9 as information to be suggested.

[0064] Furthermore, if the learned attribute of user U is user U's favorite food, and the location list selected in step S9 is a list of restaurants, terminal device 1 will select a restaurant from among the multiple restaurants in the list that serves user U's favorite food as a target for suggestion.

[0065] Furthermore, if the learned user U's attributes are that of a male student in his 20s, and the location list selected in step S9 is a list of restaurants, terminal device 1 will select from among the multiple restaurants in the list that are popular with male students in their 20s as potential targets for suggestions.

[0066] Terminal device 1 can select location information corresponding to the user U's context and location information corresponding to the characteristics of the captured image information as suggested information. Furthermore, if there is no location information corresponding to the user U's context, terminal device 1 can select location information corresponding to the characteristics of the captured image information as suggested information, or if there is no location information corresponding to the characteristics of the captured image information, terminal device 1 can select location information corresponding to the user U's context as suggested information.

[0067] Next, terminal device 1 provides user U with the information selected in step S10 as suggested information (step S11). In step S11, terminal device 1 outputs the suggested information as audio, or as text or images.

[0068] In this way, terminal device 1 uses location information, which is location information with reduced accuracy from the current location detected by the location detection unit, to obtain a location list (store list or facility list), which is a list of multiple locations (stores or facilities) located around the current location, from information providing device 2 (an example of an external device). As a result, terminal device 1 makes it difficult for information providing device 2 to identify the specific location of user U, and can meet the needs of user U who does not want their specific current location to be known.

[0069] Furthermore, based on the context of user U, terminal device 1 selects information on one or more locations from the information on multiple locations included in the location list and provides user U with information on one or more selected stores. In this way, terminal device 1 can provide information that is appropriate for user U and is relevant to user U's current location.

[0070] Furthermore, terminal device 1 learns the characteristics of the information in the captured image, selects a candidate information from among multiple candidate information that corresponds to the learned characteristics of the information in the captured image, and provides the selected candidate information to user U. This allows terminal device 1 to provide information that is more suitable for user U, based on user U's current location.

[0071] [2. Configuration of the Information Processing System] Figure 2 shows an example of the configuration of an information processing system that includes a terminal device 1 according to the embodiment. As shown in Figure 2, the information processing system 100 according to the embodiment comprises a terminal device 1 and an information providing device 2.

[0072] Terminal device 1 and information providing device 2 are connected to network N, and can send and receive information from each other via network N. Network N is a network that includes, for example, a WAN (Wide Area Network) such as the Internet. Network N is configured to include, for example, a mobile communication system such as 4G (4th Generation) or 5G (5th Generation), but is not limited to such examples.

[0073] Each terminal device 1 is a terminal device operated by a user U who uses the services provided by the information provision device 2, and can be, for example, a personal computer, tablet, PDA (Personal Digital Assistant), or smartphone. However, terminal device 1 is not limited to the above examples and may be, for example, a smartwatch or a wearable device.

[0074] Information providing device 2 is an information processing device that provides various types of information, and can be implemented as, for example, a server device or a cloud system. For example, information providing device 2 is a search server, which performs search processing based on search requests transmitted from terminal device 1, and provides information corresponding to the search request by transmitting the search results showing the results of such search processing to terminal device 1.

[0075] Furthermore, the information providing device 2 can also provide information on transaction targets in online shopping malls, internet shopping sites, flea market sites, auction sites, reservation sites for travel or restaurants, credit card contract sites, financial product provision sites, and the like.

[0076] [3. Terminal device 1] Figure 3 shows an example of the configuration of the terminal device 1 according to the embodiment. As shown in Figure 3, the terminal device 1 according to the embodiment includes a communication unit 10, a display unit 11, an operation unit 12, an audio input unit 13, an audio output unit 14, an imaging unit 15, a sensor unit 16, a storage unit 17, and a processing unit 18.

[0077] [3.1. Communications Section 10] The communication unit 10 is implemented, for example, by a NIC (Network Interface Card). The communication unit 10 is connected to the network N by wire or wireless connection and transmits and receives information to and from the information providing device 2 via the network N.

[0078] [3.2. Display section 11] The display unit 11 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.

[0079] [3.3. Operation unit 12] The operation unit 12 includes, for example, a keyboard with keys for entering letters, numbers, and spaces, an enter key and arrow keys, a mouse, and a power button. If the display unit 11 is a touch panel display, the operation unit 12 may include a touch panel.

[0080] [3.4. Voice Input Section 13] The audio input unit 13 converts the audio signal, which is the voice signal emitted by the user U, into a digital signal, and outputs the converted digital audio signal, which is the audio digital signal, to the processing unit 18 as audio information. The audio input unit 13 includes, for example, a microphone and an AD (Analog to Digital) converter that converts the audio signal, which is an electrical analog signal output from the microphone, into a digital signal.

[0081] [3.5. Audio Output Section 14] The audio output unit 14 includes, for example, a DA (Digital to Analog) converter that converts a digital audio signal, which is audio information output from the processing unit 18, into an analog audio signal, and a speaker that converts the analog audio signal output from the DA converter into sound and outputs it.

[0082] [3.6. Imaging Unit 15] The imaging unit 15 is an image sensor (camera) that captures images of the subject. For example, the imaging unit 15 may be a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge-Coupled Device) image sensor. Note that the imaging unit 15 is not limited to an internal camera; it may also be an external camera such as a wireless camera capable of communicating with the terminal device 1 or a webcam.

[0083] [3.7. Sensor section 16] The sensor unit 16 includes a position detection unit 20 and a gyro sensor 21, etc. The position detection unit 20 detects, for example, the position of the terminal device 1, which is the current position of user U, and outputs information indicating the detected current position of user U to the processing unit 18. The position detection unit 20 receives multiple positioning signals transmitted from multiple positioning satellites in GNSS, and detects the current position of user U based on the received multiple positioning signals.

[0084] The gyro sensor 21 is a sensor that detects the attitude of the terminal device 1, such as its tilt and rotation. The sensor unit 16 also includes an acceleration sensor, a geomagnetic sensor, an illuminance sensor, and a barometric pressure sensor. The acceleration sensor is a sensor that detects the acceleration of the terminal device 1. The geomagnetic sensor is a sensor that detects the Earth's magnetic field. The illuminance sensor is a sensor that detects the illuminance, which indicates the brightness around the terminal device 1. The barometric pressure sensor is a sensor that detects the atmospheric pressure around the terminal device 1.

[0085] [3.8. Storage section 17] The memory unit 17 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or by storage devices such as hard disks or optical discs.

[0086] Various types of information are stored in the memory unit 17. For example, the memory unit 17 contains information about captured images, which are images captured by the imaging unit 15, and information about captured images, which are captured images obtained through screenshot processing by the processing unit 18.

[0087] Furthermore, the storage unit 17 stores information transmitted from the information providing device 2 or other external devices and acquired by the processing unit 18 via the network N and the communication unit 10. The information acquired from the information providing device 2 or other external devices includes, for example, information on captured images saved by user U to external storage.

[0088] Furthermore, the memory unit 17 also stores feature information indicating the characteristics of the captured image information learned by the processing unit 18, information detected by the sensor unit 16, and voice information and text information corresponding to the user U's speech.

[0089] [3.9. Processing Unit 18] The processing unit 18 is a controller, which is realized, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the memory device inside the terminal device 1 using RAM as the working area.

[0090] Furthermore, the processing unit 18 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The processing unit 18 includes an acquisition unit 30, a learning unit 31, a receiving unit 32, a specification unit 33, a determination unit 34, a selection unit 35, and a supply unit 36.

[0091] [3.9.1. Acquisition part 30] The acquisition unit 30 acquires various types of information. The acquisition unit 30 acquires various types of information from the storage unit 17. The acquisition unit 30 acquires the detection results from the sensor unit 16 and stores the acquired detection results in the storage unit 17.

[0092] The acquisition unit 30 can receive various types of information from an external information processing device via the communication unit 10. The acquisition unit 30 stores the various types of information received from the external information processing device in the storage unit 17.

[0093] Furthermore, the acquisition unit 30 transmits an information transmission request to the information providing device 2 via the communication unit 10 and the network N, and receives the provided information transmitted from the information providing device 2 in response to the information transmission request via the network N and the communication unit 10. The information transmission request includes location information, which is location information obtained by reducing the accuracy of the current location detected by the location detection unit 20.

[0094] Figure 4 shows an example of the configuration of the acquisition unit 30 of the terminal device 1 according to the embodiment. As shown in Figure 4, the acquisition unit 30 comprises an image information acquisition unit 40 and a provided information acquisition unit 41. The image information acquisition unit 40 acquires information of captured images. For example, the image information acquisition unit 40 acquires information of captured images stored in the storage unit 17.

[0095] The information about captured images stored in the memory unit 17 may include, for example, information about images captured by the imaging unit 15, or information about captured images obtained through screenshot processing by the processing unit 18. However, it may also include information about captured images obtained from applications installed on the terminal device 1 (for example, SNS (Social Networking Service) applications).

[0096] The information acquisition unit 41 transmits an information transmission request to the information providing device 2 via the communication unit 10 and the network N, and receives the information provided by the information providing device 2 in response to the information transmission request via the network N and the communication unit 10.

[0097] The information acquisition unit 41 generates low-precision location information, which is location information obtained by reducing the accuracy of the current location detected by the location detection unit 20, and sends an information transmission request containing the generated low-precision location information to the information providing device 2. When the information providing device 2 receives an information transmission request from the terminal device 1, it sends information to the terminal device 1 that includes multiple candidate proposals, which are information of proposed candidates for user U, based on the low-precision location information included in the received information transmission request.

[0098] For example, the information acquisition unit 41 assumes that the current location detected by the location detection unit 20 is the latitude and longitude of the current location, and that the latitude and longitude are shown to m decimal places, where m is an integer. In this case, the information acquisition unit 41 reduces the accuracy of the current location detected by the location detection unit 20 by reducing the number of decimal places in the value indicating the latitude and longitude of the current location detected by the location detection unit 20.

[0099] For example, the information acquisition unit 41 reduces the accuracy of the current location detected by the location detection unit 20 by setting the value indicating the latitude and longitude of the current location to a value with k decimal places (m>k). k is an integer. The information acquisition unit 41 can convert the current location detected by the location detection unit 20 to a value in the order of kilometers by setting k=2, for example.

[0100] Thus, the information acquisition unit 41 transmits an information transmission request to the information providing device 2 that includes low-precision location information, which is current location information with reduced accuracy. As a result, the information providing device 2 can avoid the possibility of user U's specific current location being determined. Therefore, the terminal device 1 can meet the needs of user U, who does not want their specific current location to be known.

[0101] The information provider 2, for example, determines information about locations (e.g., stores or facilities) within a predetermined range centered on the location indicated by the low-precision location information included in the information transmission request as candidate information to be proposed to user U. The predetermined range is, for example, a range of several kilometers or a range of more than ten kilometers centered on the low-precision location information, but is not limited to such examples.

[0102] Potential proposals for user U include a variety of locations, such as gas stations, convenience stores, restaurants, parking lots, parks, home improvement stores, and mixed-use commercial facilities.

[0103] Information provider 2 transmits information to terminal device 1, which includes multiple candidate proposals for the selected user U. The information provided includes location lists for each location type. For example, it includes various location lists such as a list of gas stations, a list of convenience stores, and a list of restaurants.

[0104] The information acquisition unit 41 can, for example, reduce the accuracy of the current location detected by the location detection unit 20 based on at least one of the context of user U determined by the determination unit 34 and the search target identified by the identification unit 33.

[0105] For example, the information acquisition unit 41 changes the degree of degradation in the accuracy of the current position detected by the position detection unit 20 based on the context of user U. The context of user U is, for example, the movement state of user U, as will be described later. The movement state of user U includes the direction of user U's movement, the speed of user U's movement, and the current position of user U.

[0106] The information acquisition unit 41 can, for example, reduce the accuracy of the current location as the user U moves faster, or reduce the accuracy of the current location to a degree that corresponds to the direction of the user U's movement. Furthermore, the information acquisition unit 41 can reduce the accuracy of the current location to a higher degree when the user U's means of transportation is a train compared to when the user U's means of transportation is a car.

[0107] Furthermore, the information acquisition unit 41 can also change the degree to which the accuracy of the current location detected by the location detection unit 20 decreases based on the search target identified by the identification unit 33. For example, the information acquisition unit 41 reduces the accuracy of the current location detected by the location detection unit 20 by a degree corresponding to the search target identified by the identification unit 33.

[0108] The information acquisition unit 41 has information relating the search target and the degree of degradation for each search target, and the accuracy of the current location is reduced by the degree of degradation associated with the search target identified by the identification unit 33.

[0109] The information acquisition unit 41 can change the degree to which the accuracy of the current location detected by the location detection unit 20 decreases depending on whether the search target is a "gas station" or a "convenience store".

[0110] Furthermore, the provided information acquisition unit 41 has information indicating the degree of degradation for each combination of user U's context and the search target, and can also degrade the accuracy of the current location by a degree of degradation corresponding to the combination of user U's context determined by the determination unit 34 and the search target identified by the identification unit 33.

[0111] [3.9.2. Learning Section 31] The learning unit 31 learns the characteristics of the information in the captured image acquired by the image information acquisition unit 40 in advance before providing the information to the user U. In the following, the characteristics of the information in the captured image may be referred to as image information characteristics.

[0112] For example, the learning unit 31 determines the objects included in the captured image, the position where the captured image was captured, and the type of captured image based on the information of the captured image acquired by the image information acquisition unit 40, and learns image information features based on the results of the determination.

[0113] The learning unit 31 determines, for example, that people (e.g., user U, user U's family, user U's friends, etc.) and objects (e.g., food, shops, facilities, etc.) included in the captured image are objects included in the captured image.

[0114] The information of the captured image is, for example, file information with extensions such as "JPEG," "HEIC," and "mp4," and includes image information and additional information about the captured image. The image information is information about the captured image itself, including information about each pixel. The additional information is, for example, Exif information, and includes information about the image's capture location, the date and time of capture, the image's capture direction, and the manufacturer and model of the imaging device.

[0115] The learning unit 31 determines the imaging position and acquisition type of the image based on the additional information of the image. Based on the manufacturer and model information of the imaging device included in the additional information of the image, the learning unit 31 determines whether the image from which information has been acquired by the image information acquisition unit 40 is an image obtained by the imaging unit 15 or a captured image obtained by the processing unit 18, as the acquisition type of the image.

[0116] The learning unit 31 then learns image information features based on the objects included in the determined captured image, the location where the image was captured, and the type of image acquired. For example, the learning unit 31 learns either or both of the user U's behavior patterns and / or user U's attributes as image information features.

[0117] User U's behavioral patterns include, but are not limited to, places User U frequently visits (e.g., shops, facilities, or areas), User U's commuting patterns (commuting routes, times, and days of the week), and companions during User U's activities (attributes of companions (e.g., children or spouse) and times and days of the week). User U's attributes include, but are not limited to, User U's gender, age, family, and interests (e.g., favorite foods).

[0118] The learning unit 31, for example, has first judgment information that indicates judgment conditions for each behavior pattern, and learns image information features by estimating the behavior pattern that satisfies the judgment conditions among multiple behavior patterns as the behavior pattern of user U based on the first judgment information.

[0119] Furthermore, the learning unit 31 has, for example, second judgment information that indicates judgment conditions for each attribute, and learns image information features by estimating the attributes that satisfy the judgment conditions among multiple attributes as attributes of user U based on the second judgment information.

[0120] The learning unit 31, for example, has a learning model that takes information from captured images as input and outputs information indicating the user U's behavior patterns and attributes. By using this learning model, it is also possible to learn image information features by estimating the user U's behavior patterns and attributes.

[0121] The learning model is generated by machine learning using a neural network, such as a convolutional neural network. However, it is not limited to this example; the learning model may also be generated using machine learning algorithms other than neural networks, such as linear regression, nonlinear regression, logistic regression, or support vector machines.

[0122] Furthermore, the image information features may be the objects (e.g., people or objects) included in the captured image. For example, if the captured image includes an image of pancakes, the image information features may be pancakes. Also, if the captured image includes an image of a specific store, the image information features may be the specific store.

[0123] [3.9.3. Reception Department 32] The reception unit 32 receives search requests from user U. For example, the reception unit 32 receives a utterance from user U as a search request. The reception unit 32 also receives an operation by user U to the operation unit 12 as a search request if that operation specifies a search target.

[0124] The reception unit 32 receives utterances from user U based on the voice information output from the voice input unit 13. For example, if user U performs a specific operation using the operation unit 12, the reception unit 32 receives subsequent utterances from user U.

[0125] Furthermore, the reception unit 32 can also receive subsequent utterances from user U if user U utters a specific keyword. Whether or not user U has uttered a specific keyword is determined by speech recognition of the voice information output from the voice input unit 13.

[0126] The reception unit 32 has a speech recognition function that converts the speech information output from the speech input unit 13 in response to the user U's speech into text information. The speech recognition function is performed using a speech recognition model. The reception unit 32 inputs the speech information output from the speech input unit 13 in response to the user U's speech into the speech recognition model and obtains the text information output from the speech recognition model as the speech recognition result.

[0127] [3.9.4. Specification part 33] The identification unit 33 identifies the search target based on the search request received by the reception unit 32. The identification unit 33 identifies the search target based on the speech recognition result obtained by the reception unit 32.

[0128] The identification unit 33 has a keyword estimation model, and inputs the text information, which is the speech recognition result obtained by the reception unit 32, into the keyword estimation model, and obtains the keyword output from the keyword estimation model as the search keyword.

[0129] For example, if user U says "gasoline," the identification unit 33 identifies "gas station" as the search target. Also, if user U says "onigiri," the identification unit 33 identifies "convenience store" as the search target.

[0130] Furthermore, the identification unit 33 can also use the text information obtained by the reception unit 32, which is the result of speech recognition, as the search target.

[0131] The identification unit 33 can also identify the attributes of user U based on the voice received by the reception unit 32. For example, the identification unit 33 has an attribute identification function that identifies the attributes of user U from the voice information output from the voice input unit 13 based on the user U's speech, and identifies the attributes of user U based on the voice received by the reception unit 32.

[0132] For example, the identification unit 33 identifies the gender, age, etc. of user U from the frequency of the voice received by the reception unit 32. The identification unit 33 has, for example, an attribute identification model, and inputs the voice information output from the voice input unit 13 based on user U's speech into the attribute identification model, and obtains the attribute identification result of user U output from the attribute identification model.

[0133] Furthermore, the identification unit 33 can determine the stride length of user U based on the number of steps and distance traveled by user U detected by the sensor unit 16, and can also identify the gender and age of user U based on the determined stride length of user U.

[0134] [3.9.5. Judgment unit 34] The determination unit 34 determines the context of user U based on the detection results from the sensor unit 16.

[0135] The context of user U includes, for example, user U's movement status, the congestion status of the estimated user U's travel route, and the weather conditions of the estimated user U's travel route. User U's movement status includes, as mentioned above, user U's direction of movement, user U's speed of movement, user U's current location, user U's means of transportation, and user U's companions.

[0136] The movement status of user U is determined according to the detection results from the sensor unit 16. For example, the determination unit 34 determines the direction of movement of user U, the speed of movement of user U, the means of movement of user U, and the presence of user U's companions based on the change in user U's position detected by the position detection unit 20.

[0137] User U's means of transportation may be, for example, walking, cycling, or driving a car, and the determination unit 34 determines User U's means of transportation, for example, based on User U's speed of movement. Furthermore, User U's companion may be, for example, User U's child, and the determination unit 34 determines, for example, that User U's companion is User U's child if User U's speed of movement is slower than usual.

[0138] Furthermore, the determination unit 34 can obtain, for example from an external device, the congestion status of the estimated travel route of user U and the weather status of the estimated travel route of user U, based on the current location of user U detected by the location detection unit 20. The determination unit 34 can also determine, based on user U's travel status, whether user U is commuting, shopping, traveling, etc., as context for user U.

[0139] Furthermore, the determination unit 34 may also determine the context of user U based on at least one of the gyro sensor 21, acceleration sensor, geomagnetic sensor, illuminance sensor, and barometric pressure sensor, instead of or in addition to the position detection unit 20.

[0140] For example, the determination unit 34 can determine, based on the detection result from the position detection unit 20 and the detection result from the illuminance sensor, that the user U's means of transportation is an automobile and that the user is traveling through a tunnel.

[0141] Furthermore, the determination unit 34 can determine, for example, that user U is located on a high floor of an apartment building if, based on the detection result from the position detection unit 20 and the detection result from the barometric pressure sensor, user U's current location is at the apartment building and the barometric pressure detected by the barometric pressure sensor is high.

[0142] [3.9.6. Selection Section 35] The selection unit 35 selects, from among multiple candidate proposals included in the provided information acquired by the provided information acquisition unit 41, the candidate proposal information that corresponds to the location information identified by the identification unit 33. The candidate proposal information is, for example, a location list.

[0143] For example, the selection unit 35 selects a list of locations from among the multiple candidate information provided that includes information corresponding to the search target identified by the identification unit 33. For example, if the search target identified by the identification unit 33 is a store of the type "gas station", the selection unit 35 selects a list of stores of the store type "gas station". Also, if the search target identified by the identification unit 33 is a store of the type "restaurant", the selection unit 35 selects a list of stores of the store type "restaurant".

[0144] The selection unit 35 selects information for one or more locations as proposed information from the selected location list, which is a list of locations selected based on the search target identified by the identification unit 33, using at least one of the following: the context of user U determined by the determination unit 34, the attributes of user U identified by the identification unit 33, and the image information features learned by the learning unit 31.

[0145] For example, the selection unit 35 selects information about one or more locations from among the information about multiple locations included in the selection store list, based on the context of user U determined by the determination unit 34. For example, the selection unit 35 selects information about one or more locations within a context-appropriate range from the current location from among the information about multiple locations included in the selection store list.

[0146] For example, the selection unit 35 can define a context-appropriate range that extends beyond the range in the opposite direction of user U's movement, centered on user U's current position. This allows the selection unit 35 to select location information within a range appropriate to user U's context as information to be proposed.

[0147] Furthermore, the selection unit 35 can widen the range according to the context as the user U's movement speed increases. This allows the selection unit 35 to select location information within a range suitable for the user U's context as the information to be proposed.

[0148] Furthermore, if the user U's means of transportation is a car, the selection unit 35 can also define the range to the left of the user U's direction of travel as a context-appropriate range. This allows the selection unit 35 to select information about places that are easily accessible to the user U as the information to be suggested.

[0149] Furthermore, if the user U's mode of transportation is a train, the selection unit 35 can determine the direction of the train's movement from the user U's direction of travel and set a predetermined range centered on a station in that direction as the context-appropriate range.

[0150] Furthermore, the selection unit 35 can also select different types of locations to suggest based on the context. For example, the selection unit 35 selects information about different types of locations as suggested information based on one or more of the user U's current location, speed of movement, direction of movement, and means of transportation. For example, the selection unit 35 selects information about different types of locations as suggested information depending on whether the user U is walking, traveling by car, or taking a train. Contextually appropriate types of locations include, for example, places easily accessible on foot from the user U's current location, places easily accessible by car from the user U's current location, and places easily accessible by train from the user U's current location.

[0151] Furthermore, if the search target is a restaurant and the user selects a list of restaurants as the selection location list, the selection unit 35 will, for example, select restaurants that offer food and beverage delivery as suggested targets if the user U's current location is on a high floor of an apartment building.

[0152] Furthermore, if the search target is restaurants and a list of shoe shops is selected as the selection location list, and the user U's mode of transportation is walking, the selection unit 35 will, for example, select sneaker stores as suggested targets.

[0153] Furthermore, the selection unit 35 selects information about one or more locations corresponding to the image information features learned by the learning unit 31 as proposed information from among the information about multiple locations included in the selection location list. In this way, the selection unit 35 can select proposed candidate information corresponding to the image information features learned by the learning unit 31 from among the multiple proposed candidate information included in the provided information acquired by the provided information acquisition unit 41 as proposed information.

[0154] For example, if the image information feature is a store or facility, and the information of the captured image corresponding to the image information feature is information of the captured image, the selection unit 35 can select the information of the store or facility that has been learned as an image information feature from the location information included in the selection location list as the information to be proposed.

[0155] If the information in the captured image is the same as the information in the captured image, the selection unit 35 can select appropriate information for the suggested location, as it is highly likely that the stores and facilities included in the captured image are stores and facilities that user U would like to visit.

[0156] Furthermore, if the information of the captured image is information of an image captured by the imaging unit 15 of the terminal device 1, and the image information features are those of a store or facility, the selection unit 35 selects as a target store or facility that is different from the store or facility in question and is presumed not to have been visited by user U. The image information features being those of a store or facility are just one example of a location corresponding to the information features of the captured image.

[0157] Furthermore, if the image information features are those of user U's behavior pattern, the selection unit 35 can select location information corresponding to user U's behavior pattern as the information to be proposed. The behavior pattern is defined, for example, by the content of the activity, the time and day of the week of the activity, the area of ​​the activity, and the companions.

[0158] For example, suppose the learned behavioral pattern of user U is eating lunch with children, and the current situation (time of day, day of the week, location, etc.) is the same as or similar to the situation of eating lunch with children (time of day, day of the week, location, etc.). In this case, if the selection location list is a list of restaurants, the selection unit 35 will select from among the multiple restaurants in the list that are suitable for customers with children to visit as a suggested location.

[0159] Furthermore, if the user U is about to perform a specific action (for example, going to the toilet, smoking a cigarette, or drinking water), the selection unit 35 selects information about a location where the specific action can be performed from among the multiple locations shown in the selection location list as information to be suggested.

[0160] Furthermore, if the learned attribute of user U is user U's favorite food and the selection location list is a list of restaurants, the selection unit 35 selects information about restaurants that serve user U's favorite food from among the multiple restaurants included in the list of restaurants as information to be suggested.

[0161] Furthermore, if the learned user U's attributes are that of a male student in his 20s, and the selection unit 35 is a list of restaurants, it will select from among the multiple restaurants in the list that are popular with male students in their 20s as potential candidates for selection.

[0162] Furthermore, the selection unit 35 can also select as proposed information a candidate that corresponds to the image information characteristics from among multiple candidate proposals included in the provided information acquired by the provided information acquisition unit 41, narrowed down based on the context of user U determined by the determination unit 34.

[0163] For example, the selection unit 35 can narrow down information on multiple locations within a context-appropriate range from the current location from the information on multiple locations included in the selection store list, and from the narrowed-down information on multiple locations, it can select information on locations that match the image information features as proposed information.

[0164] Furthermore, the selection unit 35 can select one or more location pieces from among multiple location pieces, based on the attributes of user U identified by the identification unit 33 and the context of user U. For example, the selection unit 35 can narrow down the information of multiple locations within a range corresponding to the context of user U from the current location, and then select the location piece corresponding to the attributes of user U from among the narrowed-down information as suggested information.

[0165] For example, suppose the list of selected stores is a list of stores that sell sneakers in Ikebukuro, the context of user U is that they are walking around Ikebukuro, and the attribute of user U identified by the identification unit 33 is that they are female. In this case, the selection unit 35 selects information about the store in Ikebukuro that sells sneakers and is closest to user U's current location that sells women's sneakers as suggested information.

[0166] Furthermore, the selection unit 35 can select location information corresponding to the user U's context and location information corresponding to the characteristics of the captured image information as proposed information. In addition, if there is no location information corresponding to the user U's context, the selection unit 35 can select location information corresponding to the characteristics of the captured image information as proposed information, or if there is no location information corresponding to the characteristics of the captured image information, it can select location information corresponding to the user U's context as proposed information.

[0167] Furthermore, the selection unit 35 can, for example, select location information corresponding to the user U's context as suggested information when the operation mode is the first mode, and select location information corresponding to the characteristics of the captured image information as suggested information when the operation mode is the second mode.

[0168] The operating mode can be set by, for example, user U of terminal device 1. Terminal device 1 can also switch the operating mode depending on, for example, the search target identified by the identification unit 33. For example, the selection unit 35 can select a suggested target in the first mode when the search target is "gas station," and select a suggested target in the second mode when the search target is "lunch."

[0169] [3.9.7.Providing Department 36] The provisioning unit 36 ​​provides the proposed information to user U by transmitting the proposed information, which is information about one or more locations selected as proposed targets by the selection unit 35, to user U's terminal device 1.

[0170] The provisioning unit 36 ​​provides the proposed information to the user U by displaying the proposed information on the display unit 11. The provisioning unit 36 ​​can also provide the proposed information to the user U by outputting the proposed information as sound from the audio output unit 14.

[0171] [4. Processing Procedure] Next, the procedure for information processing by the processing unit 18 of the terminal device 1 according to this embodiment will be described. Figure 5 is a flowchart showing an example of information processing by the processing unit 18 of the terminal device 1 according to this embodiment.

[0172] As shown in Figure 5, the processing unit 18 of the terminal device 1 determines whether or not it is time for learning (step S100). In the process of step S100, the processing unit 18 determines, for example, that it is time for learning each time the number of newly added images in the storage unit 17 exceeds a threshold.

[0173] If the processing unit 18 determines that it is time to learn (step S100: Yes), it learns the characteristics of the information in the captured image (step S101). If the processing in step S101 is completed, or if the processing unit 18 determines that it is not time to learn (step S100: No), it determines whether or not user U has spoken (step S102).

[0174] If the processing unit 18 determines that user U has spoken (step S102: Yes), it performs the suggested information provision process (step S103). The process in step S103 is the same as the processes in steps S110 to S118 shown in Figure 6, which will be described in detail later.

[0175] When the processing in step S103 is completed, or when it is determined that there has been no utterance from user U (step S102: No), the processing unit 18 determines whether or not it is time to terminate (step S104). For example, the processing unit 18 determines that it is time to terminate when the power to terminal device 1 is turned off, or when it is determined that a termination operation has been performed by operating the operation unit 12 of terminal device 1.

[0176] If the processing unit 18 determines that it is not yet time to terminate (step S104: No), it proceeds to step S100. If it determines that it is time to terminate (step S104: Yes), it terminates the process shown in Figure 5.

[0177] Figure 6 is a flowchart showing an example of the suggested information provision process by the processing unit 18 of the terminal device 1 according to this embodiment. As shown in Figure 6, the processing unit 18 receives an utterance from user U as a search request (step S110). Then, the processing unit 18 identifies the search target based on the search request received in step S110 (step S111).

[0178] Furthermore, the processing unit 18 causes the position detection unit 20 to detect the current position (step S112). Then, the processing unit 18 reduces the accuracy of the current position detected by the position detection unit 20 in step S112 (step S113).

[0179] Furthermore, the processing unit 18 sends an information transmission request to the information providing device 2, which includes information indicating the low-precision location information, which is the current location whose accuracy was reduced in step S113 (step S114). Then, the processing unit 18 obtains the provided information transmitted from the information providing device 2 in response to the information transmission request (step S115).

[0180] Next, the processing unit 18 selects a location list from among the multiple location lists indicated by the provided information obtained in step S115 that corresponds to the search target identified in step S111 (step S116).

[0181] Next, the processing unit 18 selects information for one or more locations as proposed information from the location list selected in step S116, based on at least one of the user U's context and image information features (step S117). Then, the processing unit 18 provides the proposed information, which is the proposed information selected in step S117, to the user U (step S118), and terminates the process shown in Figure 6.

[0182] [5. Variations] In the example described above, the information acquisition unit 41 can also acquire information that includes information other than location information as candidate information. For example, the information acquisition unit 41 can transmit an information transmission request to the information providing device 2 that includes the search target information identified by the identification unit 33 but does not include location information.

[0183] In this case, when the information provider 2 receives an information transmission request from the terminal device 1, it transmits to the terminal device 1 information that includes multiple pieces of information corresponding to the search target as suggested candidate information, which are suggested candidate information for user U, based on the search target information contained in the received information transmission request. The selection unit 35 selects, from among the multiple suggested candidate pieces of information contained in the information provider information acquired by the information provider acquisition unit 41, the suggested candidate information that corresponds to the image information features learned by the learning unit 31 as suggested information.

[0184] The information transmission request may include, in place of or in addition to, the information to be searched identified by the identification unit 33, information about the context of user U determined by the determination unit 34. In this case, based on the information to be searched and the information about the context of user U included in the received information transmission request, the terminal device 1 receives provision information containing multiple pieces of information corresponding to the searched item as proposed candidate information, which is proposed candidate information for user U.

[0185] Furthermore, the information of the captured image may include the detection results of the sensor unit 16 during imaging by the imaging unit 15 as additional information. In this case, the learning unit 31 can further learn image information features using the detection results of the sensor unit 16.

[0186] Furthermore, the information provider 2 can determine information about locations within a range corresponding to the number of digits of the value indicated by the user's low-precision location as candidate information for suggestion to user U. For example, if the value indicated by the user's low-precision location is a value with two decimal places, the information provider 2 will determine information about locations within the range indicated from the user's low-precision location to one decimal place as candidate information for suggestion to user U.

[0187] In the example described above, the provided information was assumed to include, for example, a list of locations categorized by location type. However, the example is not limited to this; for example, the provided information may include various other types of information.

[0188] Furthermore, while the above examples used shops and facilities as examples of locations, the location could also be the venue where the event is held, a campsite, a beach, a lake, a mountain, or any other type of location.

[0189] [6. Hardware Configuration] The terminal device 1 according to the embodiment described above is implemented by a computer 80 having a configuration such as that shown in Figure 7. The following explanation will use the terminal device 1 as an example. Figure 7 is a hardware configuration diagram showing an example of a computer 80 that implements the functions of the terminal device 1 according to the embodiment. The computer 80 has a CPU 81, RAM 82, ROM (Read Only Memory) 83, HDD (Hard Disk Drive) 84, communication interface (I / F) 85, input / output interface (I / F) 86, and media interface (I / F) 87.

[0190] The CPU 81 operates based on programs stored in the ROM 83 or HDD 84, and controls various parts of the system. The ROM 83 stores boot programs executed by the CPU 81 when the computer 80 starts up, as well as programs that depend on the computer 80's hardware.

[0191] HDD84 stores programs executed by CPU81 and data used by such programs. The communication interface85 receives data from other devices via network N (see Figure 2) and sends it to CPU81, and transmits data generated by CPU81 to other devices via network N.

[0192] The CPU 81 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input / output interface 86. The CPU 81 acquires data from input devices via the input / output interface 86. The CPU 81 also outputs data it has generated to output devices via the input / output interface 86.

[0193] The media interface 87 reads a program or data stored in the recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads the program from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded program. The recording medium 88 can be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0194] For example, when the computer 80 functions as a terminal device 1 according to the embodiment, the CPU 81 of the computer 80 realizes the functions of the processing unit 18 by executing a program loaded on the RAM 82. The data in the storage unit 17 is stored in the HDD 84. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be obtained from other devices via the network N.

[0195] [7. Other] Furthermore, some of the processes described as being performed automatically in the above embodiments can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and various data and parameters shown in the above documents and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0196] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0197] Furthermore, for example, some or all of the storage unit 17 shown in Figure 3 may be stored in a storage server or the like, rather than being held by each device. In this case, each device obtains various information by accessing the storage server.

[0198] [8. Effects] The terminal device 1 according to this application comprises an image information acquisition unit 40, a learning unit 31, a provision information acquisition unit 41, a selection unit 35, and a provision unit 36. The image information acquisition unit 40 acquires information from an image. The learning unit 31 learns the characteristics of the information from the image acquired by the image information acquisition unit 40. The provision information acquisition unit 41 acquires provision information that includes multiple candidate proposal information, which is information of proposed candidates to the user U. The selection unit 35 selects candidate proposal information from among the multiple candidate proposal information included in the provision information acquired by the provision information acquisition unit 41 as proposed information, according to the characteristics of the information from the image learned by the learning unit 31. The provision unit 36 ​​provides the proposed information selected by the selection unit 35. As a result, the terminal device 1 can provide information that is more suitable for the user U, corresponding to the user U's current location and the characteristics of the information from the image.

[0199] Furthermore, the learning unit 31 determines the objects included in the captured image, the position where the image was captured, and the type of image acquisition based on the information of the captured image, and learns the characteristics of the information of the captured image based on the determined results. As a result, the terminal device 1 can provide information that is more suitable for user U, corresponding to user U's current location and the characteristics of the information of the captured image.

[0200] Furthermore, the learning unit 31 learns either or both of the user U's behavioral patterns and / or user U's attributes as features of the captured image information. As a result, the terminal device 1 can provide information that is more suitable for user U, corresponding to user U's current location and the features of the captured image information.

[0201] Furthermore, terminal device 1 includes a location detection unit 20 that detects the current location of user U. The provided information acquisition unit 41 uses location information, which is location information with reduced accuracy from the current location detected by the location detection unit 20, to acquire provided information that includes information about locations located around the current location as suggested candidate information from information providing device 2 (an example of an external device). In addition, since terminal device 1 transmits the reduced accuracy current location information to information providing device 2, it can meet the needs of user U who does not want their specific current location to be known.

[0202] Furthermore, the selection unit 35 selects proposed candidate information as proposed information that corresponds to a location that is presumed not to have been visited by user U, and that corresponds to a location that is corresponding to the characteristics of the information in the captured image. As a result, the terminal device 1 can provide information that is more suitable for user U, corresponding to the information of user U, and that is based on the characteristics of the information in the captured image.

[0203] Furthermore, the terminal device 1 includes a reception unit 32 that receives search requests from user U, and a specification unit 33 that identifies the search target based on the search request received by the reception unit 32. The selection unit 35 selects proposed information from among the proposed candidate information corresponding to the location corresponding to the search target identified by the specification unit 33. As a result, the terminal device 1 can provide information that is more suitable for user U, corresponding to the user U's current location, the search request, and the characteristics of the captured image information.

[0204] Furthermore, the terminal device 1 includes a sensor unit 16 having one or more sensors, including a position detection unit 20, and a determination unit 34 that determines the context of user U based on the detection results from the sensor unit 16. The selection unit 35 selects a candidate proposal to be proposed from among a plurality of candidate proposals that have been narrowed down based on the context determined by the determination unit 34, from among a plurality of candidate proposals included in the provided information acquired by the provided information acquisition unit 41. As a result, the terminal device 1 can provide information that is appropriate for user U, corresponding to the user U's current location, the characteristics of the information in the captured image, and the user U's context.

[0205] Furthermore, the selection unit 35 selects proposed information from among the proposed candidate information corresponding to locations within a context-appropriate range from the current location. As a result, the terminal device 1 can provide information that is more suitable for user U, corresponding to user U's current location, the characteristics of the captured image information, and user U's context.

[0206] Although embodiments of the present application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various other forms, including those described in the disclosure section of the invention, based on the knowledge of those skilled in the art.

[0207] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit," etc. For example, the acquisition unit can be replaced with acquisition means or acquisition circuit. [Explanation of symbols]

[0208] 1. Terminal device 2 Information provision device 10 Communications Department 11 Display section 12 Control section 13. Voice input section 14. Audio output section 15 Imaging Unit 16 Sensor section 17 Memory section 18 Processing Unit 20 Position detection unit 21 Gyro sensor 30 Acquisition Department 31. Learning Department 32 Reception Department 33 Specific part 34 Judgment section 35 Selection Section 36 Providing Department 40 Image Information Acquisition Unit 41 Provided information acquisition department 100 Information Processing Systems N Network U User

Claims

1. An image information acquisition unit that acquires information about an image, including image information of the captured image, information about the position where the captured image was taken, information about the date and time the captured image was taken, and information about the manufacturer and model of the imaging device. A learning unit that determines the objects included in the captured image and the type of captured image based on the information of the captured image acquired by the image information acquisition unit, A provision information acquisition unit acquires provision information that includes multiple pieces of proposal candidate information, which are information about proposed candidates for the user. A selection unit selects, as proposed information, a plurality of proposed candidate information included in the provided information acquired by the provided information acquisition unit, according to the target included in the captured image and the acquisition type of the captured image determined by the learning unit, from among the plurality of proposed candidate information included in the provided information acquired by the provided information acquisition unit, The system includes a providing unit that provides the proposed information selected by the selection unit. A terminal device characterized by the following features.

2. comprising an imaging unit, The aforementioned selection unit is If the acquisition type indicates a captured image obtained by screenshot processing, the proposed candidate information corresponding to the determined target is selected as the proposed information. If the acquisition type indicates a captured image taken by the imaging unit, the proposed candidate information corresponding to a location different from the location corresponding to the determined target, and a location that is presumed not to have been visited by the user, is selected as the proposed information. The terminal device according to feature 1.

3. The system includes a location detection unit that detects the current location of the user, The aforementioned information acquisition unit, Using the position information, which is location information obtained by the position detection unit with reduced accuracy from the current location, the provided information, which includes information about locations located around the current location, is acquired from an external device as proposed candidate information. The terminal device according to claim 1 or 2.

4. A reception unit that receives search requests from the aforementioned users, The system includes an identification unit that identifies the search target based on the search request received by the reception unit, The aforementioned selection unit is The proposed information is selected from the proposed candidate information corresponding to the location corresponding to the search target identified by the specified unit. The terminal device according to feature 3.

5. A sensor unit having one or more sensors including the position detection unit, The system includes a determination unit that determines the user's context based on the detection results from the sensor unit, The aforementioned selection unit is From among the multiple candidate proposals included in the provided information acquired by the provided information acquisition unit, the candidate proposal to be selected as the proposed information is selected from among the multiple candidate proposals narrowed down based on the context determined by the determination unit. The terminal device according to feature 4.

6. The aforementioned selection unit is Select the proposed information from the proposed candidate information that corresponds to a location within the range corresponding to the context from the current location. The terminal device according to feature 5.

7. A method of information processing performed by a terminal device, Image information acquisition step: Acquires information about the captured image, including image information of the captured image, information about the position where the captured image was taken, information about the date and time the captured image was taken, and information about the manufacturer and model of the imaging device. A learning step that determines the object included in the captured image and the type of captured image based on the information of the captured image obtained by the image information acquisition step, The process of acquiring information to provide, which includes acquiring information that contains multiple candidate proposals, which are information that can be proposed to the user, A selection step in which, from among a plurality of proposed candidate information included in the provided information obtained by the provided information acquisition step, proposed candidate information corresponding to the target included in the captured image and the acquisition type of the captured image determined by the learning step is selected as proposed information; The process includes a provisioning step of providing the proposed information selected by the selection step. An information processing method characterized by the following:

8. An image information acquisition procedure for acquiring information about an image, including image information of the captured image, information about the position where the captured image was taken, information about the date and time the captured image was taken, and information about the manufacturer and model of the imaging device. A learning procedure that determines the object included in the captured image and the type of captured image based on the information of the captured image obtained by the image information acquisition procedure, A procedure for obtaining information to be provided, which includes multiple pieces of information that are potential proposals to be provided to the user, A selection procedure for selecting, as proposed information, a plurality of proposed candidate information included in the provided information obtained by the provided information acquisition procedure, the proposed candidate information corresponding to the target included in the captured image and the acquisition type of the captured image determined by the learning procedure, A provisioning procedure that provides the proposed information selected by the selection procedure, and a procedure to cause the terminal device to execute the provisioning procedure. An information processing program characterized by the following features.

Images