Information processing method and information processing device

Abstract

Provided is an information processing method for setting a destination of a user on the basis of a selected post image selected by a user operation from among post images posted to a communication service constructed on a prescribed network. This information processing method comprises: determination processing for determining whether position information is associated with a selected post image; and control processing for setting a destination on the basis of the position information when the position information is associated with the selected post image, and for setting a destination on the basis of the position information by acquiring position information retrieved from the network by using the selected post image when the position information is not associated with the selected post image.

Description

Information processing method and information processing device

[0001] The present invention relates to an information processing method and an information processing device that allow a user to set a destination.

[0002] Conventionally, techniques for setting a user's destination have been proposed. For example, JP2014-134515A proposes a technique for displaying posted information on a smartphone and setting a destination using posted location information associated with the displayed posted information. Here, the posted location information is the current location acquired by a location acquisition unit of the smartphone when the posted information is created, or the location or point of a facility selected from map data. Furthermore, when the posted information is created, if an image related to a facility is included in the posted content, the shooting location of the image can be used as the posted location information.

[0003] In the above-described conventional technology, it is possible to set a location corresponding to posted location information embedded in posted information as a destination. However, when the posted information is a posted image, it is possible that the posted image does not have location information attached to it. In this case, it is difficult to set a destination using the posted image.

[0004] The present invention aims to appropriately set a destination using a posted image.

[0005] One aspect of the present invention is an information processing method for setting a user's destination based on a selected posted image selected by a user operation from among posted images posted to a communication service established on a predetermined network. This information processing method includes a determination process for determining whether location information is associated with the selected posted image, and a control process for setting a destination based on the location information if the location information is associated with the selected posted image, and for obtaining location information searched for on the network using the selected posted image and setting a destination based on the location information if the location information is not associated with the selected posted image.

[0006] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system. FIG. 2 is a block diagram illustrating an example of the configuration of a server and an electronic device. FIG. 3A is a diagram illustrating an example of transition of a display screen displayed on a display unit. FIG. 3B is a diagram illustrating an example of transition of a display screen displayed on a display unit. FIG. 4 is a diagram illustrating an example of searching for destination candidates using a posted message. FIG. 5 is a diagram illustrating an example of searching for destination candidates using character information in a posted image. FIG. 6 is a diagram illustrating an example of searching for destination candidates using an image search service. FIG. 7 is a diagram illustrating an example of searching for destination candidates using an image search service. FIG. 8 is a diagram illustrating an example of transition of a display screen displayed on a display unit. FIG. 9 is a diagram illustrating an example of transition when a destination is transferred to a vehicle. FIG. 10 is a flowchart illustrating an example of display control processing. FIG. 11 is a flowchart illustrating an example of destination setting processing. FIG. 12 is a flowchart illustrating an example of destination setting processing. FIG. 13 is a diagram illustrating an example of a database. FIG. 14 is a flowchart illustrating an example of billing information generation processing.

[0007] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0008] [Configuration Example of Information Processing System] FIG. 1 is a diagram showing a configuration example of an information processing system 1. As shown in FIG.

[0009] The information processing system 1 includes a social networking service (SNS) server 10, a server 100, an electronic device 200, and an in-vehicle device 300. These devices are configured to be connectable via a communication method using wired or wireless communication, either directly or via a network 20. The network 20 is a network such as a public line network or the Internet. While FIG. 1 illustrates an example in which the SNS server 10 and the server 100 are configured as single devices, the functions of these servers may be realized by multiple devices. While FIG. 1 illustrates an example in which the SNS server 10 and the server 100 are configured as separate devices, the SNS server 10 and the server 100 may also be configured as an integrated device. While FIG. 1 illustrates only one electronic device 200, multiple electronic devices may also be used.

[0010] The SNS server 10 is an information providing server for realizing a communication service built on a predetermined network. The SNS server 10 performs, for example, information management processing and information providing processing for providing information posted by a user to other users. The SNS server 10 includes a communication unit 11, a control unit 12, and a storage unit 13. The communication unit 11 exchanges various types of information with other devices using wired or wireless communication under the control of the control unit 12.

[0011] The control unit 12 controls each unit based on various programs stored in the storage unit 13. The control unit 12 is realized by, for example, a processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). For example, the control unit 12 executes a posted information management process that stores posted information posted by a certain user in the storage unit 13. Furthermore, for example, the control unit 12 executes a posted information provision process that provides the posted information stored in the storage unit 13 to other users.

[0012] The storage unit 13 is a storage medium that stores various types of information. For example, the storage unit 13 stores various types of information (e.g., a control program, a posted information DB) that is required for the control unit 12 to perform various processes. The storage unit 13 also stores various types of information acquired via the communication unit 11. The storage unit 13 can be, for example, a read-only memory (ROM), a random access memory (RAM), a static random access memory (SRAM), a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0013] The server 100 is an information processing device that realizes a destination search service using posted information managed by the SNS server 10 in response to a request from a user U1 who owns the electronic device 200. The configuration of the server 100 will be described in detail with reference to FIG.

[0014] The electronic device 200 is a device that can display posted information managed by the SNS server 10 and provide it to the user U1. The electronic device 200 also executes a destination search application (destination search app) that sets a destination for the user U1 using the posted information provided by the SNS server 10. The electronic device 200 is realized by, for example, an electronic device such as a smartphone, a tablet terminal, or a portable personal computer, an information processing device, or the like. The configuration of the electronic device 200 will be described in detail with reference to FIG. 2 .

[0015] The in-vehicle device 300 is an in-vehicle system that is installed inside the vehicle C1 and is composed of one or more devices that can provide various types of information. For example, at least one of a navigation device, an audio device, an IVI (In-Vehicle Infotainment), etc. can be used as the in-vehicle device 300. Note that these are merely examples, and other devices installed in the vehicle C1 may also be used.

[0016] The in-vehicle device 300 includes a communication unit 310, a control unit 320, a storage unit 330, and a UI (User Interface) unit 340. The communication unit 310 exchanges various types of information with other devices using wired or wireless communication under the control of the control unit 320.

[0017] The control unit 320 controls each unit based on various programs stored in the storage unit 330. The control unit 320 is realized by a processing device such as a CPU or a GPU, for example.

[0018] The storage unit 330 is a storage medium that stores various types of information. For example, the storage unit 330 stores various types of information (e.g., control programs, map information DB) required for the control unit 320 to perform various processes. The storage unit 330 also stores various types of information acquired via the communication unit 310. The storage unit 330 can be, for example, a ROM, a RAM, an SRAM, an HDD, an SSD, or a combination thereof.

[0019] The UI unit 340 functions as an interface that accepts operations from the occupant of the vehicle C1 and provides various types of information to the occupant of the vehicle C1, and includes a reception unit 341 and a display unit 342. The UI unit 340 outputs the operation content accepted by the reception unit 341 to the control unit 320, and displays various types of information on the display unit 342 based on the control of the control unit 320. The UI unit 340 is installed, for example, on the dashboard 2 (see FIG. 9B ).

[0020] [Configuration Example of Server and Electronic Device] FIG. 2 is a block diagram showing a configuration example of the server 100 and electronic device 200 that constitute the information processing system 1.

[0021] [Example of Server Configuration] The server 100 includes a communication unit 110, a control unit 120, and a storage unit 130.

[0022] The communication unit 110 exchanges various types of information with other devices using wired or wireless communication under the control of the control unit 120 .

[0023] The control unit 120 controls each unit based on various programs stored in the storage unit 130. The control unit 120 is realized by a processing device such as a CPU or a GPU. The control unit 120 includes a determination unit 121, a search unit 122, a setting unit 123, a generation unit 124, and a provision unit 125.

[0024] The determination unit 121 determines whether or not location information is associated with a posted image based on image information (including the posted image and the current location of the electronic device 200) acquired from the electronic device 200 via the communication unit 110, and outputs the determination result to the search unit 122 and the setting unit 123. For example, the determination unit 121 performs a determination process on a posted image selected by a user operation from among the posted images provided to the electronic device 200 from the SNS server 10. This determination process will be described in detail with reference to FIG. 11 and the like. In the present embodiment, the term "posted image" includes both still images and moving images.

[0025] The search unit 122 searches for location information (potential destinations, destinations) related to the posted image based on image information (posted image) acquired from the electronic device 200 via the communication unit 110, and outputs the search results (including the current location of the electronic device 200) to the setting unit 123 and the providing unit 125. For example, if location information is associated with the posted image, the search unit 122 acquires the location information. On the other hand, if location information is not associated with the posted image, the search unit 122 acquires location information searched for on the network using the posted image. For example, the search unit 122 acquires at least one of location information retrieved by an image search using the posted image, location information based on character information contained in the posted image, location information based on text information posted in association with the posted image, and location information based on other posted images other than the posted image. The search unit 122 then searches for destination candidates corresponding to the acquired location information. The process of searching for the location information and potential destinations will be described in detail with reference to FIGS. 4 to 7, 11, etc.

[0026] Based on location information (potential destination) related to a posted image searched for by the search unit 122, the setting unit 123 sets a destination related to the posted image and a travel route from the current location of the electronic device 200 that sent the posted image to the destination. The setting unit 123 then outputs the destination and the travel route to the generation unit 124 and the provision unit 125. For example, the setting unit 123 sets a location corresponding to the location information related to the posted image as the destination. Furthermore, for example, the setting unit 123 sets a travel route from the current location of the electronic device 200 to the destination using a known route search technique. Note that a method for setting the destination and the travel route will be described in detail with reference to FIGS. 10 and 11 , etc.

[0027] The generation unit 124 determines whether the user U1 of the electronic device 200 has arrived at the destination set by the setting unit 123, and if the user U1 has arrived at the destination, generates billing information for the business associated with the destination. The relationship between the destination and the business is stored in the database 600 (see FIG. 13). The method for generating the billing information will be described in detail with reference to FIG. 14.

[0028] The providing unit 125 provides each piece of information output from each unit to the electronic device 200 via the communication unit 110. For example, the providing unit 125 provides destination candidate information related to the destination candidates searched by the searching unit 122 to the electronic device 200 via the communication unit 110. A display example in this case is shown in FIG. 3B (A). Also, for example, the providing unit 125 provides travel route information related to the travel route to the destination set by the setting unit 123 to the electronic device 200 via the communication unit 110. A display example in this case is shown in FIG. 3B (C).

[0029] The storage unit 130 is a storage medium that stores various types of information. For example, the storage unit 130 stores various types of information (e.g., a control program, a database 600 (see FIG. 13 ), and a map information DB) that are necessary for the control unit 120 to perform various processes. The storage unit 130 also stores various types of information acquired via the communication unit 110. The storage unit 130 can be, for example, a ROM, a RAM, an SRAM, an HDD, an SSD, or a combination thereof.

[0030] [Configuration Example of Electronic Device] The electronic device 200 includes a communication unit 210, a control unit 220, a storage unit 230, an image acquisition unit 240, a location information acquisition unit 250, and a UI unit 260. The communication unit 210 exchanges various types of information with other devices using wired or wireless communication based on the control of the control unit 220.

[0031] The control unit 220 controls each unit based on various programs stored in the storage unit 230. The control unit 220 is realized by a processing device such as a CPU or a GPU, for example.

[0032] The storage unit 230 is a storage medium that stores various types of information. For example, the storage unit 230 stores various types of information (e.g., control programs, destination search apps) that are necessary for the control unit 220 to perform various processes. The storage unit 230 also stores various types of information acquired via the communication unit 210. The storage unit 230 can be, for example, a ROM, a RAM, an SRAM, an HDD, an SSD, or a combination thereof.

[0033] The image acquisition unit 240 captures an image of a subject and generates an image (image data) based on the control of the control unit 220, and outputs the generated image to the control unit 220. The image acquisition unit 240 is configured, for example, with an imaging element (image sensor) that receives light from the subject that is collected by a lens (not shown), an image processing unit that performs predetermined image processing on the image data generated by the imaging element, and the like. As the imaging element, for example, a CCD (Charge Coupled Device) type or a CMOS (Complementary Metal Oxide Semiconductor) type imaging element can be used.

[0034] The location information acquisition unit 250 acquires location information regarding the location of the electronic device MC1 and outputs the acquired location information to the control unit 220. The location information acquisition unit 250 can be realized, for example, by a GNSS (Global Navigation Satellite System) receiver that receives GNSS signals and calculates location information based on the GNSS signals. The calculated location information includes data related to the location, such as latitude, longitude, and altitude, at the time of receiving the GNSS signals. Alternatively, a location information acquisition device that acquires location information using other location information acquisition methods may be used. For example, a location information acquisition device may derive location information using information about surrounding wireless LAN access points and acquire this location information. Alternatively, a location information acquisition device may derive location information using information about base stations used for calls and communications and acquire this location information. Alternatively, a location information acquisition device may derive location information using position estimation technology based on a navigation function and acquire this location information. For example, the position of the device itself can be estimated based on sensor information from various sensors (for example, an acceleration sensor, a gyro sensor) and map information.

[0035] The UI unit 260 includes a reception unit 261 and a display unit 262. Although not shown, the UI unit 260 may also include other components such as a sound input unit and a sound output unit. The reception unit 261 and the display unit 262 are examples of user interfaces, and some of them may be omitted, or other user interfaces may be used.

[0036] The reception unit 261 receives various operations from the user U1 and outputs the received operation contents to the control unit 220. The reception unit 261 and the display unit 262 may be configured as a touch panel that allows the user to input operations by touching or approaching the display surface with their finger, or may be configured as a separate user interface. When configured as a separate user interface, various operation members such as buttons and a keyboard can be used as the reception unit 261.

[0037] The display unit 262 displays various images based on the control of the control unit 220. As the display unit 262, for example, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel can be used.

[0038] 3A and 3B are diagrams showing examples of transitions of display screens displayed on the display unit 262 of the electronic device 200. Note that in Fig. 3A and 3B, each display screen is shown in a simplified form for ease of explanation.

[0039] 3A (A) shows an example of displaying an SNS screen 400 for sharing photos, videos, etc. For example, by performing a startup operation to start an SNS app for sharing photos, videos, etc., it is possible to display the SNS screen 400 on the display unit 262. In addition, the SNS screens 400, 410, etc. are displayed based on image information provided by the SNS server 10.

[0040] On the SNS screen 400, a personal image display area 401, a main image display area 402, a posted message display area 403, and a share button 404 are displayed.

[0041] Images that allow identification of friends, acquaintances, etc. of user U1 are displayed in personal image display area 401. For example, a posted image displayed in main image display area 402 can be shared with other users whose images are displayed in personal image display area 401. For example, sharing is possible by sending a link to the image displayed in personal image display area 401 to other users.

[0042] In the main image display area 402, an image selected by a user operation from among the images viewable on the SNS screen 400 is displayed in an enlarged form.

[0043] In the posted message display area 403, posted messages that have been posted in association with the posted images displayed in the main image display area 402 are displayed.

[0044] The share button 404 is a button that is selected when the posted image displayed in the main image display area 402 is to be shared with other users, other apps, and the like.

[0045] 3A(B) shows an example of the SNS screen 400 that is displayed after the share button 404 is selected on the SNS screen 400. For example, when the share button 404 is selected on the SNS screen 400, an SNS screen 410 is displayed that allows an operation to share the posted image displayed in the main image display area 402 with other users, other apps, etc.

[0046] SNS screen 410 displays a personal image display area 401, a main image display area 402, a share button 404, and a share destination selection area 411. Note that, since personal image display area 401, main image display area 402, and share button 404 are the same as those on SNS screen 400, the same reference numerals are used and their description will be omitted.

[0047] Share destination selection area 411 displays share destination selection buttons 412 to 414 for selecting a share destination with which to share the posted image displayed in main image display area 402. Share destination selection button 412 is a button selected when sending the posted image displayed in main image display area 402 to a partner using a predetermined message application. Share destination selection button 413 is a button selected when sending the posted image displayed in main image display area 402 to a partner using email.

[0048] Share destination selection button 414 is a button selected when launching a destination search app for searching for a destination using the posted image displayed in main image display area 402. When share destination selection button 414 is selected, the destination search app is launched, and the posted image displayed in main image display area 402 is transferred to the destination search app. An example of the display in this case is shown in FIG. 3A (C). When the destination search app is installed in electronic device 200, share destination selection button 414 is added to share destination selection area 411. In other words, the destination search app is added to the share destination list of the SNS app.

[0049] FIG. 3A(C) shows an example of a destination search screen 420 that is displayed after the share destination selection button 414 is selected on the SNS screen 410. As described above, when the share destination selection button 414 is selected, the posted image displayed in the main image display area 402 is transferred to the destination search app, and the destination search app executes a search process to search for a destination using the posted image. In other words, the destination search process is executed by the server 100 that realizes the destination search app. FIG. 3B(A) shows an example of a display when destination candidates are searched for by this search process. This search process will be described in detail with reference to FIGS. 4 to 7, 11, 12, etc.

[0050] FIG. 3B(A) shows an example of a destination candidate display screen 430 that is displayed when destination candidates are searched for by the destination search application.

[0051] The destination candidate display screen 430 displays a destination candidate display area 431 and a cancel button 434. The destination candidate display area 431 displays search results (destination candidates) searched by the destination search app. If one destination candidate is searched by the destination search app, that one destination candidate is displayed in the destination candidate display area 431. If multiple destination candidates are searched by the destination search app, the multiple destination candidates are displayed in the destination candidate display area 431. Figure 3B (A) shows an example in which two destination candidates 432 and 433 have been searched.

[0052] The cancel button 434 is a button that is selected when canceling setting a destination from one or more destination candidates displayed in the destination candidate display area 431. In other words, if the destination that user U1 desires is not present among the one or more destination candidates displayed in the destination candidate display area 431, user U1 can cancel the destination search using the posted image by selecting the cancel button 434. Note that when user U1 selects the cancel button 434, the destination search process may be executed again, and new destination candidates may be presented to user U1.

[0053] As shown in Fig. 3B(A), when multiple destination candidates 432, 433 are searched, user U1 checks the contents of the multiple destination candidates 432, 433 and performs a selection operation to select a desired destination. User U1 can, for example, check the addresses, store names, etc. listed in the multiple destination candidates 432, 433 and select a desired destination. Fig. 3B(B) shows an example of a display when user U1 selects destination candidate 433, which is the second candidate.

[0054] Here, as shown in FIG. 3B(A), when multiple destination candidates are searched, the display order can be determined based on a predetermined rule. For example, the display order can be determined based on the distance between each destination candidate and the electronic device 200. For example, multiple destination candidates can be displayed in descending order of distance from the electronic device 200. Furthermore, the display order can be determined based on the name, notation, etc. of each destination candidate. For example, multiple destination candidates can be displayed in alphabetical order (or alphabetical order) of the name and notation of each destination candidate. Here, as will be described later, destination candidates can be searched using an image search service. In this way, when destination candidates are searched using an image search service, the display order can be determined based on the order of the search results (for example, in descending order of the number of searches on the image search service (popularity)). For example, multiple destination candidates can be displayed in descending order of the number of searches on the image search service.

[0055] FIG. 3B (B) shows an example of a transportation means selection screen 440 that is displayed when a destination candidate 433 is selected on the destination candidate display screen 430 .

[0056] The transportation selection screen 440 displays a transportation selection area 441. In the example shown in FIG. 3B (B), a car button 442, a car share button 443, a rental car button 444, a train button 445, and a walking button 446 are displayed. Note that these transportation modes are merely examples, and buttons for selecting other transportation modes may also be displayed. For example, the transportation mode can be changed depending on the destination selected by the user.

[0057] The user U1 executes a selection operation for selecting a moving means to move to the destination selected on the destination candidate display screen 430 shown in FIG. 3B(A) on the moving means selection screen 440. For example, FIG. 3B(C) shows an example of display when an automobile (automobile button 442) is selected by the user U1 as the moving means to move to the destination (Coffee Shop X: O-gun, □-mura, Chiba Prefecture...) selected on the destination candidate display screen 430.

[0058] FIG. 3B(C) shows an example of a moving route guidance screen 450 displayed when a selection operation of a destination candidate 433 is performed on the destination candidate display screen 430 and a selection operation of the automobile button 442 is performed on the moving means selection screen 440.

[0059] On the moving route guidance screen 450, a moving route guidance display area 451 and a map display area 452 are displayed. Various types of information related to the moving route searched according to the moving means selected on the moving means selection screen 440 are displayed in the moving route guidance display area 451 and the map display area 452.

[0060] In the moving route guidance display area 451, various types of information for guiding the moving route to the destination selected on the destination candidate display screen 430 are displayed. For example, place names, addresses, etc. existing on the moving route from the current location to the destination are appropriately displayed.

[0061] In the map display area 452, a current location marker 453, a destination area 454, and moving route information 455 are displayed in the map information. As described above, FIG. 3B(C) shows an example when the automobile button 442 is selected on the destination candidate display screen 430 shown in FIG. 3B(B), so a moving route that can be traveled by vehicle is displayed.

[0062] Furthermore, when the car button 442 is selected, the in-vehicle device 300 (see FIG. 1 ) of the vehicle C1 (vehicle owned by the user U1) registered in the electronic device 200 may be activated based on control information from the control unit 220 of the electronic device 200, and a car navigation screen may be displayed on the UI unit 340. In this case, navigation information such as the travel route guidance screen 450 shown in FIG. 3B(C) may be automatically transferred to the UI unit 340 and displayed. Note that, as shown in FIG. 9 , navigation information such as the travel route guidance screen 450 may be automatically transferred to the UI unit 340 and displayed when the user U1 gets into the vehicle C1.

[0063] For example, when the car share button 443 is selected on the transportation mode selection screen 440, a route to the nearest car share parking lot (nearest car share parking lot) based on the current location of the electronic device 200 and a route from the car share parking lot to the destination are displayed. In this case, when car sharing is initiated at a car share parking lot, navigation information such as the route guidance screen 450 may be automatically transferred to the UI unit 340 and displayed on the navigation device of the vehicle where car sharing has been initiated. Alternatively, a first route guidance screen including a route from the current location to the nearest car share parking lot may be displayed on the display unit 262, and after user U1 arrives at the car share parking lot, a second route guidance screen including a route from the car share parking lot to the destination may be displayed on the display unit 262. In this case, for example, a car share start button may be displayed on the first route guidance screen to notify the administrator that car sharing has been initiated. When the car share start button is selected, the second route guidance screen may be automatically transferred to the navigation screen of the vehicle where car sharing has been initiated.

[0064] For example, when the rental car button 444 is selected on the transportation selection screen 440, a route (which may include contact information such as a telephone number) to the nearest rental car shop (the nearest rental car shop) based on the current location of the electronic device 200 and a route from the rental car shop to the destination are displayed. In this case, when the rental car starts at the rental car shop, navigation information such as the route guidance screen 450 may be automatically transferred to the UI unit 340 and displayed on the navigation device of the vehicle where the rental has started. Alternatively, a first route guidance screen including the route from the current location to the nearest rental car shop and contact information (e.g., a telephone number) may be displayed on the display unit 262, and after the user U1 arrives at the rental car shop, a second route guidance screen including the route from the rental car shop to the destination may be displayed on the display unit 262. In this case, for example, a rental start button may be displayed on the first route guidance screen to notify the administrator that the vehicle rental has started. When the rental start button is selected, the second route guidance screen may be automatically transferred to the navigation screen of the vehicle where the rental has started.

[0065] Also, for example, when the train button 445 is selected on the transportation means selection screen 440, a travel route guidance screen is displayed on the display unit 262, which includes a travel route (e.g., walking, bus, etc.) to a nearby station (e.g., the nearest station) based on the current location of the electronic device 200, and a travel route (e.g., transfer information) from the nearest station to the destination.

[0066] Also, for example, when the walking button 446 is selected on the transportation means selection screen 440, a travel route guidance screen including travel routes that can be traveled on foot from the current location of the electronic device 200 to the destination is displayed on the display unit 262.

[0067] The map information included in the travel route guidance screen 450 may be transferable to various map applications and map search services (for example, GoogleMAP (registered trademark)) that the user U1 is familiar with.

[0068] [Example of searching for destination candidates using text information posted along with a posted image] Figure 4 is a diagram showing an example of searching for destination candidates using posted texts WC1 to WC3 associated with a posted image P1.

[0069] The posted image P1 is an example of an image provided by the SNS server 10. The posted statements WC1 to WC3 are examples of statements posted in relation to the posted image P1. It is assumed that the posted statements WC1 to WC3 may be posted by the poster of the posted image P1 or by users other than the poster of the posted image P1. In other words, the posted statements WC1 to WC3 are comments by the poster of the posted image P1, answers to those comments, comments from followers, etc. FIG. 4 shows an example in which the posted statement WC1 is posted by poster A of the posted image P1, and the posted statements WC2 and WC3 are posted by users other than poster A of the posted image P1.

[0070] 4, it is assumed that the posted text WC1 of the posted image P1 by poster A does not include location information about a place included in the posted image P1. Even in this case, it is assumed that the posted texts WC2 and WC3 posted by users other than poster A include location information about a place included in the posted image P1. Therefore, in this embodiment, if the posted texts WC1 to WC3 posted in relation to the posted image P1 include location information about a place included in the posted image P1, that location information is acquired and used.

[0071] For example, the text portion enclosed by the dotted rectangle AC1 in the text included in the posted text WC2 is considered to be a portion that clearly indicates that the locations included in the posted image P1 are "Town B, Gunma Prefecture" and "XYZ Castle." Therefore, the search unit 122 extracts the text portion enclosed by the dotted rectangle AC1 from the text included in the posted text WC2, and extracts "Town B, Gunma Prefecture" and "XYZ Castle" from the extracted text portion. In this case, known address extraction techniques can be used for address information related to address notation. For example, a database (e.g., a map information database) storing address notations can be used to extract notations that are identical (or partially identical) to the address notations stored in the database. Furthermore, known name extraction techniques can be used for name information related to notations of landmarks, tourist destinations, famous places, etc. For example, a database (e.g., a map information database) storing notations of landmarks, tourist destinations, famous places, etc. can be used to extract notations that are identical (or partially identical) to the names stored in the database. The address notation within the dotted rectangle AC2 included in the posted message WC3 can also be extracted in a similar manner.

[0072] [Example of Searching for Destination Candidates Using Text Information Included in a Posted Image] FIG. 5 is a diagram showing an example of searching for destination candidates using text information included in a posted image P2.

[0073] Posted image P2 is an example of an image provided by the SNS server 10. It is also assumed that posted image P2 includes a signboard (within the dotted rectangle AC3) with text related to a castle in the background. It is thus assumed that text information related to a location is included in posted image P2. Therefore, in this embodiment, if text information is included in posted image P2, the text information is acquired, and location information is acquired and used based on the text information. For example, if at least one character string from among a telephone number, a store name, a name, an address, etc. is included, the location information can be searched for using the character string. For example, if character strings such as "ABC Gohanya" or "XYZ Shokudo" are included, the location information can be searched for using the character string.

[0074] For example, the portion enclosed by the dotted rectangle AC3 in the posted image P2 contains the sentence "Welcome to XYZ Castle in Gunma." This sentence is considered to clearly indicate that the location included in the posted image P2 is "Gunma Prefecture" and "XYZ Castle." Therefore, the search unit 122 extracts the portion enclosed by the dotted rectangle AC3 in the posted image P2 and extracts "Gunma Prefecture" and "XYZ Castle" from the extracted portion. In this case, known character recognition technology (e.g., OCR (Optical Character Recognition) or AI technology) can be used as an extraction technology for extracting characters from the image. Furthermore, known address extraction technology and name extraction technology can be used for address information related to address notation and name information related to names of landmarks, tourist destinations, famous places, etc., as in the example shown in FIG. 4.

[0075] In the examples shown in FIGS. 4 and 5 , location information for characters extracted from a posted message or an image may be obtained using a known character search service (e.g., Google (registered trademark) or Yahoo! (registered trademark)). For example, the search unit 122 outputs the characters extracted from the posted message or the image to a character search server that provides a character search service via the network 20, and obtains search results from the character search server. The search unit 122 determines whether location information (e.g., address, name) is present in the search results from the character search server. Then, if location information is included in the search results, the search unit 122 can use the location information.

[0076] [Example of Searching for Destination Candidates Using Image Search] FIG. 6 is a diagram showing an example of searching for destination candidates by acquiring location information related to a posted image P3 using an image search service.

[0077] The posted image P3 is an example of an image provided by the SNS server 10. Here, an image search service is provided that allows a search for information related to an image based on the image. Examples of such image search services (image search engines) include Google and Yahoo!. Therefore, it is possible to search for information related to the posted image P3 using these image search services. Then, if location information is found as information related to the posted image P3, that location information is used.

[0078] Specifically, the search unit 122 outputs the posted image P3 to an image search server that provides an image search service via the network 20 and obtains search results from the image search server. When the search unit 122 obtains text information as a search result from the image search server, it determines whether the text information contains location information. If the text information contains location information, the search unit 122 can use the location information. For address information related to address notation and name information related to names of landmarks, tourist destinations, famous places, etc., publicly known address extraction techniques and name extraction techniques can be used, as in the example shown in FIG. 4. When the search unit 122 obtains image information as a search result from the image search server, it can extract text information contained in the image information and obtain location information contained in the text information. An example of obtaining similar image information from an image search server is shown in FIG. 7.

[0079] [Example of Searching for Destination Candidates Using Similar Images] FIG. 7 is a diagram showing an example of searching for destination candidates by acquiring image information similar to the posted image P3 using an image search service.

[0080] Posted image P3 is similar to posted image P3 shown in FIG. 6. As described above, image search services are provided that allow users to search for information related to an image based on that image. Therefore, it is possible to use these image search services to search for image information similar to posted image P3. Then, image information similar to posted image P3 is searched for, and location information is obtained and used using the similar image information.

[0081] Specifically, the search unit 122 outputs the posted image P3 to an image search server that provides an image search service via the network 20 and obtains search results from the image search server. When the search unit 122 obtains image information as search results from the image search server, it obtains image information of highly likely similar images from the search results. Here, highly likely similar images are, for example, images that are ranked high in the image search. Alternatively, a predetermined number of highly likely similar images may be obtained from the highly likely similar images. For example, when images P11 to P13 are searched in this order as images similar to the posted image P3, the image P11 determined to be the most similar is obtained. Note that if no search results are obtained from the image search server (i.e., if no similar images exist), another search process is performed.

[0082] The search unit 122 can then use the image P11 determined to be most similar to perform the above-described searches and obtain location information. Note that on the SNS screen, the image display screen displaying one or more images can be switched to another page, and it is possible that similar images may exist on the switched page. Therefore, image information similar to the posted image P3 may be searched for among multiple images present on other pages on the SNS screen, and location information may be obtained and used using the similar image information. The above-described image search service can also be used for the search process for similar images using the SNS screen.

[0083] [Example of Using a Captured Image or a Saved Image] Fig. 8 is a diagram showing an example of transition of display screens displayed on the display unit 262 of the electronic device 200. Note that Fig. 8 shows each display screen in a simplified manner for ease of explanation.

[0084] 8A shows an example of a case where menu screen 460 is displayed. For example, in display area 462 displaying icons 463 to 466 for launching various apps, the user can select a desired icon to display the desired app on display unit 262. FIG. 8A shows an example of a case where a selection operation to select search app icon 465 has been performed. In this way, a selection operation to select search app icon 465 can launch a destination search app, and an initial screen (destination search screen 470) of the destination search app can be displayed on display unit 262.

[0085] FIG. 8(B) shows an example of a destination search screen 470. The destination search screen 470 displays a search by image button 471, a search by text button 472, and a settings button 473. The search by image button 471 is an operation button for displaying a display screen (destination search screen 480) for searching for destinations using various images. FIG. 8(C) shows an example of a display when the search by image button 471 is selected. The search by text button 472 is an operation button for displaying a display screen for searching for destinations using text information. Note that on this text search screen, it is possible to search for destinations using, for example, a text search service for entered text. The settings button 473 is an operation button for displaying a settings screen for configuring various settings related to the destination search app.

[0086] FIG. 8C shows an example of a destination search screen 480. The destination search screen 480 displays an image display area 481 and a select saved image button 482. The image display area 481 can display captured images acquired by the image acquisition unit 240 (see FIG. 2 ). For example, a user can determine an image to use in an image search while viewing the captured images displayed in the image display area 481. The select saved image button 482 is an operation button for displaying a display screen for selecting an image to use in an image search from among images (image files) stored in the storage unit 230. In this manner, both the images acquired by the image acquisition unit 240 and the images stored in the storage unit 230 can be used in a destination search. For example, when a destination search is performed using a posted image posted by user U1, the posted image and similar images are often stored in the storage unit 230. Therefore, when a destination search is performed using a posted image posted by user U1, the images stored in the storage unit 230 can be used as search targets.

[0087] [Example of Transferring a Destination Set Using an Electronic Device to a Vehicle] FIG. 9 is a diagram showing an example of a transition when a destination set using the electronic device 200 is transferred to the vehicle C1.

[0088] 9A shows a situation in which a user U1 who has set a destination using a destination search app in the electronic device 200 gets into a vehicle C1 while carrying the electronic device 200. The travel route to the destination set using the destination search app is shown in FIG.

[0089] 9(B) shows a travel route guidance screen 450 that provides guidance on a travel route set using a destination search application in the electronic device 200. Note that the travel route guidance screen 450 is similar to the travel route guidance screen 450 shown in FIG. 3B(C), and is therefore denoted by the same reference numeral.

[0090] 9C shows a travel route guidance screen 490 displayed on the display unit 342 of the in-vehicle device 300 after the user U1 gets into the vehicle C1. For example, assume that the electronic device 200, with the travel route guidance screen 450 displayed on the electronic device 200, enters a range capable of receiving radio waves (e.g., Wi-Fi (registered trademark)) emitted by the communication unit 310 of the in-vehicle device 300. In this case, the control unit 220 of the electronic device 200 can automatically transmit screen information corresponding to the travel route guidance screen 450 to the in-vehicle device 300. That is, when the user U1, who has set a destination using a destination search app on the electronic device 200, gets into the vehicle C1 while carrying the electronic device 200, the travel route guidance screen 450 displayed on the electronic device 200 is transferred to the in-vehicle device 300 and displayed on the display unit 342, as indicated by the arrow AW1.

[0091] It is also assumed that when user U1 travels by train, he or she will travel to the nearest station in vehicle C1. In such a case, even if the travel route guidance screen 490 is displayed in vehicle C1, it is also assumed that user U1 or other passengers will not see the travel route guidance screen 490. Therefore, the above-described transfer process may be performed only for the travel route guidance screen 450 that is displayed after the car button 442 is selected on the transportation means selection screen 440 shown in FIG. 3B (B).

[0092] [Example of Operation of Electronic Device] Fig. 10 is a flowchart showing an example of display control processing in the electronic device 200. This display control processing is executed by the control unit 220 (see Fig. 2) based on a program stored in the storage unit 230 (see Fig. 2). This display control processing is executed, for example, at the timing when a user operation is performed to display an SNS screen on the electronic device 200. This display control processing will be described with appropriate reference to Figs. 1 to 9.

[0093] In step S501, the control unit 220 displays an SNS screen on the display unit 262 based on information from the SNS server 10. For example, an SNS screen 400 is displayed on the display unit 262 as shown in FIG.

[0094] In step S502, the control unit 220 determines whether a user operation has been accepted by the accepting unit 261. That is, it is determined whether a user operation has been accepted. If a user operation has been accepted, the process proceeds to step S503. On the other hand, if a user operation has not been accepted, the process continues monitoring.

[0095] In step S503, the control unit 220 determines whether or not a selection operation to select the share button has been performed. If a selection operation to select the share button has been performed, the process proceeds to step S504. On the other hand, if a selection operation to select the share button has not been performed, the process proceeds to step S505.

[0096] In step S504, the control unit 220 displays the share destination list on the display unit 262. For example, as shown in FIG.

[0097] In step S505, the control unit 220 determines whether or not a selection operation for selecting the destination search application button has been performed. If a selection operation for selecting the destination search application button has been performed, the process proceeds to step S506. On the other hand, if a selection operation for selecting the destination search application button has not been performed, the process proceeds to step S510.

[0098] In step S506, control unit 220 starts the destination search app and transfers the main image to the destination search app. For example, as shown in Fig. 3A (B), when SNS screen 410 is displayed on display unit 262 and a selection operation is performed to select share destination selection button 414, the posted image displayed in main image display area 402 is transferred to the destination search app.

[0099] In step S507, the control unit 220 erases the SNS screen displayed on the display unit 262.

[0100] In step S508, the control unit 220 displays a destination search application screen on the display unit 262. For example, as shown in FIG.

[0101] In step S509, the control unit 220 executes a destination search process. Specifically, the control unit 220 transmits the main image transferred to the destination search application in step S506 to the server 100. The destination search process by the server 100 will be described in detail with reference to FIGS. 11 and 12 .

[0102] In step S510, the control unit 220 determines whether a predetermined SNS operation has been performed on the SNS screen. If a predetermined SNS operation has been performed, the process proceeds to step S511. On the other hand, if a predetermined SNS operation has not been performed, the process proceeds to step S512.

[0103] In step S511, the control unit 220 changes the display state on the SNS screen in response to a user operation.

[0104] In step S512, the control unit 220 determines whether an SNS end operation has been performed on the SNS screen. If an SNS end operation has been performed, the process proceeds to step S513. On the other hand, if an SNS end operation has not been performed, the process returns to step S502.

[0105] In step S513, the control unit 220 erases the SNS screen.

[0106] 11 and 12 are flowcharts showing an example of a destination setting process in the server 100. The destination setting process is executed by the control unit 120 (see FIG. 2) based on a program stored in the storage unit 130 (see FIG. 2). The destination setting process is executed, for example, when selection information for a posted image (selection information for the main image transferred to the destination search app in step S506 (see FIG. 10)) is received from the electronic device 200. The destination setting process will be described with reference to FIGS. 1 to 10 as needed.

[0107] 11 and 12 show an example in which the destination setting process is executed on the server 100, but at least a part of this destination setting process may be executed on another device (e.g., the electronic device 200), or the destination setting process may be executed on the electronic device 200.

[0108] In step S521, the determination unit 121 acquires target image information (e.g., an image file) to be used in the destination setting process. For example, when the determination unit 121 receives selection information of a posted image from the electronic device 200, the determination unit 121 acquires the posted image (image file) related to the selection information as the target image information.

[0109] In step S522, the determination unit 121 determines whether location information is attached to the target image information acquired in step S521. For example, image files such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), and TIFF (Tagged-Imagefile Format) contain embedded Exif (Exchangeable image file format) data that stores various attribute information. This Exif data stores location information (e.g., GNSS information) that indicates the location where the image file was taken. However, in order to protect personal information, etc., it is also possible that the function for adding location information to an image file at the time of capture may be turned off. In this case, location information is not stored in the Exif data.

[0110] Therefore, the determination unit 121 determines whether or not location information is stored in the Exif data of the target image information (image file). If location information is attached to the target image information, the process proceeds to step S523. On the other hand, if location information is not attached to the target image information, the process proceeds to step S524.

[0111] In step S523, the search unit 122 extracts location information attached to the target image information. For example, the search unit 122 extracts location information stored in the Exif data of the image file.

[0112] In step S524, the search unit 122 determines whether text information is attached to the target image information. For example, as shown in FIG. 4, if posted images P1 are associated with posted statements WC1 to WC3, it can be determined that text information is attached to the target image information. If text information is attached to the target image information, the process proceeds to step S525. On the other hand, if text information is not attached to the target image information, the process proceeds to step S527.

[0113] In step S525, the search unit 122 extracts characters including location information based on text information attached to the target image information. For example, as shown in FIG. 4, if posted texts WC1 to WC3 are associated with the posted image P1, the search unit 122 extracts characters within dotted rectangles AC1 and AC2 from the posted texts WC1 to WC3 as characters including location information.

[0114] In step S526, the search unit 122 determines whether characters including location information have been extracted based on the text information accompanying the target image information. For example, in the example shown in FIG. 4, characters within dotted rectangles AC1 and AC2 have been extracted as characters including location information from among the posted messages WC1 to WC3, so it can be determined that characters including location information have been extracted. If characters including location information have been extracted, the process proceeds to step S538. On the other hand, if characters including location information have not been extracted, the process proceeds to step S527.

[0115] In step S527, the search unit 122 determines whether the image of the target image information contains text. For example, in the example shown in FIG. 5, the text within the dotted rectangle AC3 in the posted image P2 is extracted, so it can be determined that text has been extracted. If the image of the target image information contains text, the process proceeds to step S528. On the other hand, if the image of the target image information does not contain text, the process proceeds to step S530.

[0116] In step S528, the search unit 122 extracts characters included in the image of the target image information. For example, in the example shown in Fig. 5, characters within a dotted rectangle AC3 are extracted from the posted image P2.

[0117] In step S528, the search unit 122 determines whether or not location information can be estimated based on the characters extracted in step S528. For example, as described above, it determines whether or not location information can be acquired using a known location information estimation technique. If location information can be estimated, the process proceeds to step S538. On the other hand, if location information cannot be estimated, the process proceeds to step S530.

[0118] In step S530, the search unit 122 executes a search process to search for location information based on the target image information. For example, as shown in Fig. 6, the search unit 122 can search for location information using an image search service. Specifically, the search unit 122 outputs the target image information to an image search server and obtains the search results.

[0119] In step S531, the search unit 122 determines whether or not location information can be extracted based on the search results obtained in step S530. For example, as shown in FIG. 6, the search unit 122 searches for location information using an image search service, obtains the search results, and determines whether the search results include location information. If the search results include location information, the process proceeds to step S538. On the other hand, if the search results do not include location information, the process proceeds to step S532.

[0120] In step S532, the search unit 122 executes a search process to search for similar images that are similar to the target image information. For example, as shown in Fig. 7, the search unit 122 can use an image search service to acquire images P11 to P13 that are similar to the posted image P3. In this case, the search unit 122 acquires image P11, which is determined to be the most similar of images P11 to P13.

[0121] In step S533, the search unit 122 determines whether a similar image similar to the target image information has been found. If a similar image has been found, the process proceeds to step S537. On the other hand, if a similar image has not been found, the process proceeds to step S534.

[0122] In step S534, the search unit 122 determines whether or not there is other image information on the SNS by the same poster as the poster of the target image information. Specifically, the search unit 122 can confirm, through communication with the SNS server 10, whether or not there is other image information by the same poster as the poster of the target image information. In this case, it is preferable to determine whether or not there is other image information within a predetermined date and time (i.e., a close date and time) based on the posting date of the target image information. For example, it is possible to determine whether or not there is other image information within several days based on the posting date of the target image information. If there is other image information by the same poster, the process proceeds to step S537. On the other hand, if there is no other image information by the same poster, the process proceeds to step S535.

[0123] In step S535, the search unit 122 determines whether or not there is posted information (including a similar image) by another poster who is connected to the poster of the target image information on the SNS. Specifically, the search unit 122 can confirm, through communication with the SNS server 10, whether or not there is posted information (including a similar image) by another poster who is connected to the poster of the target image information on the SNS. In this case, it is preferable to determine whether or not there is posted information with a similar image by another poster within a predetermined date and time (i.e., a close date and time) based on the posting date of the target image information. For example, it is possible to determine whether or not there is image information with a similar image by another poster within several days based on the posting date of the target image information. If there is posted information by another poster, the process proceeds to step S537. On the other hand, if there is no posted information by another poster, the process proceeds to step S536.

[0124] In step S536, the providing unit 125 executes a notification process to notify the user U1 of the electronic device 200 that the destination cannot be estimated using the target image information. For example, it is possible to notify the user U1 of the fact that the destination cannot be estimated using the target image information by displaying a notification screen on the display unit 262 of the electronic device 200.

[0125] In step S537, the search unit 122 sets the image information acquired in steps S533, S534, and S535 (similar images, other image information by the same poster, and image information by a different poster) as new target image information. In this case, the process returns to step S522, and new processing such as acquiring location information is executed.

[0126] In step S538 shown in FIG. 12 , the search unit 122 searches for destination candidates corresponding to the target image information based on the location information acquired in steps S523, S526, S529, and S531. For example, if latitude and longitude are acquired as location information, a location corresponding to the location information can be identified based on the latitude and longitude and the map information DB. Also, for example, if the name of a specific location (e.g., a theme park, park, or amusement park) is acquired as location information, a location corresponding to the name of the specific location can be identified based on the name of the specific location and the map information DB. The location identified in this manner is identified as a destination candidate. In this case, it is possible that one destination candidate or multiple destination candidates will be identified.

[0127] In step S539, the search unit 122 determines whether any of the destination candidates found in step S538 includes a location requiring privacy protection (a private area). That is, it determines whether any of the destination candidates includes a private area. If at least one of the destination candidates includes a private area, the process proceeds to step S540. On the other hand, if none of the destination candidates includes a private area, the process proceeds to step S541.

[0128] Here, for example, a location requiring privacy protection refers to the location of a personal home, such as a residential area. For example, locations requiring privacy protection can be set in advance in the map information DB of the storage unit 130. For example, the search unit 122 can use the map information DB of the storage unit 130 to determine whether the location of a destination candidate is a location requiring privacy protection (e.g., a location in a residential area, etc.). However, in residential areas, there are also businesses (e.g., bakeries) that are privately operated in private homes. It is considered that privacy protection is not required for such businesses. Therefore, the locations of such businesses can also be registered in the map information DB of the storage unit 130. Furthermore, the search unit 122 may use artificial intelligence (AI) to determine whether a destination candidate has a private area. For example, locations on a map requiring privacy protection and the names of such locations on a map requiring privacy protection (e.g., ABC Bakery) can be learned in advance, and this learned data can be used for the determination. Then, the search unit 122 can use the learning data to determine whether or not a destination candidate corresponds to a private area.

[0129] Furthermore, for example, it may be possible to determine whether a destination candidate corresponding to target image information corresponds to a private area based on whether the target image information corresponds to a private photo. Whether the target image information corresponds to a private photo can be determined, for example, using AI. For example, images that correspond to private photos and images that do not correspond to private photos can be learned in advance, and this learning data can be used for the determination.

[0130] In this way, when a private photo is uploaded as a posted image, in order to prevent location identification from the posted image, if a location requiring privacy protection (for example, the location of a private home) is found as a search result, that location will be excluded from the destination.

[0131] In step S540, the search unit 122 excludes from the destination candidates those destination candidates determined to correspond to private areas in step S539. If multiple destination candidates were found in step S538, only those destination candidates determined to correspond to private areas are excluded from the multiple destination candidates. However, it is possible that only one destination candidate was found in step S538, or that multiple destination candidates were found in step S538 but all of the destination candidates were determined to correspond to private areas. In these cases, all destination candidates are excluded in step S540. If all destination candidates are excluded in step S540, the process proceeds to step S532, where a search process for location information using a new image related to the target image information is performed, or the destination setting process operation may be terminated.

[0132] In step S541, the providing unit 125 provides destination candidate information related to one or more destination candidates found in step S532 (excluding the destination candidates excluded in step S540) to the electronic device 200, and causes the electronic device 200 to display the destination candidate information. For example, as shown in FIG. 3B(A), a destination candidate display screen 430 is displayed on the display unit 262.

[0133] In step S542, the setting unit 123 determines whether or not a decision operation to decide a destination from among one or more destination candidates provided to the electronic device 200 in step S541 has been performed in the electronic device 200. If a decision operation to decide a destination has been performed, the process proceeds to step S543. On the other hand, if a decision operation to decide a destination has not been performed, the process returns to step S541.

[0134] 3B(A), when a selection operation of a destination candidate 433 is performed by the user U1 on the destination candidate display screen 430, the control unit 220 of the electronic device 200 transmits selection information indicating that the destination candidate 433 has been selected to the server 100. When the setting unit 123 receives the selection information from the electronic device 200, it can determine that a determination operation to determine a destination has been performed in the electronic device 200. Note that the selection information includes location information indicating the current location of the electronic device 200.

[0135] In step S543, the providing unit 125 provides the transportation means information for selecting the transportation means to be used to reach the destination determined in step S542 to the electronic device 200, and causes the electronic device 200 to display the transportation means information. For example, as shown in FIG. 3B (B), a transportation means selection screen 440 is displayed on the display unit 262.

[0136] In step S544, the determination unit 121 determines whether or not a selection operation to select a means of transportation to be used to reach the destination has been performed on the electronic device 200 on the transportation selection screen displayed on the electronic device 200 in step S543. If a selection operation to select a means of transportation has been performed, the process proceeds to step S545. On the other hand, if a decision operation to decide the destination has not been performed, the process returns to step S543.

[0137] For example, as shown in FIG. 3B (B), when the user U1 selects the car button 442 on the transportation means selection screen 440, the control unit 220 of the electronic device 200 transmits selection information indicating that the car button 442 has been selected to the server 100. When the setting unit 123 receives the selection information from the electronic device 200, it can determine that a selection operation to select a transportation means has been performed on the electronic device 200. Note that while FIGS. 3A and 3B show an example in which one transportation means is selected, a combination of multiple transportation means may also be selected. In this case, a travel route using the combination of multiple transportation means is searched for.

[0138] In step S545, the setting unit 123 searches for and sets a travel route from the current location of the electronic device 200 to the destination determined in step S542 using the travel mode selected in step S544. A known search technique can be used for this travel route search process. For example, if a car is selected as the travel mode, roads that can be traveled by the vehicle are set as the travel route. Furthermore, for example, if a car share or rental car is selected as the travel mode, a travel route, such as walking, to a location where the vehicle can be picked up (a car share parking lot or a rental car shop) and roads that can be traveled by the vehicle from that location to the destination are set as the travel route. Furthermore, for example, if a train is selected as the travel mode, a travel route, such as walking, to the nearest station to the current location of the electronic device 200 and a travel route along the train tracks from the nearest station to the destination are set. For example, if walking is selected as the travel mode, roads that can be traveled by foot are set as the travel route.

[0139] The above describes an example in which, when multiple destination candidates are searched, the user sets a destination from among the multiple destination candidates. However, it is also possible to automatically select and set a destination that can be reached using the transportation means selected by the user from among the multiple destination candidates. For example, when a car is selected as the transportation means, a destination within a range that can be reached using the vehicle (e.g., around a highway or a public road) can be selected and set. When a train is selected as the transportation means, a destination within a range that can be reached using the train (e.g., around an available train station) can be selected and set. When walking is selected as the transportation means, a destination within a range that can be reached on foot (e.g., a distance that the user can walk) can be set.

[0140] Furthermore, for example, when multiple destination candidates are searched for as a destination, a destination may be automatically selected and set from the multiple destination candidates based on the distance from the current location of the user U1. That is, it is possible to preferentially set a location close to the current location of the user U1 as a destination. For example, it is possible to set the destination candidate that is closest to the current location of the user U1 as the destination.

[0141] In step S546, the providing unit 125 provides the travel route information regarding the travel route to the destination set in step S545 to the electronic device 200, and causes the electronic device 200 to display the travel route information. For example, as shown in FIG. 3B(C), a travel route guidance screen 450 is displayed on the display unit 262.

[0142] In addition, if a car, car sharing, or rental car is selected as the means of transportation, as shown in Figure 9, the travel route guidance screen 450 displayed on the electronic device 200 may be transferred to a display device inside the vehicle when the user U1 gets into the vehicle in question.

[0143] 11 and 12 are merely examples for implementing this embodiment, and the order of some of the steps may be changed, some of the steps may be omitted, or other steps may be added, as long as this embodiment is still feasible. For example, steps S522 and S523 may be omitted, and location information searched for on a network using a posted image may be acquired, regardless of whether location information is associated with the posted image, and a destination and travel route may be set based on the location information. Also, for example, while FIG. 11 illustrates an example in which a text information search process (steps S524 to S526), ​​a character information search process (steps S527 to S529), and a location information search process using an image search service (steps S530 to S531) are executed sequentially, at least one of these processes may be executed. Also, for example, Figure 11 shows an example in which a search process for similar images (steps S532 to S533), a search process for other image information by the same poster (step S534), and a search process for other image information by a different poster (step S535) are executed sequentially, but at least one of these processes may be executed.

[0144] [Example of Making a Predetermined Bill to a Business Associated with a Set Destination] Here, in order to guide a customer to a certain destination (e.g., a commercial facility, theme park, or amusement park), it is conceivable that the server 100 introduces the destination. In this case, it is also conceivable that a fee for the introduction is charged to the business associated with the destination (e.g., an administrator). Therefore, the following shows an example in which a predetermined destination and a predetermined business are associated with each destination and stored in the database 600, and when the destination is set, a predetermined bill is made to the business associated with the destination.

[0145] 13 is a diagram showing an example of a database 600 that stores destinations 601 and businesses 602 in association with each other for each destination. The database 600 is stored in the storage unit 130 or an external device, for example.

[0146] The destination 601 is information for identifying a destination stored in the database 600. Although Fig. 13 shows an example in which only an address is stored as the destination 601, other information that can identify the destination (for example, location information such as latitude and longitude, or a name) may also be stored.

[0147] The business operator 602 is information stored in the database 600 that indicates a business operator associated with the destination.

[0148] For example, it is conceivable to propose to businesses that they register images in association with destinations in a destination search app. In this case, for example, it is possible to provide services that increase the ranking of images related to destinations registered in the destination search app in image searches or make the images easier to see (correct unclear or blurry images). This makes it easier for users of the destination search app to identify images related to the destinations registered in the destination search app. This is expected to increase the number of users who set destinations registered in the destination search app.

[0149] [Example of Server Operation] Fig. 14 is a flowchart showing an example of a billing information generation process in the server 100. This billing information generation process is executed by the control unit 120 (see Fig. 2) based on a program stored in the storage unit 130 (see Fig. 2). This billing information generation process is executed, for example, when a travel route to a destination is set using a destination search app of the electronic device 200. This billing information generation process will be described with appropriate reference to Figs. 1 to 13.

[0150] In step S551, the generation unit 124 determines whether the user U1 carrying the electronic device 200 has arrived at the destination determined in step S542 (see FIG. 12 ). For example, the control unit 220 of the electronic device 200 periodically or irregularly transmits location information acquired by the location information acquisition unit 205 to the server 100, thereby allowing the generation unit 124 of the server 100 to acquire the current location of the electronic device 200. The generation unit 124 then compares the current location of the electronic device 200 with the destination determined in step S542 (see FIG. 12 ). If the current location matches (or substantially matches), the generation unit 124 can determine that the user U1 carrying the electronic device 200 has arrived at the destination. If the user U1 has arrived at the destination, the process proceeds to step S552. On the other hand, if the user U1 has not arrived at the destination, the process continues monitoring.

[0151] In step S552, the generation unit 124 determines whether a business operator is associated with the destination determined to be the arrival destination of user U1 in step S551 in the database 600 (see FIG. 13). If a business operator is associated with the destination where user U1 has arrived, the process proceeds to step S553. On the other hand, if a business operator is not associated with the destination where user U1 has arrived, the operation of the billing information generation process ends.

[0152] In step S553, the generation unit 124 generates billing information for the business associated with the destination where user U1 has arrived. This billing information includes the amount of advertising fees, compensation, etc. for introducing the destination to user U1 using the destination search app. The billing information is generated based on preset billing details.

[0153] In step S553, the providing unit 125 transmits the billing information generated in step S553 to the business associated with the destination where the user U1 has arrived. Note that, although an example is shown in Fig. 14 in which billing information is transmitted to the business every time it is generated, billing information for a predetermined period may be transmitted collectively.

[0154] [Example of a Case Where a Final Destination is Specified by a User] The above describes an example in which a user's final destination is set using a posted image. Here, it is also possible for the user to manually set the final destination and then set the user's stop destinations (places to stop along the way) from the surrounding areas of the travel route to the final destination using a posted image. For example, if Osaka is specified as the final destination, it is also possible for the user to set the user's stop destinations (e.g., ramen shops) from the surrounding areas of the travel route to the final destination using a posted image (e.g., an image of a ramen shop). In this case, when multiple destination candidates are searched, a stop destination is set (or displayed as a top candidate) from the surrounding areas of the travel route to the final destination. For example, if Osaka is specified as the final destination and multiple destination candidates (e.g., ramen shops in Nagoya and Sapporo) are searched, the ramen shop in Nagoya, which is located in the surrounding area of ​​the travel route to the final destination, is set (or displayed as a top candidate) as a stop destination. In this way, when a final destination is specified in advance, it is possible to prioritize locations close to the final destination in the search results.

[0155] [Example of Effect of the Present Embodiment] In recent years, an increasing proportion of young people are searching using images on social networking sites (SNSs) rather than text searches. Young people, particularly those in their teens and twenties, tend to use images or videos to gather information. However, on image-based SNSs, searched images often do not include location information. For example, images posted on image-based SNSs often include both location information and images without location information. For example, when a user finds an interesting image posted on an image-based SNS (without location information), they often want to experience or receive the same service as the image. In this case, when searching for the location of a store or other establishment where they can receive that experience or service, the user must find the store's name from the posted image, using information from other users who have posted similar images, and then use that name to search for the store's location. The user then enters the location into a navigation device to set the destination. Thus, searching for a destination using images on an image-based SNS requires multiple processes, which often consumes time and effort, causing many users to struggle. Furthermore, seamless destination setting is considered difficult. If the location of the store is unclear or the method of getting to the store is unclear, it is possible that users will change their destination to another location that is easier to access. In this case, even though there are users who wish to visit the store, they will lose the opportunity to use the store, which could be a factor in reducing the store's customer attraction rate.

[0156] In particular, while traveling (driving) using a vehicle, the driver cannot manually set the destination using an image-based SNS. It is conceivable that the driver could set the destination using voice control on an image-based SNS, but voice control has limitations when safety is taken into consideration. Therefore, it is conceivable that a passenger other than the driver could search for and set the destination. Even in this case, as described above, multiple processes may be required, potentially taking a long time to set the destination. Furthermore, when safety is taken into consideration, there are limitations to sharing the destination and posted images with the driver.

[0157] Therefore, in this embodiment, a destination search app is used to seamlessly connect a travel route to a destination using posted images found on an image-based SNS or the like. This makes it possible to remove the boundary between the SNS and the real world, and easily realize a method of jumping out of the SNS into the real world. In other words, the complexity of searching for the destination of a posted image found on an image-based SNS or the like is eliminated, and the user can be connected to a navigation device or the like while maintaining their motivation to reach the destination of the posted image. This makes it possible to easily guide a user who views a posted image found on an image-based SNS or the like to the destination of the posted image. In this way, this embodiment makes it possible to guide user U1 to a destination without the user U1 having to search for the destination themselves. Furthermore, it is possible to appropriately set the destination using posted images.

[0158] Similarly, when a vehicle is selected as a means of transportation, it is possible to seamlessly connect the route to the destination. For example, by linking a destination search app with an in-vehicle device 300 (e.g., a navigation device), it is possible to omit a series of operations such as determining a destination from a posted image and providing route guidance, thereby reducing the work of determining a destination on the navigation device in the vehicle. This makes it possible to increase the likelihood of selecting a vehicle as a means of transportation and to promote new value for the vehicle.

[0159] [Example of Executing Processing in Other Devices or Systems] Although the above describes an example in which the determination process, search process, destination setting process, etc. are executed in the server 100 (or the information processing system 1), all or part of each of these processes may be executed in other devices. In this case, the information processing system is configured by the devices that execute part of each of these processes. For example, at least part of each process may be executed using various information processing devices and various electronic devices, such as the electronic device 200, the in-vehicle device 300, devices available to the user (e.g., smartphones, tablet terminals, personal computers, car navigation devices, IVIs), and servers connectable via a predetermined network such as the Internet.

[0160] Furthermore, a part (or all) of the information processing system capable of executing the functions of the server 100 (or the information processing system 1) may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[0161] [Configuration Example and Effects of the Present Embodiment] The information processing method according to the present embodiment is an information processing method for setting a destination for user U1 based on a selected posted image selected by a user operation from among posted images posted to a social networking service (SNS) (an example of a communication service) established on a predetermined network (e.g., network 20). This information processing method includes a determination process (step S522) for determining whether location information is associated with the selected posted image, and a control process (steps S523 to S546) for setting a destination and a travel route to the destination based on the location information if location information is associated with the selected posted image, and for acquiring location information searched for on the network using the selected posted image and setting a destination and a travel route based on the location information if location information is not associated with the selected posted image. Furthermore, the program according to the present embodiment is a program for causing a computer to execute each of these processes. In other words, the program according to the present embodiment is a program for causing a computer to realize each function executable by server 100. Note that, here, an example has been shown in which a determination process (step S522) is performed to determine whether location information is associated with the selected posted image, and if location information is associated with the selected posted image, a control process (step S523, etc.) is performed to set a destination and a travel route to that destination based on the location information. However, this is not limiting. For example, the above-described determination process (step S522) and part of the control process (step S523) may be omitted, and the control process (steps S524 to S546) may be performed to obtain location information searched on the network using the selected posted image, and set a destination and a travel route based on the location information, regardless of whether location information is associated with the selected posted image. This can reduce the amount of calculation required for the determination process.

[0162] This configuration makes it possible to appropriately set a route to a destination using the selected posted image, which eliminates the need for complex actions to search for the destination of the selected posted image and allows user U1 to be guided to the destination while maintaining the motivation to go to the destination of the selected posted image.

[0163] In the information processing method according to this embodiment, in the control process (steps S523 to S546), if location information is not associated with the selected posted image, the destination and travel route are set using at least one of the location information found by an image search using the selected posted image, the text information contained in the selected posted image, the text information posted in association with the selected posted image, and other posted images other than the selected posted image.

[0164] According to this configuration, even if location information is not associated with the selected posted image, it is possible to obtain location information related to the selected posted image, thereby making it possible to set an appropriate destination and travel route related to the selected posted image.

[0165] In the information processing method according to this embodiment, in the control process (steps S530 to S531, S538 to S546), location information related to the selected posted image is searched for by image search, and a destination and travel route are set based on the location information.

[0166] According to this configuration, it is possible to obtain location information related to the selected posted image through image search, thereby making it possible to set an appropriate destination and travel route related to the selected posted image.

[0167] In the information processing method according to this embodiment, the control process (steps S523 to S546) searches for similar images that resemble the selected posted image using an image search, and if location information is associated with the similar image, the destination and travel route are set based on that location information, and if location information is not associated with the similar image, location information searched for on the network using the similar image is obtained, and the destination and travel route are set based on that location information.

[0168] According to this configuration, location information can be obtained using similar images found by an image search using the selected posted image, thereby enabling an appropriate destination and travel route to be set in relation to the selected posted image.

[0169] In the information processing method according to this embodiment, the control process (steps S527 to S529, S538 to S546) searches for character information contained in the selected posted image, extracts from the character information specific characters that can be used to estimate the location, and sets the destination and travel route based on the specific characters.

[0170] According to this configuration, location information can be obtained using text information contained in the selected posted image, thereby making it possible to set an appropriate destination and travel route related to the selected posted image.

[0171] In the information processing method according to this embodiment, the control process (steps S524 to S526, S538 to S546) searches for text information posted in association with the selected posted image, extracts from the text information specific characters that can be used to estimate the location, and sets the destination and travel route based on the specific characters.

[0172] According to this configuration, location information can be obtained using text information posted along with the selected posted image, thereby enabling an appropriate destination and travel route to be set in relation to the selected posted image.

[0173] In the information processing method according to this embodiment, the control process (steps S523 to S546) searches for other images posted by the poster of the selected posted image, or other images posted by other posters connected to the poster, and if location information is associated with the other posted images, the destination and travel route are set based on that location information; if location information is not associated with the other posted images, location information searched for on the network using the other posted images is obtained, and the destination and travel route are set based on that location information.

[0174] With this configuration, even if location information cannot be obtained using the selected posted image, location information can be obtained using other images posted by the poster of the selected posted image or other images posted by other posters who are connected to the poster, making it possible to set an appropriate destination and travel route related to the selected posted image.

[0175] In the information processing method according to the present embodiment, posted images are displayed on the display unit 262, 342 of the vehicle C1 or the electronic device 200. A posted image selected by a user operation from among the multiple posted images displayed on the display unit 262, 342 is set as a selected posted image.

[0176] According to this configuration, a plurality of posted images can be displayed on the display unit 262, 342 of the vehicle C1 or electronic device 200 that the user U1 can use, and the posted image that suits the user U1's preferences can be selected.

[0177] The information processing method according to this embodiment further includes a reception process (step S544) for receiving a selection operation for selecting a mode of transportation of the user U1. In the control process (steps S538 to S545), if multiple destination candidates are found, a destination that can be reached using the mode of transportation selected by the selection operation may be set from among the multiple destination candidates.

[0178] This configuration makes it possible to set an appropriate travel route depending on the means of transportation selected by the user U1.

[0179] The information processing method according to this embodiment further includes a reception process (step S544) for receiving a selection operation for selecting a transportation mode of user U1. In the control process (steps S538 to S545), if the transportation mode selected in the selection operation is a specific transportation mode (e.g., a shared car or a rental car) that can be used at a specific location, the specific location (e.g., the nearest shared car parking lot or the nearest rental car shop) may be set as a first destination, and after user U1 arrives at the specific location, a transportation route from the first destination to the destination may be set.

[0180] According to this configuration, if the means of transportation selected by user U1 is a specific means of transportation (e.g., a shared car or a rental car) that can be used at a specific location, it is possible to set an appropriate travel route that passes through that specific location.

[0181] The information processing method according to this embodiment can use a database 600 in which predetermined destinations and predetermined businesses are associated with each other. When it is detected that user U1 has arrived at the destination set in the control process (steps S523 to S546), the method further includes a generation process (steps S551 to S554) for generating billing information for making a predetermined bill to the business associated with the destination.

[0182] With this configuration, it is also possible to provide services such as increasing the ranking of images related to destinations registered in database 600 in image searches or making the images easier to view (correcting unclear or blurry images). This makes it easier for users of the destination search app to identify images related to destinations registered in the destination search app. This is expected to increase the number of users who set destinations registered in the destination search app. In addition, the business operator managing server 100 can charge a referral fee (e.g., a referral commission) to other businesses that are pre-associated with the destination.

[0183] The information processing method according to this embodiment further includes a reception process (reception process for receiving a specification operation received by the reception unit 261) for receiving a specification operation for specifying a final destination of the user U1, and in the control process (steps S538 to S545), when a final destination is specified and multiple destination candidates are searched for as the destination, a destination candidate included in an area within a predetermined range based on the travel route to the final destination is set as the destination.

[0184] According to this configuration, for example, when Osaka is specified as the final destination and multiple destination candidates (for example, ramen shops in Nagoya and Sapporo) are searched, the ramen shop in Nagoya, which is in the surrounding area of ​​the travel route to the final destination, is set as a stop destination (or displayed as a top candidate). In this way, when the final destination is set in advance, it is possible to prioritize locations close to the final destination in the search results.

[0185] In the information processing method according to this embodiment, in the control process (steps S538 to S545), if multiple destination candidates are searched for, a destination is set from among the multiple destination candidates based on the distance from the current location of user U1.

[0186] With this configuration, for example, it is possible to set a destination that is close to the current location of user U1, and therefore it is possible to set a destination that is easy for user U1 to live in. In other words, it is possible to quickly find the destination of a posted image found on an image-based SNS or the like, and it is possible to go to the destination of the posted image while maintaining the motivation to go to the destination of the posted image.

[0187] In the information processing method according to this embodiment, the control process (steps S539 and S540) determines whether the destination is a location where privacy protection is required, and if it is determined that the destination is in such a location, the destination is not set.

[0188] For example, if a private photo is uploaded as a posted image, it is assumed that the location of the posted image is a location where privacy protection is required (for example, the location of a private home, etc.). Therefore, it is possible to prevent the location from being identified using the posted image and to exclude locations where privacy protection is required from the destination.

[0189] The server 100 is an information processing device that sets a destination for a user U1 based on a selected posted image selected by a user operation from among images posted to a social networking service (SNS) (an example of a communication service) established on a predetermined network (e.g., network 20). The server 100 includes a determination unit 121 that determines whether location information is associated with the selected posted image, a search unit 122 that sets a destination and a travel route to the destination based on the location information if location information is associated with the selected posted image, and a setting unit 123 (an example of a control unit) that acquires location information searched on the network using the selected posted image and sets a destination and a travel route based on the location information if location information is not associated with the selected posted image. The program according to this embodiment is a program that causes a computer to execute each of these processes. In other words, the program according to this embodiment is a program that causes a computer to realize each function executable by the server 100.

[0190] This configuration makes it possible to appropriately set a route to a destination using the selected posted image, which eliminates the need for complex actions to search for the destination of the selected posted image and allows user U1 to be guided to the destination while maintaining the motivation to go to the destination of the selected posted image.

[0191] Note that each processing procedure shown in this embodiment is an example for realizing this embodiment, and the order of some of the processing procedures may be changed within the scope that makes it possible to realize this embodiment, and some of the processing procedures may be omitted or other processing procedures may be added.

[0192] Each process in this embodiment is executed based on a program that causes a computer to execute various processing procedures. This embodiment can also be understood as an embodiment of a program that realizes the function of executing each process and a recording medium that stores the program. For example, an update process for adding a new function to an information processing device can store the program in the storage device of the information processing device. This makes it possible to cause the updated information processing device to execute each process described in this embodiment.

[0193] Although the embodiments of the present invention have been described above, the above embodiments merely show application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

Claims

1. An information processing method for setting a user's destination based on a selected posted image selected by user operation from among posted images posted to a communication service established on a specified network, the information processing method including: a determination process for determining whether location information is associated with the selected posted image; and, if location information is associated with the selected posted image, a control process for setting the destination and a travel route to the destination based on the location information; and, if location information is not associated with the selected posted image, a control process for obtaining location information searched for on the network using the selected posted image and setting the destination and the travel route based on the location information.

2. An information processing method as described in claim 1, wherein, in the control process, if location information is not associated with the selected posted image, the destination and the travel route are set using at least one of location information retrieved by an image search using the selected posted image, character information contained in the selected posted image, text information posted in association with the selected posted image, and other posted images other than the selected posted image.

3. An information processing method according to claim 2, wherein the control process searches for location information related to the selected posted image by the image search, and sets the destination and the travel route based on the location information.

4. An information processing method as described in claim 2, wherein the control process searches for similar images that are similar to the selected posted image using the image search, and when location information is associated with the similar image, sets the destination based on the location information, and when location information is not associated with the similar image, obtains location information searched for on the network using the similar image, and sets the destination and the travel route based on the location information.

5. An information processing method according to claim 2, wherein the control process searches the character information, extracts predetermined characters from the character information that can be used to estimate a position, and sets the destination and the travel route based on the predetermined characters.

6. An information processing method according to claim 2, wherein the control process searches the text information, extracts predetermined characters from the text information that can be used to estimate a position, and sets the destination and the travel route based on the predetermined characters.

7. An information processing method as described in claim 2, wherein the control process searches for other posted images by the poster of the selected posted image or other posted images by other posters connected to the poster, and if location information is associated with the other posted images, sets the destination and the travel route based on the location information, and if location information is not associated with the other posted images, obtains location information searched for on the network using the other posted images and sets the destination and the travel route based on the location information.

8. An information processing method according to any one of claims 1 to 7, wherein the posted image is displayed on a display unit of a vehicle or electronic device, and a posted image selected by the user operation from among the plurality of posted images displayed on the display unit is set as the selected posted image.

9. An information processing method according to any one of claims 1 to 7, further comprising a reception process for receiving a selection operation for selecting the user's means of transportation, wherein in the control process, when a plurality of destination candidates are searched for as the destination, a destination that can be reached using the means of transportation selected in the selection operation is set from among the plurality of destination candidates.

10. An information processing method according to any one of claims 1 to 7, further comprising a reception process for receiving a selection operation for selecting the user's means of transportation, wherein in the control process, if the means of transportation selected in the selection operation is a specific means of transportation that can be boarded at a specific location, the specific location is set as a first destination, and after the user arrives at the specific location, a travel route from the first destination to the specific destination is set.

11. An information processing method according to any one of claims 1 to 7, wherein a predetermined destination and a predetermined business operator are associated with each destination, and further comprising a generation process for generating billing information for making a predetermined bill to the business operator associated with the destination when it is detected that the user has arrived at the destination set by the control process.

12. An information processing method according to any one of claims 1 to 7, further comprising a reception process for receiving a designation operation for designating a final destination of the user, wherein in the control process, when the final destination is designated and multiple destination candidates are searched for as the destination, a destination candidate included in an area within a predetermined range based on a travel route to the final destination is set as the destination.

13. An information processing method according to any one of claims 1 to 7, wherein, in the control process, when a plurality of destination candidates are searched for as the destination, the destination and the travel route are set from among the plurality of destination candidates based on the distance from the user's current location.

14. An information processing method according to any one of claims 1 to 7, wherein the control process determines whether the destination is a location where privacy protection is required, and if it is determined that the destination is a location where privacy protection is required, the destination is not set.

15. An information processing device that sets a user's destination based on a selected posted image selected by user operation from among posted images posted to a communication service established on a specified network, comprising: a determination unit that determines whether location information is associated with the selected posted image; and a control unit that, if location information is associated with the selected posted image, sets the destination and a travel route to the destination based on the location information; and, if location information is not associated with the selected posted image, obtains location information searched from the network using the selected posted image and sets the destination and the travel route based on the location information.

Images