Sightseeing support device and method

Abstract

To provide a sightseeing support device having good usability for a user.SOLUTION: A sightseeing support device 2 includes: a sightseeing course creation support unit 22 which supports creation of a sightseeing course for a user who uses a user terminal 1; a sightseeing record creation support unit 23 which supports creation of a sightseeing record on the basis of information acquired from the user terminal; and a database unit 24 which is used by the sightseeing course creation support unit and the sightseeing record creation support unit. The sightseeing course creation support unit investigates a user attribute, calculates a sightseeing course on the basis of the investigated user attribute, and accumulated sightseeing record, and customizes the calculated sightseeing course according to an instruction from the user terminal. The sightseeing record creation support unit accumulates images acquired from the user terminal in the database unit, and when having received a browsing request for the accumulated images, changes at least a part of the images, provides the changed images to a browsing request source according to an access privilege of the browsing request source.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a tourism support device and method. [Background technology]

[0002] Techniques for suggesting tourist spots to tourists are known (Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-101909 A [Patent Document 2] Japanese Patent Application Publication No. 10-247213 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies only suggest famous tourist spots to users, but cannot assist in creating a travel record or in creating an original travel route by referring to the travel records of other travelers, resulting in low usability for users.

[0005] The present invention has been made in consideration of the above problems, and an object of the present invention is to provide a tourism support device and method that are easy for users to use. [Means for solving the problem]

[0006] In order to solve the above problems, a tourism support device according to one aspect of the present invention is a tourism support device capable of communicating with a user terminal, and comprises a tourism course creation support unit that supports the creation of a tourism course for a user using the user terminal, a tourism record creation support unit that supports the creation of a tourism record based on information acquired from the user terminal, and a database unit used by the tourism course creation support unit and the tourism record creation support unit, wherein the tourism course creation support unit investigates user attributes, calculates a tourism course based on the investigated user attributes and the accumulated tourism records, and customizes the calculated tourism course in accordance with instructions from the user terminal, and the tourism record creation support unit accumulates images acquired from the user terminal in the database unit, and upon receiving a request to view the accumulated images, changes at least a portion of the image in accordance with the access authority of the source of the viewing request and provides it to the source of the viewing request. Effect of the Invention

[0007] According to the present invention, it is possible to provide support for creating sightseeing courses and for creating sightseeing records, thereby improving usability for users. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an overall configuration diagram of a tourism support system including a tourism support device. [Diagram 2] Hardware configuration diagram of the tourism support system. [Diagram 3] FIG. 2 is an explanatory diagram showing the relationship between a map and an image mapped on the map. [Figure 4] A diagram showing how to create and customize a sightseeing itinerary based on recommendations on the map. [Diagram 5] FIG. 13 is an explanatory diagram showing how posted images are classified on a map. [Figure 6] FIG. 11 is a diagram illustrating how an image changes depending on the time elapsed since its creation date and time. [Figure 7] An example of attribute information associated with a captured image. [Figure 8] 13 is an example of information stored in the database section. [Figure 9] FIG. 4 is an explanatory diagram showing the relationship between the orientation of the user terminal at the time of shooting and the location on the map. [Figure 10] 11 is a flowchart showing a process for supporting the creation of a sightseeing course. [Figure 11] 13 is a flowchart showing an initial setting process. [Figure 12] 11 is a flowchart showing an assessment process. [Figure 13] 11 is a flowchart showing a process for customizing a sightseeing course. [Figure 14] FIG. 13 is an explanatory diagram of a learning model for each user. [Figure 15] 11 is a flowchart showing a process for supporting the creation of a sightseeing record. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The tourism support device according to this embodiment provides a tourism course creation support unit that supports the creation of a tourism course and a tourism record creation support unit that supports the creation of a user's tourism record in a single tourism support device. In other words, the user can use the tourism support device to receive a tourism course creation support service and a tourism record creation support service for when the user actually visits the tourism course, which is convenient.

[0010] An example of using the tourism support device of this embodiment will be described below. The following description does not limit the scope of the present invention. In this embodiment, for example, a platform (tourism support device) is provided that enables a user to upload images (photos or videos) of tourist spots that he or she has visited in the past by associating them with a map. Other users can select and register tourist spots that they would like to visit while looking at the posted tourist photos.

[0011] This allows the device to generate candidate sightseeing courses for touring the selected tourist sites, starting from the hotel where the user is staying. When generating candidate sightseeing courses, the device can take into account, for example, the time required for sightseeing and travel time. The device can dynamically readjust the sightseeing course when there is a time difference between the sightseeing course and the actual sightseeing, or when the user moves off the planned route.

[0012] A user (e.g., a traveler) can take various photos (or videos) at tourist spots, associate them with a map, and register them in a tourist support device. These photos may be associated with captured image attribute information, such as EXIF ​​(Exchangeable image file format) information. Captured image attribute information includes the date and time of capture, the location of capture, the capture conditions, and so on. Therefore, by arranging the posted images on a map in chronological order, it is possible to estimate how long a user stayed at a particular location.

[0013] By calculating the difference between the time when a photo of a first tourist spot was taken and the time when a photo of a second tourist spot closest to the first tourist spot was taken, and then subtracting the time required to travel from the first tourist spot to the second tourist spot, it is possible to roughly determine the time that the user stayed at the first tourist spot.

[0014] The captured image attribute information is not limited to EXIF ​​information. For example, information including the IPTC Information Exchange Model (IIM) of the International Press Telecommunications Council (IPTC) and the date and time of capture (UTC = Coordinated Universal Time), the location of capture (latitude, longitude, altitude), etc. may be used.

[0015] A user who is planning a trip can use the tourism support device of this embodiment to plan his / her own sightseeing course based on images posted by other users. The user may appropriately modify the candidate sightseeing courses proposed by the tourism support device.

[0016] Hereinafter, each location that the user visits will be referred to as a “tourist spot.” When a user visits a city and moves between various facilities such as theaters, museums, restaurants, and shopping malls, these facilities become tourist spots. EXAMPLES

[0017] A first embodiment will be described with reference to Figs. 1 to 15. Fig. 1 is an overall configuration diagram of a tourism support system 3. The tourism support system 3 includes, for example, at least one user terminal 1 and one server 2 connected to the user terminal 1 via a communication network CN. The server 2 corresponds to a "tourism support device." The server 2 can also be called a tourism support device 2 or a tourism support server 2.

[0018] The user terminal 1 used by the users U1 and U2 is, for example, a mobile phone (including a so-called smartphone) or a personal computer. Personal computers come in various forms, such as laptop type, notebook type, tablet type, desktop type, bracelet type, and eyeglass type.

[0019] User U1, for example, creates a sightseeing course using a sightseeing course creation service provided by server 2 and sets off on a trip. User U2 uses a sightseeing record service provided by server 2 to view images such as photos and videos taken by other users (e.g., user U1). Inspired by the images posted by user U1, user U2 can use the sightseeing course creation service of server 2 to create his or her own original sightseeing course.

[0020] User U1 can store the images he / she posted in server 2 and can freely view them at any time. Other users approved by user U1 can view images posted by user U1 in the same way as user U1. Other users who have not been approved by user U1 are restricted from viewing images posted by user U1. For example, faces or entire persons may be blurred or removed from images posted by user U1 and provided to unapproved users.

[0021] The external site 4 is a server configured separately from the server 2. The external site 4 includes, for example, a map database server that provides a map database, a storage system that stores image data, an SNS (Social Networking Service) server that provides an SNS, a news server that distributes articles or commentaries, etc. The server 2 can use the external site 4 as necessary.

[0022] The server 2 includes, for example, a data transmission / reception unit 21, a sightseeing course creation support unit 22, a sightseeing record creation support unit 23, and a database unit 24.

[0023] The data transmitter / receiver 21 transmits and receives data between the user terminal 1 and the external site 4 via the communication network CN. The communication network CN is, for example, the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0024] The sightseeing course creation support unit 22 supports the creation of a sightseeing course for the user based on information input by the user. The sightseeing course creation support unit 22 includes, for example, a user survey unit 221, a course calculation unit 222, and a customization unit 223. As described later, the user survey unit 221 surveys various attributes of the user (such as name, place of residence, and preferences) and records them in the database unit 24. As described later, the course calculation unit 222 calculates a sightseeing course predicted to be suitable for the user using a machine learning model. The customization unit 223 customizes the sightseeing course proposed by the server 2 in response to a user request.

[0025] The tourist record creation support unit 23 supports the creation of a tourist record based on images uploaded by the user terminal 1 to the server 2. A tourist record is created, for example, by allocating images on a map based on the date and time of image capture and the location of image capture. When uploading images to the server 2, the user can associate the images with, for example, text data, web pages, images posted by other users, and audio data, thereby creating a more substantial tourist record. For example, when a user dine at a restaurant in a tourist spot, the user can take pictures of the interior of the restaurant and the food and upload them to the server 2. At that time, the user can associate the posted images (images uploaded to the server 2) with a web page introducing the restaurant, background music (BGM) data played in the restaurant, and the like.

[0026] The tourist record creation support unit 23 includes, for example, an access management unit 231 and an image processing unit 232. When there is a request to view an image uploaded to the server 2 (i.e., a posted image), the access management unit 231 determines whether or not the image can be viewed. When there is a request to view an image from a user with restricted viewing (a user without viewing permission), the image processing unit 232 processes a part of the image requested to be viewed and provides it to the user terminal 1 that made the request. As will be described later, the extent or degree of image processing can be varied depending on the level of viewing restriction. The database unit 24 holds some or all of the data necessary for the server 2 to provide the function as a tourist support device. Some of the necessary data can also be acquired from the external site 4. The database unit 24 stores, for example, a map database T1, a tourist destination database T2, an image database T3, and a user information database T4. The map database T1 is information including maps of tourist destinations around the world. The tourist destination database T2 is information including names, types, locations, tourist indicators, etc. of tourist destinations. The image database T3 is information of photos or videos taken at tourist destinations. The user information database T4 is information including various attributes and preferences of the user who took the photo, authentication information for using tourist support services, etc. The information T2 to T4 stored in the database unit 24 will be described later.

[0027] 2 is a diagram showing the hardware configuration of the tourist support system 3. The user terminal 1 will be explained first, and then the server 2 will be explained.

[0028] As described above, the user terminal 1 is a portable computer terminal (so-called mobile terminal) such as a mobile phone. The user terminal 1 includes, for example, a processing unit 101, a main memory device 102, an auxiliary memory device 103, a communication device 104, an input device 105, and an output device 106, and these devices 101 to 106 are connected by a communication line 107. The user terminal 1 does not need to be configured from one housing, and may be configured from multiple housings. For example, the input device 105 and the output device 106 may be configured as housings separate from the housings that house the other devices 101 to 104. If the processing capacity of the processing unit 101 is insufficient, the processing resources of the server 2 or the external site 4 may be used. If the storage area of ​​the auxiliary storage device 103 is insufficient, the storage resources of the server 2 or the external site 4 may be used.

[0029] The arithmetic processing device 101 reads and executes a specific computer program stored in the main memory device 102, thereby realizing the functions of the user terminal 1. The functions of the user terminal 1 include an information browsing function using a web browser, a posting function for images, text, or audio data, a sending and receiving function for emails or messages, and the like.

[0030] The auxiliary storage device 103 stores the above-mentioned predetermined computer programs and data. The main storage device 102 reads out the predetermined computer programs and the like from the auxiliary storage device 103 and stores them.

[0031] The communication device 104 communicates with the server 2 via the communication network CN. The user terminal 1 can also communicate with an external site 4 as necessary.

[0032] The input device 105 is a device for inputting information into the user terminal 1. The input device 105 includes, for example, a keyboard 1051, a microphone (abbreviated as mic) 1052, a touch panel 1053, a camera 1054, a position information sensor 1055, an air pressure sensor 1056, and the like.

[0033] The keyboard 1051 may be either a physical keyboard or a virtual keyboard. The camera 1054 may be either a front camera or a rear camera. The position information sensor 1055 is a sensor that detects the position and direction of the user terminal 1, and includes, for example, a GPS (Global Positioning System), a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like. The air pressure sensor 1056 measures air pressure.

[0034] The output device 106 is a device that provides information to the user from the user terminal 1. The output device 106 includes, for example, a monitor display 1061 and a speaker 1062.

[0035] The server 2 will be described. The server 2 includes, for example, a processor 201, a main memory 202, an auxiliary memory 203, a communication device 204, and a user interface device 205, and these devices 201 to 205 are connected via a communication line 206. In the figure, the user interface is abbreviated as "UI." The server 2 may be composed of multiple computers.

[0036] The auxiliary storage device 203 stores a predetermined computer program for causing the server 2 to function as a tourist support device. Furthermore, the auxiliary storage device 203 also stores various types of information T1 to T4 shown in the database unit 24 of Fig. 1. The predetermined computer program is transferred from the auxiliary storage device 203 to the main storage device 202 as appropriate. The arithmetic processing device 201 reads out and executes the predetermined computer program from the main storage device 202, causing the server 2 to function as a tourist support device.

[0037] The arithmetic processing device 201 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), or the like.

[0038] The server 2 may be configured as a so-called cloud system in which a plurality of information processing devices are configured to be able to communicate with each other via a communication network.

[0039] FIG. 3 is an explanatory diagram showing the relationship between a map M and images P, Pu, and Pua mapped on the map M.

[0040] A tourist attraction A1 is displayed on a map M. Assume that a user is attracted to the tourist attraction A1 and plans a trip. The user specifies a place of accommodation on the map M (not shown). The server 2 displays on the map M a recommended sightseeing course to visit surrounding tourist attractions that can be visited during a standard sightseeing time slot, based on the place of accommodation and the tourist attraction database T2.

[0041] FIG. 4 is an explanatory diagram showing how a user customizes a recommended sightseeing course displayed on a map M. As shown in FIG.

[0042] As shown in map M1 at the top of Fig. 4 and map M2 at the center of Fig. 4, the user can view thumbnail photos of tourist spots on the map and click on any that interest him or her to enlarge it. As described above, a recommended sightseeing course (hereinafter sometimes abbreviated as a recommended course) may be displayed on the map based on the user's accommodation location.

[0043] As shown in map M3 at the bottom of Fig. 4, the user can add favorite sightseeing spots to the sightseeing course by looking at thumbnail photos, and customize the recommended course. The sightseeing course creation support unit 22 of the server 2 creates new sightseeing course candidates for efficiently moving between the added sightseeing spots. The sightseeing course creation support unit 22 reschedules the recommended departure and arrival times for each sightseeing spot, taking into account the time required for sightseeing at each sightseeing spot on the course and the travel time between the sightseeing spots.

[0044] Return to Figure 3. The user can delete places that are not of interest while checking the images P of tourist spots recommended by the server 2. If the image Pu posted on the map M contains personal information such as the poster's face or person, the poster can view the image as is, but if the poster chooses to keep the personal information private, the face or person in the image P can be detected using machine learning model technology, and the face or person part can be mosaicked to generate an image Pus in which the personal information is protected. A user who travels along the recommended tourist course as is or customized can take photos and other images at each destination on the trip and post them on the map M. The server 2 maps the posted images on the map M in chronological order and provides them to other users. As mentioned above, the user can also give permission to others, such as family or friends, to view the posted images. Those who have permission to view the original image Pu can view the original image Pu in the same way as the poster. Those who do not have permission to view the image Pu can view only the changed image Pu, with the face in the image Pu blurred. As will be described later, not only the face but also the entire person can be deleted from the image Pu.

[0045] FIG. 5 shows how images posted on a map are classified. The server 2 can classify images posted on a map into categories such as "food," "landscape," and "art," for example, using a machine learning model. The examples are not limited to these, and images can also be classified from the perspective of, for example, "sports," "thrilling," and "romantic." Images may be classified based on keywords or phrases specified by the user.

[0046] Figure 6 shows how an image changes depending on the amount of time that has passed since it was created. If the image is digital data, the image will not deteriorate. However, if it is a film photograph, it will deteriorate over time. The degree of deterioration over time (aging) can inform the viewer of the image of the amount of time that has passed since it was posted.

[0047] Image P1 at the top of Fig. 6 is the original image when the image was created. Image P2 at the center of Fig. 6 is an image to be provided to the viewer when a predetermined time (e.g., several years) has passed since the image was created. For example, if the original image P1 is a full-color photograph, the server 2 processes the image to a sepia-toned photograph or a black-and-white photograph and displays it.

[0048] Image P3 at the bottom of Fig. 6 is an image taken some time after image P2. By displaying it faintly, the viewer can intuitively tell that image P3 is a long time old. Note that if the poster himself views the image, he or she can view it in its original colors without processing to show aging.

[0049] FIG. 7 shows an example of attribute information (photographed image attribute information) associated with a photographed image. For example, image data such as JPEG data or TIFF data is provided with EXIF ​​information as an example of photographed image attribute information. The EXIF ​​information includes, as meta information, for example, the photographed date and time, location information, photographed direction, GPS time (the most recent time when a GPS signal was captured, which usually matches the photographed date and time but may differ depending on the location), manufacturer name, model name, resolution, shutter speed, aperture, ISO (International Organization for Standardization) sensitivity, photometry mode, whether or not a flash was used, exposure correction step value, focal length, color space, thumbnail, file modification date and time, etc. The location information includes the latitude and longitude of the photographed location, altitude, etc. The server 2 can map the photographed location of the posted image on a map based on the GPS information.

[0050] 8 shows an example of each of the databases T2 to T4 stored in the database unit 24. Details of the map database T1 will be omitted.

[0051] The tourist destination database T2 manages tourist destinations and includes, for example, a tourist destination ID C21, a type C22, a name C23, a location C24, a required time C25, a tourist index C26, and a remark C27.

[0052] The tourist destination ID C21 is identification information that uniquely identifies a tourist destination within the tourism support system 3. The type C22 indicates the type of tourist destination. Examples of the type C22 include famous places (natural landscapes and buildings), art (artworks and other works: the location is the location of the art gallery / museum where the work is stored), gourmet (meals and desserts: the location is the location of the restaurant that serves them), shopping (items for sale such as famous souvenirs: the location is the store that sells them), etc.

[0053] The name C23 is the name of the tourist spot. The location C24 is information indicating the location of the tourist spot, such as latitude and longitude. The required time C25 is the average sightseeing time required to see the tourist spot. A default value preset for each type C22 may be used.

[0054] The tourism index C26 is an index indicating the attractiveness of a tourist destination. For the tourism index C26, a default value set in advance for each type C22 may be used.

[0055] The note C27 indicates other information about the tourist destination, such as the attractions, features, and tourist points of the tourist destination identified by the tourist destination ID C21.

[0056] The image database T3 will now be described. The image database T3 manages images taken at tourist spots and posted to the server 2, which serves as a tourist support device. The image database T3 includes, for example, a tourist spot ID C31, a poster ID C32, an image file C33, a popularity rating C34, a visual appearance rating C35, and photographed image attribute information C36.

[0057] The tourist attraction ID C31 is identification information that uniquely identifies a tourist attraction, and is set to the same value as the tourist attraction ID C21 in the tourist attraction database T2. The tourist attraction IDs C31 and C21 link the tourist attraction database T2 and the image database T3. The poster ID C32 is information that uniquely identifies the user who posted the image. The poster ID can also be called a user ID. The image file C33 indicates the storage location and file format of the image.

[0058] The popularity C34 indicates the popularity of the image. The popularity C34 can be calculated based on the number of references (number of views, viewing time, etc.). The appearance C35 is information indicating the appearance of the image. The appearance C35 is information indicating, for example, the splendor of the scenery, the degree of prominence, etc. The appearance C35 can be calculated using a machine learning model.

[0059] The captured image attribute information C36 is attribute information associated with an image. An example of this is described in Fig. 7. However, as described later, even if the location where the image was taken is close to the location of a tourist attraction, if the direction of the image is different from the tourist attraction's famous historical site, the captured image is determined to be unrelated to the tourist attraction and is excluded from the image database T3.

[0060] An example of a method for indexing appearance using artificial intelligence technology will be described. For example, images on SNS and the number of "Likes" given to the images are input as learning data into a machine learning model that learns the trends of popular images. Using this machine learning model, it is possible to calculate the appearance. However, the appearance from the viewpoint of the user of this system 3 may differ from the appearance of the image posted on the SNS. Therefore, when a certain amount of data (e.g., 1,000 items) of popularity (number of clicks) data on images posted by all users is accumulated by operating the tourism support system 3, the machine learning model that calculates the appearance is retrained in addition to the learning data. This makes it possible to correct the machine learning model that takes into account the appearance from the viewpoint of the user (traveler).

[0061] The degree of appearance often depends on the individuality of the user. Some users are more sympathetic to images of famous places, while others are more sympathetic to images of art or food. Images that are very popular with one user may be unpopular with another, or vice versa. In order to mass-customize the content recommended based on such user personalities, one method is to conduct an assessment of each user in advance and directly ask about the user's personality (asking questions such as, "Are you a scenic place lover, an art lover, a gourmet lover, or a shopping lover?"). Another method is to show the user several patterns of images that contain a mixture of various types of photos, and have them select the places they want to go, the things they want to see, and the things they want to eat from among them, thereby indirectly investigating whether the user's personality is a scenic place lover, an art lover, a gourmet lover, or a shopping lover, and to prepare a learning model that suits each personality in advance. Another method is to retrain and recreate the learning model for each user based on the assessment answers of each user. If mass customization is not necessary, an average machine learning model is sufficient.

[0062] The image files C33 linked from the tourist destination database T2 are sorted in order of tourist indices calculated from popularity (number of clicks) and appearance, for example. The image files C33 are linked to actual image data groups (thumbnail images and full-resolution image data files).

[0063] The user information database T4 is a database for managing users who use the tourism support system 3. The user information database T4 includes, for example, a poster ID C41, authentication information C42, basic attributes C43, preferences C44, browsing permission ID C45, and browsing permission authentication information C46.

[0064] The poster ID C41 is set to the same value as the poster ID C32 described in the image database T3. The authentication information C42 is information used for personal authentication when the user uses the tourism support system 3. The basic attributes C43 indicate basic attributes among the user's attribute information. Basic attributes include, for example, name, area of ​​residence, and age. The preferences C44 indicate the user's preferences regarding their interests. Examples of preferences C44 include liking famous places, liking gourmet food, and liking art.

[0065] The viewing permission ID C45 is information that identifies other users who can view the image posted by the poster. The information that identifies other users is, for example, the poster ID (user ID). Other users who have the user ID set in the viewing permission ID C45 can view the image, just like the poster of the image.

[0066] The viewing permission authentication information C46 is authentication information for viewing the posted image. Even if a user is not set in the viewing permission ID C45, if the user knows the viewing permission authentication information C46, ​​the user can view the image in the same way as the poster. Multiple ranks of authentication information may be set in the viewing permission authentication information C46. When low-ranked viewing permission authentication information is used, the image quality (resolution, gradation, etc.) is reduced or the entire person, etc. is removed, and the image is provided to a low-ranked viewer. When high-ranked viewing permission authentication information is used, the image can be viewed in the same way as the poster.

[0067] It is not necessary to use both the viewing permission ID and the viewing permission authentication information C46, ​​and viewing of images can be controlled using only one of them.

[0068] FIG. 9 shows the relationship between the orientation of the user terminal 1 at the time of shooting and the location on the map M. Even if an image is shot at the same location, the subject may be different if the camera is oriented differently. For example, if a photo is taken at a certain location facing east, the "Archbishop's Palace" may be captured, and if a photo is taken at the same location facing south, the "Cathedral" may be captured. In FIG. 9, like a normal map, the top indicates north, the bottom indicates south, the left side indicates west, and the right side indicates east. In FIG. 9, for example, a triangle indicates the "Alcazar," an oval indicates the "Cathedral," and a star indicates the "Archbishop's Palace." In this way, the shooting location and the actual subject may differ depending on the shooting direction.

[0069] Therefore, the coordinates of the shooting location of the posted image can be moved slightly toward the shooting direction to bring it closer to the subject's position. For images of "art", "food", or "shopping", it is likely that the image was taken indoors, so there is no need to correct the coordinates of the shooting location. In contrast, for images taken outdoors, such as of buildings, the image is taken in an urban area, so the coordinates are corrected in the shooting direction by, for example, 15-20 m. For images of natural scenery such as landscapes, a correction of, for example, 100-200 m is required. For example, for images of large mountains such as Mt. Fuji or the Matterhorn, the actual distance to the peak may require a correction of several tens of kilometers, but if the visitor is planning to visit an observation deck on Mt. Fuji, it is sufficient to guide the visitor around the shooting location, so there is no need to correct by several tens of kilometers.

[0070] 10 is a flow chart showing the whole of the sightseeing course creation support process. The sightseeing course creation support unit 22 first performs an initial setting process (S10). The initial setting process will be described in detail later.

[0071] The sightseeing course creation support unit 22 determines (S11) whether the user is a first-time user of the sightseeing support system 3. If the sightseeing course creation support unit 22 determines that the user is a first-time user (S11: YES), it executes an assessment process described below (S12).

[0072] The sightseeing course creation support unit 22 acquires the user's accommodation location (lodging destination) from the user by displaying a question on the user terminal 1 (S13). The user may search for a hotel name and input it to the sightseeing course creation support unit 22, or may select it from a map.

[0073] The sightseeing course creation support unit 22 displays the user's accommodation location on the map displayed on the user terminal 1 (S14), and asks the user to confirm whether the accommodation location is correct (S15).

[0074] If the accommodation location is incorrect (S15: NO), the process returns to step S13. If the accommodation location is correct (S15: YES), the sightseeing course creation support unit 22 calculates a recommended sightseeing course that matches the user's preferences and the accommodation location, and presents it to the user (S16). That is, the sightseeing course creation support unit 22 extracts at least one tourist attraction close to the accommodation location based on the location information of the accommodation location and the tourist attraction database T2, calculates a recommended sightseeing course in accordance with the user's assessment result, and presents it to the user (S16).

[0075] The tourist course creation support unit 22 may suggest a way to tour the recommended tourist courses for that year that have been created in advance, starting from the place of accommodation, or may arrange tourist destinations in order of the number of images posted, the total number of popularity (number of clicks), the total appearance rating, the total tourism index, or the like, and present the shortest route from the place of accommodation to tour the top tourist destinations.

[0076] The sightseeing course creation support unit 22 determines whether the user wishes to customize the sightseeing course (S17), and if it determines that customization is desired (S17: YES), executes a sightseeing course customization process described below (S18).

[0077] If the sightseeing course creation support unit 22 determines that the user does not wish to customize the sightseeing course (S17: NO), it determines whether to end this process (S19). If there is a next user or next accommodation to be processed (S19: YES), the sightseeing course creation support unit 22 returns to step S11. If there is no next user to be processed or next accommodation (S19: NO), it ends this process.

[0078] 11 shows a flowchart of the initial setting process. The tour course creation support unit 22 sets an initial value n to 1 (S100), selects the n-th tourist attraction from the tourist attraction database T2 (S101), and acquires images of the vicinity of the selected tourist attraction from the image database T3 (S102). From among the acquired images, the tour course creation support unit 22 discards images that are not taken facing the tourist attraction (S103).

[0079] The sightseeing course creation support unit 22 uses a machine learning model to determine whether the acquired image is an image of a tourist spot (S104), and if it is determined that the image is not an image of a tourist spot (S104: NO), the process returns to step S102.

[0080] When the tourist course creation support unit 22 determines that the image is of a tourist spot (S104: YES), it determines the type of image (famous place, food, art, shopping, etc.) (S105). The tourist course creation support unit 22 uses a machine learning model to calculate the attractiveness of the image of the tourist spot, and scores the image with a score of 100 points, assuming that the image is one that everyone would want to see. The tourist course creation support unit 22 calculates a tourist index based on the attractiveness (S106).

[0081] Here, the tourism index can be calculated using the following formula 1.

[0082] Tourism index = (visibility determined by AI (out of 100) + number of users who clicked on the image / number of users who visited the tourist spot and viewed the image (i.e., the click rate; if everyone clicked, it would be 100%)) / 2 (Formula 1) The tourism index calculated from formula 1 is corrected to a perfect score of 100 points. At the time of the initial setting process, no one has visited this tourist spot yet, so the tourism index = appearance level is set. The tourist course creation support unit 22 rearranges the images of the tourist spots in order of tourism index (S107). The process of FIG. 11 is repeated (S108: NO) while incrementing the variable n by 1 (S109) until the initial setting process is completed for all tourist spots registered in the tourist spot database T2. When all tourist spots have been processed (S108: YES), the tourist course creation support unit 22 ends this process.

[0083] 12 shows a flowchart of the assessment process. When a first-time user accesses the tourism support system 3, a user assessment is first performed to investigate the user's attributes (basic attributes, preferences, etc.). The tourism course creation support unit 22 checks what the user places the most importance on when sightseeing, and sets a machine learning model for calculating tourism indices.

[0084] When the tourist course creation support unit 22 determines that the user places the most importance on "famous places" (S120: YES), it sets a machine learning model for famous places for calculating a tourist index (S121). When the tourist course creation support unit 22 determines that the user places the most importance on "art" (S122: YES), it sets a machine learning model for art appreciation for calculating a tourist index (S123). When the tourist course creation support unit 22 determines that the user places the most importance on "meals" (S124: YES), it sets a machine learning model for gourmet for calculating a tourist index (S125). When the tourist course creation support unit 22 determines that the user places the most importance on "shopping" (S126: YES), it sets a machine learning model for shopping for calculating a tourist index (S127).

[0085] Furthermore, when the sightseeing course creation support unit 22 determines that the user will not be satisfied with a sightseeing course similar to that of others (S128: YES), it creates a machine learning model unique to the user and sets it for calculating the sightseeing index (S129).

[0086] 13 shows a flowchart of the customization process of a sightseeing course. The sightseeing course creation support unit 22 first selects images of the top few (e.g., three) sightseeing indexes for each sightseeing spot around the place where the user stays, and maps and displays thumbnail images at the respective latitude and longitude points (S180).

[0087] If the user finds an image that interests him or her, he or she can click on it to enlarge it and check the details, or add another top three images (S181).

[0088] The click count of the image enlarged by the user is incremented by 1 and stored in the tourist spot database T2 (S182).

[0089] When the user adds a tourist attraction (S183: YES), the sightseeing course creation support unit 22 recreates the sightseeing course to include the added tourist attraction (S184). For example, the sightseeing course is modified to make the sightseeing course the shortest possible according to a solution to the Traveling Salesman Problem (TSP), such as the Nearest Neighbor algorithm.

[0090] The sightseeing course creation support unit 22 recalculates the total travel time that changes due to the addition of a tourist destination. Furthermore, the sightseeing course creation support unit 22 recalculates the total sightseeing time required for all tourist destinations by adding the sightseeing time required for the added tourist destination. If the hotel departure time and return time can be adjusted, the time required for the added new tourist destination can be absorbed by departing earlier or returning later. If there is no room for adjustment of the departure time and return time, the sightseeing time required is adjusted using the following formula 2.

[0091] Total available sightseeing time = Last hotel return time - Earliest hotel departure time = Total travel time to all tourist spots + Total time required for sightseeing at all tourist spots (Equation 2) Since the travel time to tourist spots cannot be changed in principle, it is adjusted according to the time required for sightseeing using Equation 3.

[0092] Time required for sightseeing after the change to point n = (time required for sightseeing before point n (standard)) ÷ total (total time required for sightseeing before all points (m = 1 to n) (standard)) × (total available time for sightseeing - travel time to all tourist spots) (Equation 3) If the required sightseeing time for the tourist spot n after the change falls short of the minimum required sightseeing time for the tourist spot, the sightseeing course creation support unit 22 advises the user to delete one of the tourist spots from the sightseeing course.

[0093] When the user decides which tourist spot to delete (S185: YES), the tourist course creation support unit 22 automatically generates a tour course after the deletion (S186), and the required sightseeing time for each tourist spot n is also increased by the reverse calculation method. This is repeated and customization processing is performed until the tourist course is decided (S187: NO). When the tourist course is decided (S187: YES), the estimated arrival time is calculated by adding the travel time to the first tourist spot from the departure time from the hotel, and the required sightseeing time for the tourist spot is added to calculate the estimated departure time to the next point. Similarly, the estimated arrival time and the estimated departure time for tourist spot n are calculated and mapped on a map to be presented to the user.

[0094] Although users go sightseeing based on a sightseeing course they have customized, it often happens that sightseeing progresses faster than planned, or slower than planned. In such cases, the tourism support system 3 dynamically adjusts the sightseeing time required for each tourist spot to catch up. If the sightseeing time required for each tourist spot needs to fall below the minimum required sightseeing time, the tourism support system 3 prompts the user to delete a tourist spot. In this way, the tourism support system 3 dynamically changes the itinerary for visiting tourist spots even during sightseeing.

[0095] 14 shows an example of a learning process of a machine learning model that determines the appearance level for an individual user. Some users may want to use a machine learning model that determines the appearance level for themselves.

[0096] However, since it is not realistic to collect a large amount of input data and its answers for learning for each individual user, we adopt a transfer learning method based on a machine learning model that uses a large amount of data in advance, as shown in the upper part of Figure 14.

[0097] In this embodiment, in order to investigate the interests of each individual user, several patterns (e.g., 10 patterns) of collections of representative images (e.g., 20 images) of famous places, gourmet food, art, shopping, etc. are prepared, and the method of having the user select multiple images (e.g., 5 images) of places they would like to visit is repeated multiple times. In this way, by adding the interest data of each individual user and re-learning by transfer learning, a machine learning model that calculates the appearance degree unique to each individual user can be created.

[0098] The method for estimating the average sightseeing time will be described. The time required to be stored in the tourist destination database T2 can be obtained from the database T3 of images taken near each tourist destination linked to the tourist destination in the tourist destination database T2. When searching the image database T3 by the poster ID and finding images of nearby tourist destinations (close latitude and longitude), these images are arranged in chronological order. Among these images, first, images of the same date are narrowed down. This is to eliminate the unlikely case where, for example, a traveler who visits the same tourist destination every year submits images to the system 3 every year, which means that he or she has been sightseeing at that location for a whole year. Then, for example, if the images are submitted in chronological order, such as three images of point A, two images of point B, and four images of point C, the time that the poster spent at points A and B is estimated using the following formulas 4 and 5.

[0099] Estimated time spent at point A = time when the first posted photo at point A was taken - time when the first posted photo at point B was taken - travel time from point A to point B (estimated from distance) (Equation 4) Estimated time spent at point B = time when the first photo posted at point B was taken - time when the first photo posted at point C was taken - travel time from point B to point C (estimated from distance) (Equation 5) In this way, the images posted for each tourist spot are arranged in chronological order by poster, the estimated stay time is calculated taking into account travel time, and the estimated stay times of multiple posters are averaged as a sample. This makes it possible to estimate how long it would take a typical tourist to visit a particular tourist spot. However, if the poster stays longer than a certain threshold (e.g., 3 hours), they are not included in the sample as they are considered to be outside the normal tourist pattern.

[0100] FIG. 15 is a flowchart showing the process performed by the sightseeing record creation support unit 23 to support the creation of a sightseeing record.

[0101] The tourist record creation support unit 23 manages images posted to the server 2, thereby supporting the user who posted the image in creating a tourist record. Simply mapping images on a map in chronological order can serve as a tourist record. Users can further enrich and enjoy their travel records by adding comments to the images, adding music data, and linking them to web pages on external site 4. When a user other than the poster requests to view an image, the tourist record creation support unit 23 controls as follows.

[0102] When the tourist record creation support unit 23 receives a viewing request from the user terminal 1 (S20: YES), it checks the access authority of the user who made the viewing request (S21). The tourist record creation support unit 23 judges whether the user who made the viewing request is the poster of the image requested to be viewed or a person who has received permission from the poster (S22). If the user is the poster or a person who has received permission from the poster (S22: YES), the tourist record creation support unit 23 decides to provide the unedited original image (S23).

[0103] If the user who made the viewing request is neither the contributor nor an authorized person (S22: NO), the tourist record creation support unit 23 checks whether the user who made the viewing request has the authority to view part of the image (S24). If the viewing request is from a person who has the authority to view part of the image (S24: YES), the tourist record creation support unit 23 decides to provide the image with only the face of the person blurred (S25).

[0104] If the user who has made the viewing request does not have the authority to view even a portion of the image (S24: NO), the tourist record creation support unit 23 decides to delete the entire person from the image requested to be viewed and provide it (S26).

[0105] The tourist record creation support unit 23 transmits the image to the user terminal that made the viewing request according to the provision method determined in either step S23, s25, or S26 (S27). Note that only an image with the face blurred, or an image with the person entirely removed, may be provided to anyone other than the poster or a person who has received permission from the poster. If step S24 is omitted and "NO" is determined in step S22, the process may proceed to either step S25 or step S26.

[0106] Even if a person has not obtained viewing permission from the poster, a user who has made a great contribution to the tourism support system 3 can be given partial viewing authority. A person who has made a great contribution to the tourism support system 3 is, for example, a person who has posted a large number of images to the tourism support system 3, a person who has submitted useful comments on the authenticity of images posted by other users, a person who has submitted new information about tourist spots, a person who has paid a special fee to the tourism support system 3, etc.

[0107] According to this embodiment configured as above, the same server 2 can provide the user with both a service for supporting the creation of a sightseeing course and a service for supporting the creation of a sightseeing record when the user actually visits the sightseeing course, which is convenient for the user.

[0108] According to this embodiment, by using the tourism support system 3, a user can easily plan his / her own sightseeing itinerary based on images posted by other users, and can easily create a sightseeing record by posting images taken during the trip to the tourism support system 3.

[0109] According to this embodiment, the original image can be provided to the poster or to those who have received permission from the poster, and the image can be provided to others with some changes. Therefore, the user can contribute to the travel planning of other users while protecting the privacy of the user and his / her family or friends.

[0110] In this embodiment, the machine learning model used to calculate tourism indices is not only uniformly prepared according to hobbies and preferences, but also an individual machine learning model is provided. This can improve the satisfaction of users who like original travel, and improve usability for users.

[0111] In this embodiment, the image is discolored (discolored) according to the time that has passed since the image was created, and is mapped on the map. This allows viewers to determine at a glance whether the posted image is new or old, and allows them to make appropriate decisions about selecting tourist spots.

[0112] The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and modifications within the scope of the present invention. The above-described embodiment is not limited to the configuration example shown in the attached drawings. The configuration and processing method of the embodiment can be appropriately changed within the scope of achieving the object of the present invention.

[0113] In addition, the components of the present invention may be selected arbitrarily, and the invention having the selected components is also included in the present invention. Furthermore, the components described in the claims may be combined in combinations other than those explicitly stated in the claims. [Explanation of symbols]

[0114] 1: user terminal, 2: server (tourism support device), 3: tourism support system, 4: external site, 21: data transmission / reception unit, 22: tourism course creation support unit, 23: tourism record creation support unit, 24: database unit

Claims

1. A tourism support device capable of communicating with a user terminal, a tour course creation support unit that supports the creation of a tour course for a user who uses the user terminal; a tourist record creation support unit that supports the creation of a tourist record based on information acquired from the user terminal; a database unit used by the tour course creation support unit and the tour record creation support unit; Equipped with the sightseeing course creation support unit investigates user attributes, calculates a sightseeing course based on the investigated user attributes and accumulated sightseeing records, and customizes the calculated sightseeing course in response to instructions from the user terminal; The tourist record creation support unit accumulates images acquired from the user terminal in the database unit, and upon receiving a request to view the accumulated images, changes at least a part of the images in accordance with the access authority of the source of the image request and provides the images to the source of the image request. Tourism support equipment.

2. When the access authority of the view request source is restricted, the tourist record creation support unit modifies an area including a face of a person included in the image and provides the image to the view request source. The tourist support device according to claim 1.

3. When the access authority of the view request source is restricted, the tourist record creation support unit changes an area including a person included in the image and provides the image to the view request source. The tourist support device according to claim 1.

4. The tourist record creation support unit provides the image as is to the viewing request source when the access authority of the viewing request source is not restricted, modifies an area including a face of a person included in the image and provides the image to the viewing request source when the access authority of the viewing request source permits partial viewing, and modifies an area including a person included in the image and provides the image to the viewing request source when the access authority of the viewing request source is restricted. The tourist support device according to claim 1.

5. The tourist support device according to any one of claims 2 to 4, wherein the tourist record creation support unit changes the image depending on the time that has elapsed since the image was generated, and provides the image to the source of the viewing request.

6. The sightseeing course creation support unit calculates the sightseeing course based on a predetermined sightseeing index calculated from the user's preferences included in the user attributes. The tourist support device according to claim 5.

7. A tourism support method using a computer as a tourism support device, comprising: A user terminal used by a user is communicatively connected to the computer; a sightseeing course creation support step of supporting creation of a sightseeing course for a user who uses the user terminal; a tourist record creation support step of supporting creation of a tourist record based on information acquired from the user terminal; Run the tourist course creation support step includes investigating user attributes, calculating a tourist course based on the investigated user attributes and accumulated tourist records, and customizing the calculated tourist course in response to an instruction from the user terminal; The tourist record creation support step includes storing images acquired from the user terminal in a database unit, and upon receiving a request to view the stored images, providing the images to the view request source after modifying at least a part of the images in accordance with an access authority of the view request source. How to support tourism.

Images