Programs, information processing methods, servers

Abstract

To improve the convenience of how created itineraries are presented. [Solution] A program executed by a server communicating with a terminal includes generating a first journey based on first information input into the terminal by the user, generating a first video related to the first journey based on the first journey, and transmitting the first video to the terminal by the control unit.

Description

Technical Field

[0001] This disclosure relates to programs, information processing methods, servers, etc.

Background Art

[0002] When a user plans to travel and makes a travel itinerary, the itinerary may be made based on the user's hobbies. For example, Patent Document 1 discloses a travel plan creation system for proposing a travel that matches each user's preference for each individual user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] According to a first aspect of the present invention, a program executed by a server communicating with a terminal includes generating, by a control unit of the server, a first travel itinerary based on first information input by a user to the terminal, generating, by the control unit, a first video related to the first travel itinerary based on the first travel itinerary, and transmitting, by the control unit, the first video to the terminal.

[0005] According to a second aspect of the present invention, an information processing method of a server communicating with a terminal includes generating, by a control unit of the server, a first travel itinerary based on first information input by a user to the terminal, generating, by the control unit, a first video related to the first travel itinerary based on the first travel itinerary, and transmitting the first video to the terminal.

[0006] According to a third aspect of the present invention, a server communicating with a terminal includes a control unit that generates a first travel itinerary based on first information input by a user to the terminal, generates a first video related to the first travel itinerary based on the first travel itinerary, and transmits the first video to the terminal.

Brief Description of the Drawings

[0007] [Figure 1] A diagram showing an example of the system configuration of a communication system according to the first embodiment. [Figure 2] A diagram showing an example of a function realized by the control unit of the server according to the first embodiment. [Figure 3] A diagram showing an example of information stored in the storage unit of the server according to the first embodiment. [Figure 4] A diagram showing an example of account registration data related to the first embodiment. [Figure 5] A diagram showing an example of a function realized by the control unit of the terminal according to the first embodiment. [Figure 6] A diagram showing an example of information stored in the memory unit of the terminal according to the first embodiment. [Figure 7] A diagram showing an example of a screen displayed on the display unit of a terminal according to the first embodiment. [Figure 8] A diagram showing an example of a screen displayed on the display unit of a terminal according to the first embodiment. [Figure 9] A diagram showing an example of a screen displayed on the display unit of a terminal according to the first embodiment. [Figure 10] A diagram showing an example of a screen displayed on the display unit of a terminal according to the first embodiment. [Figure 11] A flowchart showing an example of the processing flow performed by each device according to the first embodiment. [Figure 12] A diagram showing an example of a screen displayed on the display unit of the terminal relating to the first modified example. [Modes for carrying out the invention]

[0008] <Compliance with legal matters> Please note that the disclosures described herein are subject to compliance with the laws of the implementing country, including the confidentiality of communications, which are necessary for the implementation of these disclosures.

[0009] <Embodiment> In this specification, there are instances where the phrase "for example" is used for clarity, but please note that this does not limit the content of the descriptions in those instances to the entire set of embodiments described below.

[0010] Embodiments for implementing the program etc. related to this disclosure will be described with reference to the drawings.

[0011] The production of the terminal of the claimed invention (the terminal of the claimed invention) may include the concept that, for example, a state is created in which the functions of the claimed invention become achievable in the terminal when the program described herein (e.g., an application program) is received (or received and stored in the terminal) on a terminal owned (possessed) by a user, thereby creating a state in which the functions of the claimed invention become achievable (a state is created in which the claimed invention becomes executable).

[0012] Furthermore, the production of the system of the claimed invention (the system of the claimed invention) may also include the concept that, for example, a state is created in which the function of the claimed system becomes achievable (a state is created in which the claimed invention becomes executable) by receiving a program described herein (e.g., an application program) transmitted from a server included in the system to a terminal included in the system (or by the received program being stored in the terminal).

[0013] Furthermore, in this specification, a system may be defined as having, for example, multiple devices. Multiple devices may be a combination of devices of the same type, a combination of devices of different types, or a combination of devices of the same type and devices of different types. Furthermore, a system can also be thought of as, for example, a system in which multiple devices work together to perform some kind of processing.

[0014] Also, a system related to a client (client device) and a server can be considered, for example, as at least any one of the following. (1) Terminal & Server (2) Server (3) Terminal

[0015] (1) is, for example, a system including at least one terminal and at least one server. An example of this is a client-server system.

[0016] The server is, for example, composed of the following devices, and it may be a single device or a combination of multiple devices.

[0017] Specifically, the server is, for example, configured to have any one of at least one processor (e.g., CPU: Central Processing Unit, GPU: Graphics Processing Unit, APU: Accelerated Processing Unit, DSP: Digital Signal Processor (e.g., ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array), etc.), a computer device (processor + memory), a control device, an arithmetic device, a processing device, etc. It may have a configuration with multiple of the same type of any one device (e.g., CPU + CPU, homogeneous multi-core processor, etc.) or a configuration with multiple different types of any one device (e.g., CPU + DSP, heterogeneous multi-core processor, etc.), or it may be a combination of multiple devices (e.g., processor + computer device, processor + arithmetic device, a combination of multiple devices made heterogeneous, etc.). Note that the processor may be a virtual processor.

[0018] Furthermore, when a server performs some processing, if it is configured as a single device, the processing described in the embodiment will be performed by that single device. If it is configured as having multiple devices, some processing may be performed by one device and other processing may be performed by the other devices. For example, if it is configured as having a processor and an arithmetic unit, the first processing may be performed by the processor and the second processing may be performed by the arithmetic unit. Furthermore, if the system consists of multiple devices, each device may be arranged in a location that is physically separated from the others.

[0019] Furthermore, server functionality may be provided in the form of PaaS, IaaS, or SaaS in cloud computing, for example.

[0020] Furthermore, the system's control unit can be at least one of either the terminal's control unit or the server's control unit. In other words, for example, the system's control unit can be any of the following: (1A) only the terminal's control unit, (1B) only the server's control unit, or (1C) both the terminal's control unit and the server's control unit.

[0021] Furthermore, the control and processing performed by the system's control unit (hereinafter collectively referred to as "control, etc.") may be (1A) performed solely by the terminal's control unit, (1B) performed solely by the server's control unit, or (1C) performed by both the terminal's control unit and the server's control unit. Furthermore, in (1C), for example, some of the controls that the system would normally perform may be performed by the terminal's control unit, while the remaining controls may be performed by the server's control unit. In this case, the allocation of controls may be equal, or they may be allocated in different proportions.

[0022] Furthermore, when referring to the server's communication unit, if the server is composed of a single device, it may refer to the communication unit itself provided by that single device. Also, if the server is composed of multiple devices, the server's communication unit may include the communication units provided by each of those devices. For example, if a server comprises a first device and a second device, with the first device having a first communication unit and the second device having a second communication unit, the communication unit of the server may be considered a concept that includes both the first and second communication units.

[0023] (2) can be, for example, a system consisting of multiple servers (hereinafter referred to as the "server system"). In this case, the configuration described above can be applied to each server in the same way.

[0024] The control and other functions performed by the server system may be performed by (2A) only one server, (2B) only the other servers, or (2C) one server and the other servers. Furthermore, in (2C), for example, one server may perform some of the control functions that the server system would normally perform, while other servers perform the remaining functions. In this case, the allocation of the controls may be equal, or they may be allocated in different proportions.

[0025] (3) can be, for example, a system composed of multiple terminals. This system can be, for example, as follows: A system that assigns server functions to terminals (a distributed system). This can be achieved, for example, using blockchain technology. A system in which terminals communicate wirelessly with each other. This can be achieved, for example, by using short-range wireless communication technologies such as Bluetooth (registered trademark) to communicate in a P2P (peer-to-peer) manner.

[0026] Furthermore, the above applies not only to the control unit but also to each functional unit that can be a component of the system, such as the input / output unit, communication unit, memory unit, and clock unit.

[0027] In the following embodiments, for example, a system including a terminal and a server (e.g., a client-server system) is illustrated. Furthermore, it is also possible to apply the server system described in (2) above as the server.

[0028] Furthermore, instead of a system that includes terminals and servers, it is also possible to apply a system that does not include servers, such as the system described in (3) above. In this case, the embodiment can be configured based on the aforementioned blockchain technology. Specifically, for example, data stored and managed on the server described in the following embodiment is stored on the blockchain. Then, when a terminal generates a transaction to the blockchain and the transaction is approved on the blockchain, the data stored on the blockchain is updated.

[0029] Furthermore, even when the term "terminal" is used, it is not limited to the meaning of a terminal as a client device in a client-server system. In other words, the term "terminal" can sometimes include the concept of a device that is not part of a client-server relationship.

[0030] Furthermore, the expression "by communication I / F" is used as appropriate in this specification. This may, for example, indicate that the device transmits and receives various information and data via a communication I / F (via a communication unit) based on the control of a control unit (such as a processor).

[0031] Furthermore, in this specification, terms such as "related to" or "concerning" may refer to "B concerning A" or "B related to A," for example, "B" which has some kind of relationship with "A." Specific examples will be given later.

[0032] Furthermore, in this specification, when a device processes two or more things, such as "transmitting A and B" or "receiving A and B," it may include both cases where "A" and "B" are performed at the same time (hereinafter referred to as "simultaneous") and cases where "A" and "B" are performed at different times (hereinafter referred to as "non-simultaneous"). For example, when transmitting first information and second information, the concept may include both cases: transmitting first information and second information at the same time, and transmitting first information and second information at different times. Furthermore, taking into consideration lag (time lag), "simultaneous" may include "almost simultaneous."

[0033] Furthermore, even if "A" and "B" are performed at different times, this only requires that the processing targets "A" and "B," and their purposes do not necessarily have to be the same. For example, when transmitting the first piece of information and the second piece of information as described above, it is sufficient to simply transmit the first piece of information and the second piece of information, and this may include cases where the first piece of information and the second piece of information are transmitted for different purposes, as well as cases where they are transmitted for the same purpose.

[0034] In the following examples, a messaging service is given as an example of a service for users to chat (hereinafter referred to as the "chat service"). Furthermore, an application for implementing the chat service is referred to as a "chat application," and an application for implementing the messaging service is referred to as a "messaging application." A chat application, for example, can allow users to chat in a chat room.

[0035] Furthermore, messaging services (MS) (including instant messaging services: IMS) can be considered a form of social networking services (SNS). Therefore, a distinction may or may not be made between messaging services and social networking services. In other words, messaging services may be included within the definition of social networking services.

[0036] Furthermore, the following embodiment illustrates an example of a messaging service: IMS (Instant Messaging Service), which enables the sending and receiving of simple messages between multiple devices (e.g., terminals) via a server. Instant messaging applications, for example, allow users to have conversations in chat rooms.

[0037] A chat room (for example, a talk room) can be a UI (User Interface) or GUI (Graphical User Interface) that allows each user to view content sent and received between multiple users' devices.

[0038] In messaging services, users can, for example, add any account as a "friend" and exchange messages through a chat room.

[0039] Furthermore, "friends" can be defined as the relationship between a first user and a second user in a messaging application, for example, when the accounts of the first user and the second user are linked unilaterally or bidirectionally. For example, if there is a mutual association between the first user's account and the second user's account, then it can be said that the first user and the second user are "friends." Also, for example, if there is an association between the first user's account and the second user's account, then it can be said that the first user and the second user are "friends" even if there is no association between the second user's account and the first user's account.

[0040] Furthermore, the chat rooms may include one-on-one chat rooms (hereinafter referred to as "one-on-one chat rooms"), group chat rooms containing multiple users (hereinafter referred to as "group chat rooms"), and chat rooms with OA service providers (for example, businesses that partner with messaging service providers) (hereinafter referred to as "OA chat rooms").

[0041] Furthermore, a messaging application account belonging to a business rather than a regular user may be referred to as "(OA (Official Account))", and a user of such an official account may be referred to as an "official user". This may also be referred to as an "official account user" or "official account business," etc. In contrast, a messaging application account belonging to a user who is not an office automation provider is referred to as a "general account," and a user of a general account is referred to as a "general user." This may also be referred to as a "general account user," etc. In other words, messaging application accounts may include both general accounts and official accounts.

[0042] Furthermore, office automation (OA) service providers may also enable users to send and receive messages with other devices via a server, for example, by using terminals similar to those used by general account users.

[0043] Furthermore, a message (message information) may be, for example, information used in a messaging service that defines the sender and recipient, and may consist of identification information (message ID) for identifying the message and the message content. Furthermore, message content may also refer to the contents of a message excluding the message ID. Message content may consist of one or more pieces of content.

[0044] Furthermore, the information set as identification information for identifying a message is called the "message ID." It can be argued that message content contained within messages with the same message ID is identified by that message ID. Therefore, the identification information used to identify a message can be considered substantially synonymous with the identification information used to identify message content. Alternatively, you may or may not set individual identification information (message content ID) for each message content.

[0045] Content may include, for example, text content in text format, image content in image format (including at least one of still images or moving images), and sound content in sound format (including voice). In addition, other elements such as buttons and icons for user interaction, as well as link content such as URIs (including URLs), may also be included.

[0046] The text may include, for example, at least one of the following: national characters represented by character codes, extended characters, machine-dependent characters, numbers, symbols, figures, and codes. Note that the text does not have to contain at least one of the above characters, extended characters, machine-dependent characters, numbers, symbols, figures, and codes, but may also contain other text.

[0047] An image can contain at least one of several types of image information, such as icons, buttons, stamps, emojis, or banner images.

[0048] Contrary to the above definition, it is also acceptable to define "message" as a higher-level concept than "content," or to define "content" and "message" as synonymous, or not to define them in this way.

[0049] <Embodiment> For example, when creating a travel plan or itinerary using an application, the created itinerary can be output as text information, photo information, and map information. However, for example, when a user is visiting a destination for the first time, it may be difficult for the user to grasp the atmosphere of the destination from the text information of the created itinerary.

[0050] This embodiment relates to a service that can create (generate) video information based on a created itinerary and present it to the user. Hereinafter, the video information based on the created itinerary may be referred to as an "itinerary video."

[0051] Here, "itinerary" may include, for example, an action plan for a trip or travel. The itinerary may also include chronological and / or spatial designations (sometimes referred to as "itinerary elements"), such as any of the following. Furthermore, the itinerary may also be called a "travel plan." • Departure location and scheduled departure time • Destination and estimated time of arrival • Transportation methods and travel time • Name and address of the accommodation facility • Tourist place names and addresses • Order of visiting tourist attractions and duration of stay • Dining location and meal content • Budget and payment methods

[0052] The following examples illustrate how itineraries and itinerary videos can be generated based on interactions with a chatbot account in a messaging service. Note that this is not limited to messaging services; itinerary video generation could also be implemented in AI dialogue systems such as ChatGPT.

[0053] Furthermore, in the following, a question-answering system (which may also be called a "question-answering model") may be used as an example of a method for implementing the processing. The question-answering system may be a closed-book question-answering system implemented by a language model such as HyperCLOVA, GPT-4, or Llama. Alternatively, the question-answering model may be an open-book question-answering system implemented by a combination of an external knowledge retrieval model such as DPR (Dense Passage Retriever) or BPR (Binary Passage Retriever) and a language model. The question-answering system may also be capable of generating output sentences (response sentences) in conjunction with an arbitrary external database.

[0054] <First Example> The first embodiment is an embodiment in which the server creates a travel itinerary and travel video for the user based on the content of messages exchanged with the user. The contents described in the first embodiment are equally applicable to any of the other embodiments and any of the other modifications.

[0055] In the following embodiment, for example, a service that provides assistance in creating a travel itinerary will be referred to as a "travel plan creation support service." Furthermore, the travel plan creation support service may be accessible, for example, through a messaging application where the user interacts with a chatbot.

[0056] Furthermore, the following forms can be applied to implement the travel plan creation support service. (A1) A form in which a travel plan creation support service function is provided as a feature of the messaging application. (A2) A form in which a messaging service is provided as one of the functions of a travel plan creation support application that offers a travel plan creation support service. (B1) Forms of an application (integrated application) that has messaging service functionality and travel plan creation support service functionality. (C1) A configuration in which the travel plan creation support application is configured as a separate application from the messaging application. (C2) A configuration in which a messaging application is configured as a separate application from the travel plan creation support application. (D) A configuration in which the travel plan creation support application is configured as a standalone application.

[0057] In the forms of (A1), (A2), and (B1), for example, the messaging service provider can be the same provider as the travel plan creation support service provider. In this case, one possible approach is to use a common account for both the user account in the messaging application and the user account in the travel plan creation support application. Alternatively, in this case, the user's account in the messaging application and the user's account in the travel plan creation support application can be automatically associated (linked).

[0058] In the forms (C1) and (C2), for example, the messaging service provider can be a different company from the travel plan creation support service provider. Furthermore, in configurations (C1) and (C2), it is possible to perform a process that associates the user's account in the messaging application with the user's account in the travel plan creation support application (a process that links them).

[0059] In the following, we will explain assuming, for example, that a messaging service provider operates and manages a server 10 that provides messaging services and travel plan creation support services, in accordance with the form of (A1). That is, the following describes an example in which a travel itinerary (and travel itinerary video) is generated through text communication with a chatbot on a messaging application, but the travel plan generation method in the present invention is not limited to this. For example, in the case of the travel plan creation support application described in (A2) (or another application described in (C1) and (C2)), a means may be used to generate a travel itinerary (and travel itinerary video) by repeatedly asking the user multiple questions about travel destinations, etc. (for example, questions in which the user can select "like" or "dislike" by swiping on the application, etc., and are not limited to this).

[0060] Furthermore, in the following explanation, the user of terminal 20A communicating with server 10 will be referred to as "User AA," the user of terminal 20B as "User BB," the user of terminal 20C as "User CC," and so on.

[0061] <System Configuration> Figure 1 shows an example of the system configuration of the communication system 1 in this embodiment. In the communication system 1, for example, a server 10 and one or more terminals 20 (terminal 20A, terminal 20B, terminal 20C, ...) are connected via a network 30.

[0062] Server 10 has the function of providing services such as messaging services and travel plan creation support services to user-owned terminals 20, etc., via the network 30. Server 10 can also be described as a messaging service server, a travel plan creation support server, an SNS server, etc. In this embodiment, for example, the user of Server 10 is a company that provides messaging services or travel plan creation support services (a provider of messaging services or travel plan creation support services). Note that the number of Server 10 and the number of Terminals 20 connected to Network 30 are not limited to those described above. In this embodiment, the "travel plan creation support service" is a service provided by a business operator (server 10) such as a company that provides travel plan creation support services, and may be provided, for example, to a user (user's terminal 20).

[0063] Terminal 20 (Terminal 20A, Terminal 20B, Terminal 20C, ...) can be any information processing terminal capable of realizing the functions described in each embodiment. Terminal 20 includes, for example, smartphones, mobile phones (feature phones), computers (e.g., desktops, laptops, tablets, etc.), media computer platforms (e.g., cable, satellite set-top boxes, digital video recorders), handheld computer devices (e.g., PDAs (personal digital assistants), email clients, etc.), wearable devices (glasses-type devices, watch-type devices, etc.), VR (Virtual Reality) terminals, smart speakers (voice recognition devices), or other types of computers or communication platforms. Terminal 20 may also be referred to as an information processing terminal.

[0064] The configurations of terminals 20A, 20B, and 20C can, for example, be identical. Furthermore, if necessary, the terminal used by user X may be referred to as terminal 20X, and user information in a predetermined service associated with user X or terminal 20X may be referred to as user information X, or not. User information refers to information about a user associated with an account used by that user in a given service. User information includes, for example, information associated with the user, such as the user's name, user icon image, user's age, user's gender, user's address, user's hobbies and preferences, and user identifier, which are entered by the user or assigned by the given service. It may consist of any one of these items, a combination thereof, or no combination thereof.

[0065] Network 30 plays the role of connecting each device that constitutes communication system 1. In other words, network 30 refers to a communication network that provides connection paths so that the various devices described above can send and receive data after they have been connected.

[0066] One or more parts of network 30 may or may not be wired or wireless networks. Network 30 may include, for example, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), part of the internet, part of a public switched telephone network (PSTN), a mobile phone network, an integrated service digital network (ISDN), wireless LAN, LTE (long term evolution), CDMA (code division multiple access), Bluetooth (registered trademark), satellite communications, or two or more combinations thereof. Network 30 may include one or more networks 30.

[0067] Server 10 (not limited to, but an example of a server, information processing device, or information management device) has the function of providing predetermined services (in this embodiment, messaging services and travel plan creation support services) to terminals 20, etc. Server 10 may be any information processing device that can realize the functions described in each embodiment. Server 10 includes, for example, a server device, a computer (e.g., desktop, laptop, tablet, etc.), a media computer platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computer device (e.g., PDA, email client, etc.), or other types of computers or communication platforms. Server 10 may also be referred to as an information processing device. If it is not necessary to distinguish between Server 10 and Terminal 20, Server 10 and Terminal 20 may or may not be referred to as information processing devices.

[0068] [Hardware configuration of each device] The hardware configuration of each device included in communication system 1 will be described below.

[0069] (1) Terminal hardware configuration Figure 1 shows an example of the hardware configuration of terminal 20. Terminal 20 includes, for example, a control unit 21 (CPU: central processing unit), a storage unit 28, a communication interface 22, an input / output unit 23, a clock unit 29A, and a position calculation information detection unit 29B. Each hardware component of terminal 20 is interconnected via, for example, bus B. It is not necessary for terminal 20's hardware configuration to include all components. For example, terminal 20 may have a configuration in which individual components or multiple components are removed, or it may not.

[0070] The communication interface 22 transmits and receives various types of data via the network 30. Communication may be performed via wired or wireless connections, and any communication protocol may be used as long as communication between the interfaces is possible. The communication interface 22 has the function of communicating with various devices, such as the server 10, via the network 30. The communication interface 22 transmits various types of data to the server 10 and other devices according to instructions from the control unit 21. The communication interface 22 also receives various types of data transmitted from the server 10 and other devices and transmits them to the control unit 21. The communication interface 22 may also be simply referred to as the communication unit. Furthermore, if the communication interface 22 is composed of a physically structured circuit, it may be referred to as the communication circuit.

[0071] The input / output unit 23 includes devices for inputting various operations to the terminal 20, and devices for outputting processing results processed by the terminal 20. The input / output unit 23 may have an integrated input unit and an output unit, or it may have separate input and output units, or it may not have to be integrated.

[0072] The input unit is implemented by any or a combination of any type of device capable of receiving user input and transmitting the information related to the input to the control unit 21. The input unit includes, for example, hardware keys such as touch panels, touch displays, and keyboards, pointing devices such as mice, cameras (operation input via video), and microphones (operation input via voice).

[0073] The output unit is implemented by any or a combination of any type of device capable of outputting the processing results processed by the control unit 21. The output unit includes, for example, a touch panel, a touch display, a speaker (for audio output), a lens (for example, 3D output or hologram output), a printer, and the like.

[0074] This is just one example, but the input / output unit 23 may include, for instance, a display unit 24, an audio input unit 25, an audio output unit 26, and an imaging unit 27.

[0075] The display unit 24 is implemented by any or a combination of any type of device capable of displaying data according to the display data written to the frame buffer. The display unit 24 includes, for example, touch panels, touch displays, monitors (e.g., liquid crystal displays and OLEDs (organic electroluminescence displays)), head-mounted displays (HDMs), projection mapping, holograms, and devices capable of displaying images, text information, etc., in air (which may or may not be a vacuum). These display units 24 may or may not be capable of displaying display data in 3D.

[0076] The sound input unit 25 is used for inputting sound data (including voice data; the same applies hereinafter). The sound input unit 25 includes a microphone, etc. The sound output unit 26 is used for outputting sound data. The sound output unit 26 includes a speaker and the like. The imaging unit 27 is used to acquire image data (including still image data and moving image data; the same applies hereinafter). The imaging unit 27 includes a camera, etc.

[0077] If the input / output unit 23 is a touch panel, the input / output unit 23 and the display unit 24 may be arranged facing each other and have substantially the same size and shape.

[0078] The clock unit 29A is the built-in clock of the terminal 20 and outputs time information (timing information). The clock unit 29A is configured to include, for example, a clock using a crystal oscillator. The clock unit 29A can also be described as, for example, a timing unit or a time information detection unit.

[0079] The clock unit 29A may or may not have a clock that conforms to the NITZ (Network Identity and Time Zone) standard, etc.

[0080] The position calculation information detection unit 29B is a functional unit that detects (measures) information necessary for the control unit 21 to calculate (measure) the position of its own terminal 20 (hereinafter referred to as "position calculation information"). The position calculation information detection unit 29B can also be described as, for example, a position calculation sensor unit.

[0081] The position calculation information detection unit 29B includes, for example, a satellite positioning sensor (satellite positioning unit) which is a sensor or unit for calculating the position of the terminal 20 using a satellite positioning system such as GPS (Global Positioning System), an inertial measurement sensor (inertial measurement unit (IMU)) which is a sensor or unit for calculating the position of the terminal 20 using an inertial navigation system, and a UWB positioning sensor (UWB positioning unit) which is a sensor or unit for calculating the position of the terminal 20 using UWB (Ultra Wide Band).

[0082] The satellite positioning unit includes, for example, an RF receiving circuit that converts RF (Radio Frequency) signals, including positioning satellite signals transmitted from positioning satellites received by an antenna (not shown), into digital signals, and a baseband processing circuit that performs correlation calculations on the digital signals output from the RF receiving circuit to capture positioning satellite signals, and outputs information such as satellite orbit data and time data extracted from the positioning satellite signals as position calculation information.

[0083] The inertial measurement unit has inertial sensors, which are sensors that detect information necessary to calculate the position of terminal 20 by inertial navigation calculations. The inertial sensors include, for example, a 3-axis accelerometer and a 3-axis gyroscope, and output the acceleration detected by the accelerometer and the angular velocity detected by the gyroscope as position calculation information.

[0084] The UWB positioning unit includes, for example, an ultra-wideband RF (Radio Frequency) receiving circuit that converts an ultra-wideband RF signal, including a positioning ultra-wideband pulse signal transmitted from a positioning beacon received by an antenna (not shown), into a digital signal, and a relative position calculation processing circuit that calculates the relative position between the terminal 20 and the positioning beacon based on the digital signal output from the ultra-wideband RF receiving circuit. For example, the UWB positioning unit may or may not make terminal 20 function as a positioning beacon by transmitting an ultra-wideband RF signal including a positioning ultra-wideband pulse signal from an antenna (not shown).

[0085] The control unit 21 calculates the position of its own terminal 20 at regular or specific intervals, for example, based on the position calculation information detected by the position calculation information detection unit 29B. The position of the terminal is referred to as the "terminal position," and the calculated terminal position is referred to as the "calculated terminal position." The control unit 21 may or may not store the calculated terminal position in the storage unit 28 as calculated terminal position history data, associated with the date and time the calculated terminal position was calculated.

[0086] The control unit 21 has physically structured circuitry to perform functions realized by code or instructions contained within the program, and is realized, for example, by a data processing device built into the hardware. Therefore, the control unit 21 may or may not be described as a control circuit.

[0087] The control unit 21 includes, for example, a central processing unit (CPU), a microprocessor, a processor core, a multiprocessor, an ASIC (application-specific integrated circuit), and an FPGA (field programmable gate array).

[0088] The storage unit 28 has the function of storing various programs and data necessary for the operation of the terminal 20. The storage unit 28 includes various storage media such as HDD (hard disk drive), SSD (solid state drive), flash memory, RAM (random access memory), and ROM (read-only memory). The storage unit 28 may or may not be referred to as memory.

[0089] Terminal 20 stores program P in storage unit 28, and by executing program P, the control unit 21 executes the processing of each part included in the control unit 21. In other words, program P stored in storage unit 28 enables terminal 20 to realize the various functions executed by the control unit 21. Furthermore, this program P may or may not be described as a program module.

[0090] (2) Server hardware configuration Figure 1 shows an example of the hardware configuration of server 10. Server 10 includes, for example, a control unit 11 (CPU), a storage unit 15, a communication interface 14, an input / output unit 12, a display unit 13, and a clock unit 19. Each component of the server 10's hardware is interconnected, for example, via bus B. It should be noted that the server 10's hardware configuration does not necessarily include all components. For example, the server 10's hardware may be configured to exclude individual components or multiple components, or it may not.

[0091] The control unit 11 has physically structured circuitry to perform functions realized by code or instructions contained within the program, and is realized, for example, by a data processing device built into the hardware.

[0092] The control unit 11 is typically a central processing unit (CPU), but may also be a microprocessor, processor core, multiprocessor, ASIC, FPGA, or otherwise. In this disclosure, the control unit 11 is not limited to these.

[0093] The storage unit 15 has the function of storing various programs and data necessary for the operation of the server 10. The storage unit 15 is implemented using various storage media such as HDDs, SSDs, and flash memory. However, in this disclosure, the storage unit 15 is not limited to these. Also, the storage unit 15 may or may not be referred to as memory.

[0094] The communication interface 14 transmits and receives various types of data via the network 30. Communication may be performed via wired or wireless connections, and any communication protocol may be used as long as communication between the interfaces is possible. The communication interface 14 has the function of communicating with various devices, such as terminals 20, via the network 30. The communication interface 14 transmits various types of data to various devices, such as terminals 20, according to instructions from the control unit 11. The communication interface 14 also receives various types of data transmitted from various devices, such as terminals 20, and transmits them to the control unit 11. The communication interface 14 may also be simply referred to as the communication unit. Furthermore, if the communication interface 14 is composed of a physically structured circuit, it may be referred to as the communication circuit.

[0095] The input / output unit 12 includes devices for inputting various operations to the server 10, and devices for outputting processing results processed by the server 10. The input / output unit 12 may have an integrated input unit and an output unit, or it may have separate input and output units, or it may not have to be integrated.

[0096] The input unit is implemented by any or a combination of any type of device capable of receiving user input and transmitting the information related to the input to the control unit 11. The input unit is typically implemented by hardware keys such as a keyboard or a pointing device such as a mouse. The input unit may or may not include, for example, a touch panel, a camera (operation input via video), or a microphone (operation input via voice).

[0097] The output unit is implemented by any or a combination of any type of device capable of outputting the processing results processed by the control unit 11. The output unit includes, for example, a touch panel, a touch display, a speaker (sound output), a lens (e.g., 3D output or hologram output), a printer, and the like.

[0098] This is just one example, but the input / output unit 12 may include, for instance, a display unit 13.

[0099] The display unit 13 is implemented by a display or the like. Typically, the display is implemented by a monitor (for example, a liquid crystal display or an OLED (organic electroluminescence display)). The display may also be a head-mounted display (HDM) or not. These displays may or may not be capable of displaying data in 3D. In this disclosure, the display is not limited to these.

[0100] The clock unit 19 is the built-in clock of the server 10 and outputs time information (timing information). The clock unit 19 is configured to include, for example, a hardware clock such as an RTC (Real Time Clock) or a system clock. The clock unit 19 can also be described as, for example, a timing unit or a time information detection unit.

[0101] (3) Others Server 10 stores program P in storage unit 15, and by executing program P, the control unit 11 performs the processing of each part included in the control unit 11. In other words, program P stored in storage unit 15 enables server 10 to realize each function executed by the control unit 11. This program P may or may not be described as a program module. The same applies to other devices.

[0102] Each embodiment of this disclosure is described as being realized by the CPU of terminal 20 and / or server 10 executing program P. The same applies to other devices.

[0103] Furthermore, the control unit 21 of terminal 20 and / or the control unit 11 of server 10 may implement each process not only with a CPU having a control circuit, but also with logic circuits (hardware) or dedicated circuits formed on an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)), etc., or they may not. Also, these circuits may be implemented by one or more integrated circuits, and the multiple processes shown in each embodiment may be implemented by a single integrated circuit, or they may not. In addition, LSIs are sometimes called VLSI, Super LSI, Ultra LSI, etc., depending on the degree of integration. For this reason, the control unit 21 may or may not be described as a control circuit. The same applies to other devices.

[0104] Furthermore, the program P (e.g., a software program, a computer program, or a program module) of each embodiment of the present disclosure may or may not be provided stored on a computer-readable storage medium. The storage medium is a “non-temporary tangible medium” capable of storing the program P. Also, the program P may or may not be for the purpose of implementing some of the functions of each embodiment of the present disclosure. Furthermore, the program P may or may not be a so-called differential file (differential program) that can implement the functions of each embodiment of the present disclosure in combination with a program P already recorded on the storage medium.

[0105] The storage medium may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital cards, or drives, any other suitable storage medium, or any two or more suitable combinations thereof. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate. The storage medium is not limited to these examples and may be any device or medium capable of storing program P. The storage medium may or may not be referred to as memory.

[0106] The server 10 and / or terminal 20 can realize the functions of the multiple functional units shown in each embodiment by reading the program P stored in the storage medium and executing the read program P. The same applies to other devices.

[0107] Furthermore, the program P of this disclosure may or may not be provided to the server 10 and / or terminal 20 via any transmission medium capable of transmitting programs (such as a communication network or broadcast waves). The server 10 and / or terminal 20 realize the functions of the multiple functional units shown in each embodiment by executing the program P downloaded, for example, via the Internet. The same applies to other devices.

[0108] Furthermore, each embodiment of this disclosure can also be realized in the form of a data signal in which the program P is embodied by electronic transmission. At least part of the processing on server 10 and / or terminal 20 may or may not be implemented by cloud computing consisting of one or more computers. The configuration may be such that at least some or all of the processing on terminal 20 is performed by server 10, or it may not be so. In this case, the configuration may be such that at least some or all of the processing of each functional unit of the control unit 21 of terminal 20 is performed by server 10, or it may not be so. The configuration may be such that at least some or all of the processing in the server 10 is performed by the terminal 20, or it may not be so. In this case, the configuration may be such that at least some or all of the processing of each functional unit of the control unit 11 of the server 10 is performed by the terminal 20, or it may not be so. Unless otherwise explicitly stated, the configuration of the determination in the embodiments of this disclosure is not mandatory, and a predetermined process may be performed if the determination condition is met, or if the determination condition is not met, or not.

[0109] The programs disclosed herein are implemented using, for example, scripting languages ​​such as ActionScript and JavaScript®, compiled languages ​​such as Objective-C and Java®, and markup languages ​​such as HTML Living Standard.

[0110] [Functional Configuration of Each Device] (1) Server Functional Configuration Figure 2 shows an example of a function implemented by the control unit 11 of the server 10 in this embodiment. The control unit 11 includes, for example, an application management processing unit 111 as a functional unit.

[0111] The application management processing unit 111 has the function of executing application management processing according to an application management processing program 151 stored in the storage unit 15, for example.

[0112] Figure 3 shows an example of the information stored in the storage unit 15 of the server 10 in this embodiment. The memory unit 15 stores, for example, an application management processing program 151 that is executed as an application management process, and account registration data 153.

[0113] The account registration data 153 is registration data related to an application account (in this embodiment, a messaging application), and an example of its data structure is shown in Figure 4. The account registration data 153 stores, for example, the username, application ID, and other registration information in association with each other.

[0114] The username is the name of the account on device 20 using this application. For example, the name registered by the user of device 20 when using the application is stored.

[0115] An application ID is information used to identify an application account, or the account itself. This application ID is preferably a unique value for each account, for example, a unique value (unique value) is set and stored for each account by the server 10. The Application ID is information associated with terminal 20 or the user of terminal 20, and is an example of information about the terminal or the user of the terminal.

[0116] Other registration information may include, for example, identification information to identify terminal 20, the phone number of terminal 20 (terminal phone number), email address (terminal email address), and authentication information such as passwords used for various authentications in the application (login password, authentication password, etc.).

[0117] Identification information for identifying terminal 20 can be, for example, a terminal ID (e.g., IMEI (International Mobile Equipment Identity)).

[0118] For example, user information may be associated with and stored in other registration information.

[0119] Note that you may or may not use a "User ID" instead of an "Application ID". Also, if the application can only register one account per terminal 20, then for example, "Identification information for identifying terminal 20 = Identification information for identifying the user of terminal 20 = Application ID" can be used.

[0120] Furthermore, for example, it may or may not be possible to assign multiple terminal IDs to a single application ID. In this case, it may or may not be possible to launch the application simultaneously on multiple terminals 20 using a single application ID as the target for identification (login).

[0121] Furthermore, it is possible to apply a method of managing accounts using information such as the terminal's phone number instead of various IDs such as application IDs. In this case, instead of storing information such as the application ID in the account registration data 153, information such as the terminal phone number can be stored in the account registration data 153. Alternatively, the application ID information may be associated with the terminal phone number information on a one-to-one basis, or it may not be necessary to do so.

[0122] For the sake of simplicity, in the following embodiment, it will be assumed that one account is registered for each terminal 20. Furthermore, in this case, as described above, "identification information for identifying terminal 20 = identification information for identifying the user of terminal 20 = application ID," therefore, the term "user of the account" used in the following explanation may or may not be considered synonymous with "terminal of the account."

[0123] (2) Terminal Functional Configuration Figure 5 shows an example of a function realized by the control unit 21 of the terminal 20 in this embodiment. The control unit 21 includes, for example, an application processing unit 211 as a functional unit for executing application processing according to an application processing program 281 stored in the storage unit 28.

[0124] Figure 6 shows an example of data stored in the storage unit 28 of the terminal 20 in this embodiment. The memory unit 28 stores, for example, an application processing program 281 that is executed as application processing, and an application ID 283 that corresponds to the terminal 20 or the user account of the terminal 20.

[0125] <Display screen> The following describes examples of display screens. The screen transitions described below are merely examples of screen transitions for implementing the method of this disclosure. Some screens may be omitted or additional screens may be added to the screen transitions exemplified below. Furthermore, the terminology used in the display screens described below may differ from the terminology used in the processes described later (some terms may not be consistent).

[0126] In the following example, we will describe a case where terminal 20 is a smartphone equipped with a vertically oriented display unit 24. The smartphone has, for example, a touch panel that functions as an input unit, positioned opposite its display, to form a touchscreen. When an element such as an icon, button, item, or input area is displayed on the display, and a part of the touch panel that is opposite the area on which the element is displayed is operated by the user, the program associated with that element or a subroutine of that program may be executed.

[0127] In addition, the flow of screen transitions may be explained by showing predetermined screens displayed on different terminals 20, either within a single drawing or across multiple drawings. In this case, for example, the predetermined screen may be displayed on different terminals 20 based on (or triggered by) at least one of the following: • Automatically displayed • When a notification (including push notifications, etc.; the same applies hereinafter) is sent to device 20, and the user takes action in response to the notification, it will be displayed. • No notification is sent to device 20, but it will be displayed when the user performs a specific action on the application. • A notification is sent to terminal 20, and the user performs a specified action on the application that sent the notification, which then displays the message.

[0128] The following example illustrates the case where user AA's terminal 20A is used as an account for creating a travel plan. In this example, the travel plan will be created within a messaging application, and the messaging application will be referred to as "Messaging App" as an example. Furthermore, the creation of travel plans may be performed on a separate application from the messaging application, such as a travel plan creation support application. In this case, the same screen configuration may be used.

[0129] Figures 7 to 10 show examples of screens displayed on the display unit 24 of terminal 20A in this embodiment. Additionally, where the diagrams refer to a "travel plan," we will explain it as an "itinerary" where appropriate.

[0130] The left side of Figure 7 shows an example of a talk room screen displayed on terminal 20A, showing a one-on-one talk room where the talk partner (chat partner) is a chatbot. Note that while the term "chatbot" is used here, it is not limited to this. For example, the account could be referred to as an AI chatbot or AI agent. This screen may be displayed when the user selects the chatbot within a messaging application, similar to how a message is sent to a friend within the messaging application. Alternatively, this screen may be displayed when, for example, a function button for launching the travel plan creation support service is tapped from the main menu or selection menu within a chat room of the messaging application.

[0131] The chat room screen is configured with a content display area (hereinafter referred to as the "content display area"), and for example, the content you have sent (content originating from you) may be displayed on the right side of the screen, and the content the other party has sent (content originating from the other party) may be displayed on the left side of the screen. Furthermore, the content may be displayed in association with information that allows for the identification or identification of the source, for example, in association with the user's registered icon image. The content may also be displayed in a speech bubble. In this case, the text within the speech bubble may be considered the content, or the speech bubble as a whole may be considered the content. Additionally, depending on the type of content and the information it contains, you may choose not to display it in a speech bubble. A single piece of content may include different elements (for example, text, images, and objects such as buttons and icons), and such content may be referred to as "composite content."

[0132] This screen shows a conversation between user AA and a chatbot about travel, with user AA considering a trip to Okinawa. The chatbot then proposes a 2-night, 3-day trip to Okinawa, and user AA requests that the chatbot change it to a 1-night, 2-day trip. The chatbot agrees and displays that it is currently creating the itinerary.

[0133] In the center of Figure 7, the continuation of the conversation in the central chat room screen is shown. Below content CT1, which indicates that the itinerary created (generated) by the chatbot will be presented as a video, the itinerary video information display area TVR1 is displayed as an area for displaying content sent by the chatbot.

[0134] The travel itinerary video information display area TVR1 is configured to display and play the travel itinerary video created during the travel itinerary video creation process described later. For example, a seek bar indicating the video playback position is displayed at the bottom of the travel itinerary video information display area TVR1. Also, for example, a share button SB1 for sharing the currently playing travel itinerary video or the travel itinerary on which the video was created with other users, and a map display button MB1 for viewing the itinerary on an electronic map are displayed at the bottom right of the travel itinerary video information display area TVR1.

[0135] Above the travel itinerary video information display area TVR1, a video scene title display area TE1 is displayed, indicating which travel itinerary element the currently playing video scene corresponds to. Below the video scene title display area TE1, a detailed information display button BT1 is displayed, which allows the user to view detailed information about travel itinerary elements such as tourist destinations (places to be visited) included in the created travel itinerary information. The detailed information may be considered as content, and may be configured as, for example, composite content. Alternatively, the detailed information may be considered as travel itinerary information. As shown in Figure 7, the video scene title display area TE1 is displayed in a position that overlaps with the video, but this is not limited to that. For example, it may be displayed in an area above or below the video that does not overlap with the video, or it may be displayed near the seek bar. Furthermore, the display position of TE1 may change in accordance with the content of the video (for example, it may change to a position that avoids overlapping with major landmarks in the video), or the display position of TE1 may change in conjunction with the playback position indicator marks on the seek bar.

[0136] On this screen, for example, a scene from a video showing someone traveling by airplane is displayed. In addition, the scene text "3 hours from Tokyo," which will be described later, is superimposed and displayed by the control unit 21 of terminal 20A, in accordance with the scene. The scene text may also be embedded as image information within the video scene. In the video scene title display area TE1, the text indicating the itinerary element, "Travel to Miyako Airport," is displayed. For example, when the detailed information display button BT1 is tapped, the screen may transition to a page displaying flights for dates and times that match the itinerary. Alternatively, for example, it may be possible to transition from the flight display page to the flight reservation page of each airline.

[0137] On the right side of Figure 7, the continuation (next scene) of the itinerary video in the itinerary video information display area TVR1 in the talk room screen in the center of Figure 7 is displayed. On this screen, for example, it can be seen that the itinerary after arriving at Miyako Airport includes the element "Take a break at ○× Park". For example, when the detailed information display button BT1 is tapped, the guide for "○× Park" may be displayed.

[0138] On the left side of Figure 8, the continuation (next scene) of the itinerary video in the itinerary video information display area TVR1 in the talk room screen on the right side of Figure 7 is displayed. On this screen, for example, it can be seen that the itinerary includes the element "playing in the water at XX Beach" as part of the itinerary after spending time at "XX Park". For example, when the detailed information display button BT3 is tapped, the guide for "XX Beach" may be displayed.

[0139] In the center of Figure 8, the final scene of the itinerary video in the itinerary video information display area TVR1 is displayed. On this screen, for example, it can be seen that the itinerary element "Relax at △△ Hotel until departure" is incorporated as the final plan of the itinerary. For example, when the detailed information display button BT9 is tapped, the guide and reservation page for "△△ Hotel" may be displayed.

[0140] The right side of Figure 8 shows the continuation of the talk room screen that appears when the playback of the itinerary video in the itinerary video information display area TVR1 has finished. On this screen, for example, below the content CT1 mentioned above, the itinerary video content TVC1, which is a reduced (thumbnail) version of the itinerary video information display area TVR1, is displayed. For example, if the itinerary video content TVC1 is tapped, the itinerary video information display area TVR1 may be expanded again and the itinerary video may start playing.

[0141] Below the travel itinerary video content TVC1, travel itinerary information CT2 is displayed, which is an example of information based on the created itinerary. The travel itinerary information can also be considered as content, and may be structured as, for example, composite content. In this example, it is displayed as text-based travel itinerary information. Below that, content CT3 is displayed, in which user AA indicates that there is an additional tourist attraction (in this example, "sea kayaking") that they would like to add to the tourist attractions included in the above itinerary.

[0142] On the left side of Figure 9, the conversation continues from the talk room screen on the right side of Figure 8. Below Content CT4, which indicates that the chatbot will present a video of the itinerary recreated based on the content of Content CT3, the itinerary video information display area TVR2 is displayed. For example, if the user swipes left on the itinerary video information display area TVR2, it may be possible to skip to the next scene in the itinerary video without waiting for the video to play.

[0143] In the center of Figure 9, the next scene of the itinerary video in the TVR2 display area of ​​the itinerary video information in the talk room screen on the left side of Figure 9 is displayed. In this screen, for example, it can be seen that the itinerary element "Swimming at XX Beach" has been incorporated as part of the itinerary after traveling to "Miyako Airport". In other words, it can be seen that the itinerary element "OXX Park" has been omitted. When the user swipes left, a chapter change operation is performed, and the display changes to the chat room screen shown on the right side of Figure 9. For example, in the travel itinerary video information display area TVR2, a right swipe by the user may return the display to the previous scene. Chapter navigation is also effective in the travel itinerary video information display area TVR1.

[0144] On the left side of Figure 8, the continuation (next scene) of the itinerary video in the itinerary video information display area TVR1 in the talk room screen on the right side of Figure 7 is displayed. On this screen, for example, it can be seen that the activity "sea kayaking" is incorporated as an itinerary element following "playing in the water at XX Beach". For example, when the detailed information display button BT4 is tapped, the guide page or tour booking screen for "sea kayaking" may be displayed.

[0145] Figure 10 shows an example of the talk room screen when the playback of the itinerary video has finished in the itinerary video information display area TVR2. In this screen, the itinerary video content TVC2, which is a reduced version of the itinerary video information display area TVR2, is displayed below content CT4. The user can easily reconfirm the itinerary before and after modification and select their preferred itinerary by, for example, tapping on the itinerary video content TVC1 or itinerary video content TVC2.

[0146] <Processing> Figure 11 is a flowchart showing an example of the processing flow performed by each device in this embodiment. In this figure, from left to right, it shows an example of the processing performed by the control unit 21 of user AA's terminal 20A, and an example of the processing performed by the control unit 11 of server 10. The processes described below are merely examples of processes for implementing the method of this disclosure, and are not limited to these. Additionally, you may add other steps to the process described below, or omit (delete) some steps from the process described below.

[0147] First, the control unit 21 of terminal 20A sends itinerary generation request information to the server 10 for generating an itinerary, based on, for example, input to the input / output unit 23 of terminal 20A (for example, user input (user operation input, sound input, etc.); the same applies hereinafter). (A110) The itinerary generation request information may also be in the form of a natural language string such as "Create a travel plan." Alternatively, the itinerary generation request information may also be, for example, a command in an API of a travel plan creation support service.

[0148] The itinerary generation request information may include, for example, information about the general destination (location), information about the length of stay (e.g., "one night, two days"), information about the dates (e.g., "next weekend" or "from December 23rd to December 26th"), and information about the purpose of the trip.

[0149] The input method for the itinerary generation request information may be as follows, for example: • Posting messages to chat rooms in messaging services • Operation of the itinerary generation button from the chat room menu in messaging services, or input from a selection menu (form). • Text input (interactive instructions) in travel plan creation support applications • Inputting selection menus (forms) in travel plan creation support applications.

[0150] When the server 10 receives the itinerary generation request information from terminal 20A, the control unit 11 of the server 10 executes the itinerary generation request acceptance process (S110). In the process of receiving a travel itinerary generation request, the control unit 11 of the server 10 determines, for example, whether or not a travel itinerary has been instructed to be created, using a question-answering model (not shown).

[0151] For example, in the process of receiving a travel itinerary generation request, if the travel itinerary generation request information does not include information about the travel destination or itinerary, the control unit 11 of the server 10 may send travel itinerary confirmation information to the terminal 20A to confirm the travel destination and itinerary. The control unit 21 of the terminal 20A may, for example, display the travel itinerary confirmation information on the display unit 24 upon receiving it. Based on user input regarding the itinerary confirmation information, the control unit 21 of terminal 20A may send itinerary specification information such as the travel destination and dates to the server 10.

[0152] For example, if it is determined that an instruction has been given to create a travel itinerary, the control unit 11 of the server 10 executes the travel itinerary summary creation process (S120).

[0153] In the process of creating a travel itinerary summary, the control unit 11 of the server 10 creates travel itinerary summary information based on the instructions in the travel itinerary generation request information and travel itinerary specification information, for example, using a question answering model (for example, a closed-book question answering system).

[0154] In the process of creating travel itinerary summary information, for example, if the control unit 11 of server 10 receives the string "I want to go to Okinawa, so please create a travel plan" as travel itinerary generation request information and the string "Make it a one-night, two-day trip" as travel itinerary specification information, it may input a first prompt statement to the question response model, for example, separating the travel itinerary generation request information and the travel itinerary specification information with punctuation marks, such as "I want to go to Okinawa, so please create a travel plan. Make it a one-night, two-day trip." Note that the travel itinerary specification information may be omitted.

[0155] Furthermore, in the process of creating the itinerary summary information, for example, if the control unit 11 of the server 10 receives the selection menu item "Create a travel plan" as itinerary generation request information, and the selection menu items "Region: Okinawa" and "Dates: 1 night, 2 days" as itinerary specification information, it may input the first prompt statement to the question response model, for example, by separating the string "Please generate a travel plan" which is associated with the selection menu item "Create a travel plan" as itinerary generation request information, the string "Please set the destination to Okinawa" which is associated with the selection menu item "Region: Okinawa" as itinerary specification information, and the string "Please set the dates to 1 night, 2 days" which is associated with the selection menu item "Dates: 1 night, 2 days" with punctuation marks. Note that itinerary specification information may be omitted.

[0156] The itinerary summary information may include any of the following elements, for example: • Departure point (for example, the station or airport where the outbound journey will end) • Place of visit (e.g., name or address of the facility) • Dining establishment (for example, the name of the restaurant or the menu) • Accommodation location (e.g., hotel name and address) • Destination (for example, the station or airport that will be the departure point for the return journey)

[0157] The first prompt statement may include content indicating that any of these elements should be included as output. The first prompt statement may also include prompts requesting more detailed schedule or cost output.

[0158] The itinerary summary information may, for example, be a generated statement describing the general content of the travel plan. Alternatively, the itinerary summary information may be a list listing the departure point, destinations, and accommodations. The itinerary summary information may also include information such as a general schedule and estimated costs.

[0159] For example, once the itinerary summary information is created, the control unit 11 of the server 10 executes the itinerary element detail search process (S130). In the detailed itinerary element search process, the control unit 11 of the server 10 uses, for example, a question answering model (e.g., an open-book question answering system) to create itinerary information, which is a feasible travel plan, based on the itinerary summary information, using an unillustrated travel information database as external knowledge (passage). A travel information database may store various types of information, such as guide information about tourist destinations (e.g., required sightseeing time, fees, URIs of tourist facility websites, etc.), information on route and travel time searches, map information, restaurant reservation information, hotel rates and availability information, and weather forecasts.

[0160] In the detailed itinerary element search process, the control unit 11 of the server 10 may input a second prompt to the question answering model, requesting it to search for feasible routes and / or accommodations available on the planned dates of stay, using the departure point, destinations, and accommodations output as itinerary summary information as a draft. It may then generate itinerary information, which is the output statement of the question answering model. The itinerary information may include any of the aforementioned elements related to the itinerary.

[0161] The detailed itinerary element search process can eliminate unrealistic travel plans generated by hallucination in language models.

[0162] Furthermore, by separating the itinerary summary creation process, which requires relatively few resources and is achievable through a closed-book question-answering system, from the itinerary element detail search process, itinerary summary information can be generated efficiently. Then, by executing the itinerary element detail search process based on the generated itinerary summary information and generating itinerary information that is refined to reflect reality, the prompt length in the itinerary element detail search process, which requires access to a large amount of external resources, can be reduced.

[0163] Furthermore, the process of exploring detailed itinerary elements is not limited to using a question-answering model. For example, the control unit 11 of the server 10 may refer to a travel information database based on the departure point, destinations, etc., in the itinerary summary information and the schedule, and search for an itinerary that can be realized by DP (Dynamic Programming). The searched itinerary may then be output as itinerary information.

[0164] Furthermore, it is not necessary to explicitly distinguish between the itinerary summary creation process and the itinerary element detail search process. For example, the control unit 11 of the server 10 may create itinerary information by referring to the travel information database and performing the same processing as the itinerary summary creation process.

[0165] Once the itinerary information is created, the control unit 11 of the server 10 executes the itinerary video creation process (S140).

[0166] In the process of creating a travel itinerary video, the control unit 11 of the server 10 performs, for example, section part determination processing, subtitle generation processing, video image preparation processing, and video image editing processing.

[0167] In the section part determination process, the control unit 11 of the server 10 determines, for example, the number of sections in the itinerary video and the scene content associated with each section, based on the itinerary information. For example, the control unit 11 of the server 10 may input a prompt message to the question answering model such as, "Please divide the content included in the following itinerary into 'N' parts. Please associate the divided content with the itinerary summary. ##<<Itinerary Information>>" (where <<Itinerary Information>> is entered is the created itinerary information). Here, the number of sections "N" in the itinerary video may be, for example, a number instructed by the user, or a number predetermined by the system (for example, "N=9"). Alternatively, the number of sections "N" in the itinerary video may be determined to an appropriate number by the question answering model.

[0168] Furthermore, the number of sections (number of scenes) may differ depending on the duration of the itinerary based on the generated itinerary information. For example, the number of sections "N" in the itinerary video may be determined such that the number of sections for a five-night, six-day itinerary (e.g., 20) is greater than the number of sections for a one-night, two-day itinerary (e.g., 4). Here, since the length of the generated itinerary video differs depending on the determined number of sections (and the corresponding number of scenes), the control that determines the number of sections based on the duration of the itinerary is an example of the control that determines the length of the itinerary video based on the duration of the itinerary.

[0169] In the section part determination process, once the number of sections in the travel itinerary video and the content of the travel information (travel elements) corresponding to each scene are determined, the control unit 11 of the server 10 executes, for example, a subtitle generation process. In the subtitle generation process, the control unit 11 of the server 10 inputs a prompt message to the question-answering model for each scene of the travel video, for example, "Think of a catchy slogan that matches the <<travel element>>" (where the created travel element is input for <<travel element>>), and generates a catchy slogan for each scene of the travel video (for example, "The island closest to heaven"). Alternatively, in the subtitle generation process, the control unit 11 of the server 10 may input a prompt message to the question-answering model for each scene of the travel video, for example, "Think of a factual caption that corresponds to the content of the <<travel element>>", and generate a caption for each scene of the travel video (for example, "A 3-hour flight from Tokyo").

[0170] The catchphrases and captions generated during the subtitle generation process may be collectively referred to as "scene text."

[0171] The control unit 11 of server 10 may perform subtitle generation processing for all scenes. Alternatively, the control unit 11 of server 10 may perform subtitle generation processing for any selection of scenes. Furthermore, the control unit 11 of server 10 may choose not to perform subtitle generation processing at all. In other words, scene text may be added to any scene in the travel video.

[0172] For example, when the subtitle generation process is executed, the control unit 11 of the server 10 executes, for example, the video image preparation process. In the video image preparation process, for example, for each scene of the travel video, the control unit 11 of the server 10 searches for a video image appropriate to the scene using a video image search engine (not shown) based on the scene and the corresponding travel elements and / or scene text.

[0173] The search destination for video may be, for example, a video database pre-stored in the storage unit 15 of server 10. Alternatively, the search destination may be, for example, any image database on the internet. Alternatively, the search destination may be, for example, a server of an image stock service partnered with the server operator of server 10. The video to be searched may be UGC (User Generated Content) posted by users to video sharing services, etc.

[0174] The process of preparing video and image materials to search for video and image materials can also be called, for example, a "video and image material search process."

[0175] In the video image preparation process, the control unit 11 of the server 10 may, for example, generate one or more still images for each scene of the travel video based on the travel elements and / or scene text corresponding to the scene, and then generate a video based on the generated still images. Alternatively, in the video image preparation process, the control unit 11 of the server 10 may, for example, directly generate a video for each scene of the travel video based on the travel elements and / or scene text corresponding to the scene. For video generation, a video generation model such as Sora, Make-A-Video, or Imagen may be used.

[0176] The process of preparing video and image materials to generate video and image materials can also be called, for example, "video and image material generation process."

[0177] For example, if the scene text includes phrases like "making friends with sea turtles," the video image search process may be configured to select video images from user-generated content (UGC) related to sea turtle tours as source material. Furthermore, for example, if the scene text includes phrases such as "the island closest to heaven," the video image generation process may be configured to generate a video image of the word "heaven" written on a beach as source material.

[0178] For example, when the video image preparation process is executed, the control unit 11 of the server 10 executes, for example, the video image editing process. In the video image editing process, the control unit 11 of the server 10, for example, stitches together the videos of each searched or generated part. Then, for example, it performs color and tone conversion (e.g., LUT conversion) on the stitched videos to generate a travel itinerary video.

[0179] For example, in the video image editing process, the control unit 11 of the server 10 may render the scene and the corresponding scene text string as an image (subtitle) within the video for each scene of the travel video and then composite them together.

[0180] Furthermore, for example, in the video image editing process, the control unit 11 of the server 10 may perform BGM search processing and BGM generation processing. In the BGM search process, for example, the control unit 11 of the server 10 may input a prompt message such as "Please search for BGM that matches the content included in the following itinerary. ##<<Itinerary Information>>" to the question answering model, determine a song to be used as background music for the itinerary video, download the sound source, and synthesize it into the audio track of the itinerary video. Furthermore, in the BGM generation process, for example, the control unit 11 of the server 10 may input a prompt message such as "Please compose background music that matches the content included in the following itinerary. ##<<Itinerary Information>>" using a music generation model such as Jukedeck or SOUNDRAW, generate a song to be used as background music for the itinerary video, and synthesize it into the audio track of the itinerary video.

[0181] For example, when a travel itinerary image is created (generated), the control unit 11 of the server 10 transmits travel itinerary video information including the travel itinerary image to the terminal 20A (S150). The travel itinerary video information may include, for example, the travel itinerary image edited in the video image editing process, information about the time when the video images were stitched together in the travel itinerary image (which may also be called chapter information), and information about the travel itinerary summary information corresponding to each video image (for example, information about the visited locations corresponding to the video images). The travel itinerary image information may also include, for example, scene text corresponding to each scene.

[0182] Upon receiving the travel itinerary video information, the control unit 21 of terminal 20A outputs the received travel itinerary video information (for example, display output to the display unit 24 or sound output to the sound output unit 26) (A120). The travel itinerary video information is displayed on terminal 20A in the chat room in which the user of terminal 20A participates. In other words, the control by the control unit 11 of server 10 to send the travel itinerary video information to terminal 20A is an example of control that displays the travel itinerary video information in the chat room in which user A of terminal 20A participates.

[0183] For example, when travel itinerary video information is transmitted, the control unit 11 of the server 10 transmits the travel itinerary information to the terminal 20A (S160). Furthermore, the control unit 11 of the server 10 may choose not to transmit the itinerary information to the terminal 20A. In other words, it may transmit only the itinerary video information to the terminal 20, out of the itinerary information and the itinerary video information.

[0184] Upon receiving the itinerary information, the control unit 21 of terminal 20A outputs the received itinerary information (for example, by displaying it on the display unit 24) (A130). The itinerary information is displayed on terminal 20A as content related to the itinerary video information, after the itinerary video information, in the chat room in which the user of terminal 20A participates. In other words, the control by the control unit 11 of server 10 to send the itinerary information to terminal 20A is an example of control that displays the itinerary information in the chat room in which user A of terminal 20A participates.

[0185] Here, the terminal 20A may display the itinerary information after at least a portion of the itinerary video information (for example, 1 minute out of 3 minutes, or 3 minutes out of 3 minutes) has been output. For example, the control unit 21 of the terminal 20A may request the itinerary information from the server 10 based on the fact that at least a portion of the itinerary video information has been output, and in response to the request, the server 10 may send the itinerary information to the terminal 20A, and the received itinerary information may be displayed in the terminal 20A as content after the itinerary video information.

[0186] Furthermore, the control unit 21 of terminal 20A may perform control to display additional received itinerary information based on the fact that at least a portion of the previously displayed itinerary video information and itinerary information received from server 10 has been output. Alternatively, the control unit 11 of server 10 may, based on the fact that at least a portion of the itinerary video information has been output in the talk room in which user A of terminal 20A is participating, send an instruction to terminal 20A to display the received itinerary information, and in response to that instruction, the received itinerary information may be additionally displayed on terminal 20A.

[0187] However, the output order is not limited to this; the control unit 21 of terminal 20A may start outputting the itinerary video information after displaying the itinerary information. For example, the server 10 may send the itinerary information to terminal 20A first, and then send the itinerary video information (i.e., display the itinerary information first in the chat room, and then display the itinerary video information).

[0188] For example, if it is determined that the user has selected to modify an itinerary element based on the user's input regarding the displayed itinerary information (A140:YES), terminal 20A obtains the itinerary element modification request information instructed by the user and sends it to server 10 (A150).

[0189] Furthermore, the itinerary element modification request information may be an API command based on an operation on the itinerary information, or it may be a text or voice modification instruction input using natural language. As shown in CT3 in Figure 10, when instructions are received to add certain elements via modification instruction text, those elements may be added to the video and travel plan. Similarly, when instructions are received to remove certain elements, those elements may be removed from the video and travel plan. Furthermore, it is conceivable that a user who has watched the video may grasp the overall atmosphere of the trip and give instructions such as "I would like a more relaxed trip." To accommodate such instructions, the video and travel plan may be modified using a generative model such as GPT-4.

[0190] Then, terminal 20A returns to waiting to receive travel itinerary video information again.

[0191] If it is determined that no modification of the itinerary elements has been chosen (A140: NO), terminal 20A performs, for example, a termination check (A190). If it is determined to continue processing (A190: NO), the control unit 21 of terminal 20A returns processing to, for example, A110. On the other hand, if it is determined to terminate processing (A190: YES), the control unit 21 of terminal 20A terminates processing.

[0192] If the server determines that it has received a request to modify itinerary elements from terminal 20A (S170: YES), for example, the server 10 creates itinerary summary information that reflects some or all of the changes to the itinerary elements based on the itinerary element modification request information, and then executes the itinerary element detail search process again (S130). That is, it generates itinerary information that reflects the changes to the itinerary elements, executes the itinerary video creation process based on the generated itinerary information, sends the modified itinerary video information to terminal 20A, and sends the modified itinerary information to terminal 20A. The display control of the modified itinerary video information and the modified itinerary information is the same as the display control of the original itinerary video information and the original itinerary information described above. As a result, on terminal 20A, thumbnails of the itinerary video content before and after the revision are displayed as selectable options (see Figure 10), and either content can be played depending on the user's operation.

[0193] Furthermore, if, for example, a significant change occurs to the itinerary elements in the itinerary element modification request information (for example, a change in the travel destination itself), the server 10 may return processing to the itinerary summary creation process. Then, it may create itinerary summary information based on the itinerary element modification request information.

[0194] If it is determined that no travel itinerary element modification request information has been received (S170: NO), for example, server 10 performs a termination determination (S190). If it is determined to continue processing (S190: NO), the control unit 11 of the server 10, for example, waits for the arrival of itinerary generation request information and returns processing to S110. On the other hand, if it is determined to terminate processing (S190: YES), the control unit 11 of the server 10 terminates processing.

[0195] <Effects of the First Example> In this embodiment, a control unit 11 (an example of a control unit) of a server 10 (an example of a server 10) that communicates with a terminal 20 (an example of a terminal) generates itinerary information (an example of a first itinerary) based on itinerary generation request information (an example of first information) entered into the terminal by the user. Furthermore, the control unit of the server executes an itinerary video generation process to generate itinerary video information (an example of a first video) related to the first itinerary based on the first itinerary, and transmits the first video to the terminal. This makes it possible to provide users with a first video, allowing them to use the first video as a reference to understand information about the place they want to visit.

[0196] Furthermore, according to this embodiment, since the control unit can transmit the first itinerary to the terminal, the user can view the first video and confirm the provided first itinerary.

[0197] Furthermore, according to this embodiment, after the first video is played, the control unit can execute a control to display the first itinerary on the terminal. As a result, the user can confirm the first itinerary after gaining a concrete understanding of its outline through the video.

[0198] Furthermore, according to this embodiment, the first video consists of multiple scenes, and in the subtitle generation process, the control unit generates scene text (an example of text corresponding to a scene), and in the video image editing process, the control unit synthesizes the text with the scenes. Therefore, text that abstractly and concisely represents the scenes can be presented to the user.

[0199] Furthermore, according to this embodiment, the first video consists of multiple scenes, and in the subtitle generation process, the control unit can generate scene text (an example of text corresponding to a scene) and send the text to the terminal. Therefore, the terminal can display the text in any position and manner corresponding to the scenes.

[0200] Furthermore, according to this embodiment, in the BGM generation process, the control unit generates BGM for the first video based on the first journey, and the control unit synthesizes the first video and the BGM. The first video with the synthesized BGM can then be sent to the terminal, allowing the user to grasp the impression of the first journey in a multimodal manner.

[0201] Furthermore, this embodiment shows an example configuration in which the server's control unit generates new itinerary information (an example of a second itinerary) based on itinerary element modification request information (an example of modification information for the first information entered into the terminal by the user), and executes an itinerary video generation process to generate itinerary video information (an example of a second video) related to the second itinerary based on the second itinerary, and transmits the second video to the terminal. According to this, the server can generate a second itinerary based on the first information and corrected information entered by the user into the terminal, and a second video related to the second itinerary, and send the second video to the terminal. Therefore, if a user wants to watch a video based on an itinerary different from the first itinerary, the server can provide the user with a second video related to the second itinerary, which is the first itinerary modified by the corrected information.

[0202] Furthermore, according to this embodiment, as shown in Figure 10, an example of a configuration is shown in which the first video can be played on the terminal even after the second video has been played on the terminal. According to this, even after the second video has been provided, the user of the device can rewatch the first video and easily compare the first and second videos.

[0203] Furthermore, according to this embodiment, the server controls the information displayed in the chat room shown on the terminal using a control unit, and sending the first video to the terminal includes an example of a configuration that includes performing control to display the first video in the chat room. According to this, users can easily access the first video within the chat room.

[0204] <First variation (1)> In the above embodiment, the video image preparation process involves searching for a video image appropriate to the scene based on the scene and corresponding itinerary elements and / or scene text, or generating a video image using a video generation model, but is not limited to this. For example, an open-book question answering system with RAG (Retrieval-Augmented Generation) applied may be used to search for / generate a video image based on a prompt sentence containing the scene and corresponding itinerary elements and / or scene text.

[0205] For example, to search for still images, an image search engine is used to find relevant images based on the user's query. For instance, if the query is "landscape photos," images related to landscape photos will be searched. To incorporate the searched images into the generation process, image caption generation and feature extraction techniques can be used to convert the image content into text, which can then be fed into the generative model.

[0206] For example, to search a video, a video search engine can be used to search for specific frames or scenes, and the search can be performed using the video's metadata and captions. Alternatively, specific scenes can be extracted from the video and converted into text, which can then be used in the RAG generation process. Similarly, the video's audio and subtitle data can also be used for searching.

[0207] Furthermore, RAG can be extended using multimodal image and language models such as CLIP (Contrastive Language-Image Pre-training) or BLIP (Bootstrapping Language-Image Pre-training). This would allow for linking text and images to perform search and generation.

[0208] <First variation (2)> In the above embodiment, an example was given in which one itinerary is presented as itinerary video information or itinerary information in response to a single itinerary generation request, but the invention is not limited to this. For example, instead of just one itinerary, multiple itinerary video information or itinerary information may be generated and presented so that the user can select any itinerary information. For example, the user may refer to multiple itinerary video information, and terminal 20A may output only the itinerary information corresponding to the selected itinerary video information.

[0209] Furthermore, the itinerary information may include information on transportation and other related matters. For example, in the detailed itinerary element search process, server 10 may be optimized to satisfy geographical and temporal constraints.

[0210] <First variation (3)> In the above embodiment, the server 10 is configured to perform the itinerary outline creation process, the itinerary element detail search process, and the itinerary video creation process, but it is not limited to this. For example, the processing may be performed in a server system consisting of a dialogue reception server 10L, a travel information server 10T, and a video generation server 10V.

[0211] For example, when the dialogue reception server 10L executes the itinerary generation request reception process and the itinerary summary creation process, it may send itinerary element detail query information to the travel information server 10T to create a more specific itinerary based on the itinerary summary information. The travel information server 10T may then execute the itinerary element detail search process based on the itinerary element detail query information. The travel information server 10T may then send itinerary element details, including the searched itinerary elements, to the dialogue reception server 10L.

[0212] When the dialogue reception server 10L receives detailed itinerary element information from the travel information server 10T, it may, for example, convert the detailed itinerary element information into a format that can be displayed and output, and then send the converted itinerary information to the terminal 20A. Alternatively, the dialogue reception server 10L may send the detailed itinerary element information to the video generation server 10V.

[0213] The video generation server 10V may perform a process to create a travel video based on the travel element details. It may then send the travel video information, including the generated travel video, to the dialogue reception server 10L. The dialogue reception server 10L may then send the travel video information to the terminal 20A.

[0214] <First variation (4)> In the above embodiment, the server 10 performs itinerary creation and itinerary video creation based on the receipt of itinerary creation request information, but is not limited to this. For example, the server 10 may perform the itinerary video creation process again based on an already created itinerary based on the receipt of itinerary video regeneration request information from the terminal 20. The server 10 may then send the regenerated itinerary video information to the terminal 20. This allows the user to create a first itinerary video and a second itinerary video from the first itinerary if they want to know more about the itinerary, or if they are satisfied with the itinerary itself but dissatisfied with the itinerary video. This allows the user to create and share itinerary videos that suit their preferences, for example, when sharing an itinerary video with friends.

[0215] According to this modified version, the server's control unit can generate a second video relating to the first journey based on the first journey, and transmit the second video to the terminal by the control unit. As a result, the terminal user can refer to a second video that is different from the first video generated from the first journey.

[0216] <First variation (5)> In the above embodiment, the video image preparation process involves searching for and generating video images appropriate to the scenes of the travel video based on the itinerary, but it is not limited to this. For example, in the video image preparation process, some or all of the scenes in the travel video may be used as advertising images.

[0217] For example, in the video image preparation process, server 10 may search for advertising images from an advertising server (not shown) based on the scene and the corresponding itinerary element and / or scene text, and select a matching advertising image as a scene. If an advertising image is inserted as a scene, server 10 may also composite text such as "This video contains advertisements" into the itinerary image during the video image editing process.

[0218] Figure 12 shows an example of the display screen in this modified example. On the left side of Figure 12, an advertisement scene for "XX Hotel" is displayed, which is embedded before the final scene of the itinerary video in the itinerary video information display area TVR1 is shown. On this screen, for example, an advertisement video for "XX Hotel" is displayed, selected based on the itinerary elements of the hotel that will appear in the next scene. In addition, the advertisement scene title display area CE1 is displayed instead of the video scene title display area to indicate that it is an advertisement. Below the advertisement scene title display area CE1, a detailed information display button CT1 is displayed so that the user can check detailed information about the advertisement.

[0219] For example, during the display of an advertisement scene, the share button SB1 and the map display button MB1 may be displayed in a grayed-out manner, effectively disabling their functionality. In other words, while viewing an advertisement scene, users may be prevented from skipping the advertisement to view or share their itinerary. Furthermore, during the display of an advertisement scene, the seek bar operation of the itinerary video information display area TVR1 may be disabled, thereby preventing users from skipping the advertisement scene.

[0220] For example, once the advertisement scene finishes playing, playback of the scene corresponding to the final plan of the itinerary in the center of Figure 12 begins. Then, as shown on the right side of Figure 12, the continuation of the talk room screen that is displayed when the itinerary video playback finishes in the itinerary video information display area TVR1 is shown. That is, in the talk room, the itinerary information CT2 is displayed after the itinerary video content TVC1, which is a thumbnail of the itinerary video information display area TVR1. In other words, the itinerary information is displayed based on the fact that the entire itinerary video, including the advertisement scene, has finished playing.

[0221] For example, if the user skips the advertisement scene using the seek bar on the left side of Figure 12, the itinerary information CT2 may not be displayed on the right side of Figure 12. In other words, viewing the itinerary information may require viewing the advertisement scene. In this case, for example, when the end of playback of the advertisement scene is detected, the control unit 21 of terminal 20 may send advertisement playback end information to the server 10. Then, when the control unit 11 of server 10 determines that it has received advertisement playback end information from terminal 20, it may send the itinerary information to terminal 20. In other words, when the advertisement scene ends, the itinerary information may be displayed without waiting for the entire itinerary video to finish playing.

[0222] This modified example shows a configuration in which the first video consists of multiple scenes, at least one of which is an advertisement video, and the control unit executes a control to display the first journey on the terminal after the advertisement video has been played on the terminal. According to this, since the first journey can be displayed on the terminal after the advertisement video has been played, the probability of viewing the advertisement video can be improved and the advertising effect can be enhanced.

[0223] Here, the number of ads (ad scenes) included in the first video is not limited to one; it may be multiple. For example, as mentioned above, if the number of sections (number of scenes) differs depending on the duration of the itinerary based on the generated itinerary information, the number of ads may be determined according to that number of sections. For example, the number of ads (e.g., 5) corresponding to the number of sections (e.g., 20) for a five-night, six-day itinerary may be greater than the number of ads (e.g., 1) corresponding to the number of sections (e.g., 4) for a one-night, two-day itinerary. Here, since the length of the generated itinerary video differs according to the determined number of sections (and the corresponding number of scenes), the control that determines the number of sections based on the duration of the itinerary is an example of the control that determines the number of ads based on the duration of the itinerary.

[0224] If the first video includes multiple advertisements (for example, 5), the itinerary information may be displayed only after all (5) of the advertisement scenes have finished playing, or itinerary information may be displayed only after a predetermined number (for example, 3) of the advertisement scenes have finished playing.

[0225] <Other> The above example describes a one-to-one interaction between a user and a chatbot in a messaging service, but it is not limited to this. For example, a chat room can be created in which multiple users and a chatbot participate (a chat room containing each user's account and the chatbot's account). In that case, the generated video and travel plan will be shared with multiple users simultaneously, making it highly convenient when planning a trip with multiple users.

[0226] Furthermore, even if a video and travel plan are generated through a one-on-one exchange, the video and travel plan may be made transferable to other chat rooms in which the user and their friends participate (other chat rooms containing each user's account). For example, by operating the share buttons SB1 or SB2 mentioned above, the video may be shared in another chat room specified by the user. Also, when an operation to share a video in another chat room is received, the travel plan may be (automatically) shared in the other chat room as well.

[0227] In other words, when the generated first video and first itinerary are displayed in the first chat room, in which the first user of the first terminal and the chatbot participate (the second user of the second terminal does not participate), the first video and first itinerary may be displayed (transferred) to the second chat room, in which the first user and the second user participate, based on the first user's transfer operation in the first chat room on the first terminal. At this time, the server 10 executes control to display the first video and first itinerary in the second chat room (sending them to the first and second terminals and displaying them on the second chat room) based on the first user's transfer operation. Furthermore, in the second chat room, which is the destination of the transfer, it may be possible to tap the displayed video or travel plan to move to the reservation screen and reserve a service facility, just as in the first chat room. Note that the second chat room may be a chat room of the same SNS (first SNS) as the first chat room, or it may be a chat room of a different SNS (second SNS) than the first chat room, as described below.

[0228] Furthermore, by operating the share buttons SB1 or SB2, the video and travel plan may be shared on a different SNS (SNS) than the messaging service provider (SNS) using the talk room (SNS) (SNS 1) as the destination. On the destination SNS (SNS 2), other users who have viewed the video and travel plan may be able to make reservations for the services (food and beverage services, accommodation services, etc.) included in the travel plan by tapping the displayed video or travel plan and entering the information for the reservation. In this case, on the source SNS (SNS 1), it is possible to tap the video or travel plan displayed on SNS 1 to move to the reservation screen associated with SNS 1 (for example, the reservation screen for SNS 1 linked to SNS 1), but on the destination SNS (SNS 2), it is possible to tap the video or travel plan displayed on SNS 2 to move to the reservation screen associated with SNS 2 (for example, the reservation screen for SNS 2 linked to SNS 2).

[0229] In other words, the control unit 11 of the server 10 may transmit to the first terminal information of a first reservation service linked to the first SNS that allows the first user to reserve service facilities included in a video or itinerary when input is made on the first SNS at the first terminal of the first user. The control unit 11 may then execute the service facility reservation process based on the input of the first user to the first reservation screen information. Furthermore, the control unit 11 of the server 10 may control the sharing of videos or itineraries shared on the first SNS to also be shared on the second SNS based on the transfer operation of the first user on the first SNS. For example, on the second SNS, videos or itineraries are published in association with the first user's account for the second SNS, and when the second user views these videos or itineraries and makes an input on their second terminal, the control unit 11 may send information on the second reservation screen of a second reservation service linked to the second SNS, which allows the user to reserve service facilities included in the videos or itineraries, to the second terminal. Based on the input of the second user on their second terminal to the second reservation screen information, the control unit 11 may then execute the service facility reservation process.

[0230] Furthermore, for multiple videos published on the first user's account on the second SNS, the second user viewing these videos may accept selection operations such as "like" or "dislike" (which may be implemented by means of a UI that allows selection via swipe gestures in the application, but is not limited to this), and based on the selection results, new videos and travel plans may be generated and associated with the second user's account on the second SNS, and the regenerated videos and travel plans may be shared. In addition, users who can view the videos and itineraries on the second SNS (including the first and second users) may be able to make reservations for service facilities through input operations on the regenerated videos and travel plans.

[0231] Furthermore, in the second SNS to which the content is forwarded, a second user, different from the first user who performed the forwarding operation, may modify the video or travel plan, thereby enabling the modified video or travel plan to be edited on the second SNS to which it is forwarded. For example, to enable this modification, a chatbot account may be added to the second chat room of the second SNS where the video or travel plan was shared, either automatically or with the permission of other users in that second chat room.

[0232] Furthermore, at least some of the processing that the server 10 is supposed to perform in the above embodiments may be performed by the terminal 20. Conversely, at least some of the processing that the terminal 20 is supposed to perform in the above embodiments may be performed by the server 10.

[0233] Furthermore, the operator of server 10 may also be an image generation service provider that partners with messaging service providers. In this case, the processing described in the above embodiment may be implemented by a single server, or the processing described in the above embodiment may be implemented by a server system consisting of multiple servers, with each server handling a portion of the processing.

[0234] For example, a system consisting of one or more servers may be defined as a server system, and the server of the present invention may be considered a server system. Furthermore, for example, the server system may consist of an information processing device, an image generation device, and a language generation device.

[0235] Furthermore, in the above embodiments, the server for distributing various applications (the server from which terminal 20 downloads applications) may be configured as a different server from the server that provides the corresponding services (applications). In other words, the server for distributing applications and the server that performs the application management processing described in the above embodiments may be configured as physically separate servers, or they may be configured as a single server.

[0236] In addition, the application may include not only programs for various applications, but also, for example, programs for providing functions of other services as one function of the underlying application (for example, a program for providing the function of an advertising distribution service as one function of a messaging application, or vice versa), programs for updating the underlying application, etc. Further, data utilized in the application program (which may also include data for updating the application, etc.) may be included.

[0237] Also, in the above-described embodiments and the like, for example, a method of implementing the present invention by a client-server system has been described, but the present invention is not limited thereto. As described above, it may be implemented by a system that gives the functions of a server or a server system, such as a distributed system, to the terminal 20. For example, the processing described as being performed by a server or a server system in the flowchart described in the above embodiment may be performed by the terminal.

[0238] Also, as described above, the contents described in each of the above embodiments, each modification, other embodiments, and the like can be applied in combination.

Description of Reference Numerals

[0239] 1 Communication system 10 Server 20 Terminal 30 Network

Claims

1. A program executed by a server that communicates with a terminal, The control unit of the server generates a first itinerary based on first information entered into the terminal by the user, Based on the first journey, the control unit generates a first video relating to the first journey, The control unit transmits the first video to the terminal, A program that includes this.

2. The program according to claim 1, The control unit transmits the first itinerary to the terminal. A program that includes this.

3. The program according to claim 2, After the first video is played on the terminal, the control unit executes a control to display the first itinerary on the terminal. A program that includes this.

4. The program according to claim 2, The aforementioned first video consists of multiple scenes, At least one of the aforementioned scenes is an advertisement video. After the advertisement video is played on the terminal, the control unit executes a control to display the first itinerary on the terminal. A program that includes this.

5. The program according to claim 1, The aforementioned first video consists of multiple scenes, The control unit generates text corresponding to the aforementioned scene, The control unit synthesizes the aforementioned text into the aforementioned scene, A program that includes this.

6. The program according to claim 1, The aforementioned first video consists of multiple scenes, The control unit generates text corresponding to the aforementioned scene, The control unit transmits the aforementioned text to the terminal, A program that includes this.

7. The program according to claim 1, Based on the first itinerary, the control unit generates the background music for the first video, The first video and the BGM are combined by the control unit, The control unit transmits the first video, on which the BGM has been synthesized, to the terminal. A program that includes this.

8. The program according to claim 1, The control unit generates a second journey based on the correction information for the first information input into the terminal by the user, Based on the second journey, the control unit generates a second video relating to the second journey, The control unit transmits the second video to the terminal, A program that includes this.

9. The program according to claim 1, Based on the first journey, the control unit generates a second video relating to the first journey, The control unit transmits the second video to the terminal, A program that includes this.

10. A program according to claim 8 or claim 9, Even after the second video has been played on the terminal, the first video can still be played on the terminal. program.

11. The program according to claim 1, The server controls the information displayed in the talk room shown on the terminal using the control unit. Transmitting the first video to the terminal includes performing control to display the first video in the talk room. program.

12. A method for processing information on a server that communicates with a terminal, The control unit of the server generates a first itinerary based on first information entered into the terminal by the user, Based on the first journey, the control unit generates a first video relating to the first journey, Transmitting the first video to the terminal, Information processing methods, including those mentioned above.

13. A server that communicates with a terminal, A server comprising a control unit that (i) generates a first itinerary based on first information input to the terminal by a user, (ii) generates a first video relating to the first itinerary based on the first itinerary, and (iii) transmits the first video to the terminal.

Images