Information processing device, information processing method, and program

Abstract

To provide an information processing system, an information processing device, an information processing method, and a program that perform a display with various information added thereto when reproducing a movement trajectory.SOLUTION: A wrist terminal 10 includes a structured data generation unit 56. The structured data generation unit 56 generates structured data that defines display of a moving image that traces a movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information about the moving image tracing the movement trajectory, and movement trajectory information representing the movement trajectory. The movement trajectory is a movement trajectory corresponding to the user's history of predetermined action. The structured data generation unit 56 generates the structured data so as to have the display control information display the moving image, according to contents of the user's predetermined action.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention provides Information processing device, information processing method, and programs. [Background technology]

[0002] A technology has been disclosed in which location information is detected using a global positioning system (GPS), a moving object is displayed on a user's movement trajectory based on the location information and time information of a captured image, and an image of the object is displayed when the moving object reaches the display position of the captured image (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-69313 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technology, a moving object is displayed on the user's movement trajectory based on the position information of the captured image taken by the user, and the target image is displayed when the moving object reaches the display position of the captured image, but the target image is displayed in a uniform display format, resulting in a monotonous display.

[0005] The present invention has been made in view of the above circumstances, and has as its object to display a variety of additional information when reproducing a movement trajectory. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to one aspect of the present invention comprises: The system includes a structured data generating means for generating structured data for displaying a track of the movement history and content related to the movement history within a map display provided as map information, based on data of a history of activity including a user's movement history and the type of the activity, wherein the structured data defines a direction in which the line of sight is directed when displaying a point of interest tracing the track of the movement history, depending on the type of the activity. The structured data generation means determines the type of activity performed by the user based on data on the type of activity included in the activity history data, sets attributes of the video according to the type of activity, and generates the structured data based on the attributes of the video according to the type of activity. It is characterized by: [Effects of the Invention]

[0007] According to the present invention, when a movement trajectory is played back, it is possible to display the movement trajectory with various pieces of information added thereto. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating a system configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Figure 3] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Figure 4] 3 is a functional block diagram showing the functional configuration for executing history recording processing and structured data generation processing, among the functional configurations of the wrist terminal configured by the information processing device of FIG. 2. FIG. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a video attribute table. [Figure 6] FIG. 10 is a schematic diagram illustrating an example of structured data for outputting position information. [Figure 7] 4 is a functional block diagram showing a functional configuration for executing structured data analysis processing, among the functional configurations of the map information providing server configured by the information processing device of FIG. 3. FIG. [Figure 8] 4 is a functional block diagram showing a functional configuration for executing storage service processing, among the functional configurations of a cloud storage server configured by the information processing device of FIG. 3. FIG. [Figure 9] 4 is a functional block diagram showing a functional configuration for executing video playback processing, among the functional configurations of a PC configured by the information processing device of FIG. 3. FIG. [Figure 10] FIG. 10 is a schematic diagram illustrating an example of activity history data. [Figure 11] 11 is a schematic diagram showing a display example in which the data of the activity history shown in FIG. 10 is converted into a video. FIG. [Figure 12] FIG. 10 is a schematic diagram showing the operation of a photography event at position P4 where the photograph was taken. [Figure 13] FIG. 10 is a schematic diagram showing a state in which an image of a photograph taken at position P4 is displayed in a pop-up. [Figure 14] 5 is a flowchart illustrating the flow of a history recording process executed by a wrist terminal having the functional configuration of FIG. 4. [Figure 15] 5 is a flowchart illustrating the flow of a structured data generation process executed by a wrist terminal having the functional configuration of FIG. 4. [Figure 16] 8 is a flowchart illustrating the flow of a structured data analysis process executed by a map information providing server having the functional configuration of FIG. 7. [Figure 17] 9 is a flowchart illustrating the flow of a storage service process executed by a cloud storage server having the functional configuration of FIG. 8. [Figure 18] 10 is a flowchart illustrating the flow of a video playback process executed by a PC having the functional configuration of FIG. 9. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] FIG. 1 is a schematic diagram showing the system configuration of an information processing system 1 according to one embodiment of the present invention. As shown in FIG. 1, the information processing system 1 includes a list terminal 10, a photographing device 20, a smartphone 30, a map information providing server 40, a cloud storage server 50, and a PC (Personal Computer) 60. Among these, the list terminal 10, the photographing device 20, and the smartphone 30 are configured to be able to communicate with each other by wireless communication such as Bluetooth low energy / Bluetooth LE (registered trademark) (hereinafter referred to as "BLE") or Wi-Fi (Wireless Fidelity). Also, the smartphone 30, the map information providing server 40, the cloud storage server 50, and the PC 60 are configured to be able to communicate with each other via a network 70 such as the Internet. Note that in the present embodiment, the list terminal 10, the photographing device 20, and the smartphone 30 may also be provided with a function of communicating with other devices via a network 70 such as the Internet.

[0011] Hereinafter, an outline of each device constituting the information processing system 1 will be described. The list terminal 10 is composed of a wristwatch-type information processing device (so-called smartwatch) and is worn on the wrist of a user who performs activities such as mountain climbing or road biking (cycling). Further, the list terminal 10 has a positioning function by GPS, and performs a process of recording the history of activities including the movement history of the user (hereinafter referred to as "history recording process"), and a structured data described for displaying position information and content related to the position information based on the recorded history data (hereinafter referred to as "structured data for position information output"). (Hereinafter referred to as "structured data generation process"). For example, when the user wearing the list terminal 10 performs various activities by executing the history recording process, the list terminal 10 records the data of the history of activities including the movement history in association with the type of activity. Further, the list terminal 10 automatically generates structured data for position information output described for displaying position information and content related to the position information based on the recorded history data by executing the structured data generation process. Then, the list terminal 10 uploads the generated structured data for position information output to the cloud storage server 50 via the smartphone 30.

[0012] The photographing device 20 is composed of an information processing device having a photographing function such as a digital camera. The photographing device 20 photographs an image (still image or moving image) according to a photographing instruction of the user, and stores the data of the photographed image in association with the photographing date and time. In the present embodiment, the photographing device 20 automatically uploads the data of the image associated with the photographing date and time to the cloud storage server 50 via the smartphone 30 according to the instruction of the user or according to the setting.

[0013] The smartphone 30 is composed of an information processing device having a call function and an information processing function. The smartphone 30 uploads the structured data for position information output transmitted from the list terminal 10, the data of the image transmitted from the photographing device 20, etc. to the cloud storage server 50 via the network 70.

[0014] The map information providing server 40 is configured by an information processing apparatus such as a server computer or a PC. The map information providing server 40 has a three-dimensional map information providing function, interprets the structured data for position information output, and displays map information and related content while moving the positions described in the structured data for position information output in time series (hereinafter referred to as "structured data analysis process"). In the present embodiment, the structured data for position information output describes the position (latitude, longitude, altitude) of the point of interest, the moving speed of the viewpoint, the altitude of the viewpoint, the azimuth of the viewpoint (the direction in which the line of sight is directed), the horizontal angle (depression angle) of the viewpoint, and the attributes of the video such as the actions at various events. The map information providing server 40 outputs the data of the video that traces the movement history while changing the position in the map information or performing a predetermined action according to the described video attributes. Note that when the map information providing server 40 executes the structured data analysis process, it acquires content such as the captured image and the text of the spoken comment from the storage destination of the content described in the structured data for position information output (here, the cloud storage server 50), and displays the acquired content at a predetermined position in the movement history.

[0015] The cloud storage server 50 is configured by an information processing apparatus such as a server computer or a PC. The cloud storage server 50 is a server that provides a cloud storage service, and performs a process of associating and storing the structured data for position information output and content (image, voice, or text, etc.) uploaded by the user with the identification information of the user, and transmitting it to other devices in response to a request (hereinafter referred to as "storage service process").

[0016] The PC 60 is an information processing device used by a user, and executes a process for playing a video based on the structured data for outputting location information (hereinafter referred to as "video playback process") in response to a user instruction. For example, the PC 60 transmits a request for playing a video together with the structured data for outputting location information to the map information providing server 40. The PC 60 also receives and displays video data that is the result of the structured data analysis process executed by the map information providing server 40.

[0017] [Hardware configuration of information processing device] Next, the hardware configuration of each device that constitutes the information processing system 1 will be described. In the present embodiment, as an example, the wrist terminal 10, the photographing device 20, and the smartphone 30 constituting the information processing system 1 are configured as portable information processing devices 100, and the map information providing server 40, the cloud storage server 50, and the PC 60 are configured as stationary information processing devices 200. Below, the hardware configurations of the information processing devices 100 constituting the wrist terminal 10, the photographing device 20, and the smartphone 30, and the information processing devices 200 constituting the map information providing server 40, the cloud storage server 50, and the PC 60 will be described. Note that the hardware configurations of the information processing devices 100 and the information processing devices 200 can be changed as appropriate depending on the specific devices in which they are implemented.

[0018] FIG. 2 is a block diagram showing the hardware configuration of the information processing device 100. As shown in FIG. As shown in FIG. 2, the information processing device 100 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a bus 114, an input / output interface 115, a GPS receiving unit 116, an imaging unit 117, a sensor unit 118, an input unit 119, an output unit 120, a memory unit 121, a communication unit 122, and a drive 123.

[0019] The CPU 111 executes various processes according to a program recorded in the ROM 112 or a program loaded from the storage unit 121 to the RAM 113 .

[0020] The RAM 113 also stores data and the like necessary for the CPU 111 to execute various processes.

[0021] The CPU 111, ROM 112, and RAM 113 are connected to one another via a bus 114. An input / output interface 115 is also connected to this bus 114. A GPS receiving unit 116, an imaging unit 117, a sensor unit 118, an input unit 119, an output unit 120, a storage unit 121, a communication unit 122, and a drive 123 are connected to the input / output interface 115.

[0022] The GPS receiving unit 116 detects the position (latitude, longitude, altitude) of the information processing device 100 and the current time indicated by the GPS based on the GPS signal received from the GPS satellite.

[0023] Although not shown, the imaging unit 117 includes an optical lens unit and an image sensor.

[0024] The optical lens unit is made up of lenses that condense light to photograph a subject, such as a focus lens and a zoom lens. A focus lens is a lens that focuses a subject image on the light-receiving surface of an image sensor, while a zoom lens is a lens that can freely change its focal length within a certain range. The imaging unit 117 is also provided with peripheral circuits for adjusting setting parameters such as focus, exposure, and white balance as needed.

[0025] The image sensor is composed of a photoelectric conversion element, an AFE (Analog Front End), and the like. The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. An object image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (images) the object image, accumulates the image signal for a certain period of time, and sequentially supplies the accumulated image signal to the AFE as an analog signal. The AFE performs various signal processes such as A / D (Analog / Digital) conversion process on this analog image signal. By the various signal processes, a digital signal is generated and appropriately supplied to the CPU 111, an image processing unit (not shown), etc. as the output signal of the imaging unit 117.

[0026] The sensor unit 118 includes various sensors such as a barometric pressure sensor, a temperature sensor, a humidity sensor, a wind speed sensor, a biological sensor (pulse sensor, blood pressure sensor, etc.), a geomagnetic sensor, a three-axis acceleration sensor, and a three-axis angular velocity sensor. These various sensors provided as the sensor unit 118 can be in either an implementation form built into the information processing apparatus 100 or an implementation form connected to an external unit.

[0027] The input unit 119 is composed of various buttons etc. and inputs various information according to the instruction operations of the user. The output unit 120 is composed of a display, a speaker, etc. and outputs images and sounds. In this embodiment, the input unit 119 of the information processing apparatus 100 includes a touch sensor, and a touch panel is configured by arranging this touch sensor overlapping with the display of the output unit 120. The storage unit 121 is composed of a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and stores various data. The communication unit 122 controls the communication with other devices via a network including the Internet. Also, the communication unit 122 controls the communication with other devices by direct wireless communication between terminals.

[0028] A removable medium 131 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 123. The program read from the removable medium 131 by the drive 123 is installed in the storage unit 121 as necessary. Further, the removable medium 131 can also store various data stored in the storage unit 121 in the same manner as the storage unit 121.

[0029] Figure 3 is a block diagram showing the hardware configuration of the information processing apparatus 200. As shown in Figure 3, the information processing apparatus 200 includes a CPU 211, a ROM 212, a RAM 213, a bus 214, an input / output interface 215, an input unit 216, an output unit 217, a storage unit 218, a communication unit 219, and a drive 220. In the information processing apparatus 200 shown in Figure 3, the hardware configuration of each part is the same as the corresponding hardware configuration of the information processing apparatus 100 shown in Figure 2. However, in this embodiment, the communication unit 219 can omit the function of communicating with other devices by direct wireless communication between terminals. Further, a removable medium 231 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 220 in the same manner as the drive 123.

[0030] [Functional Configuration] Next, the functional configuration of the information processing system 1 will be described. Figure 4 is a functional block diagram showing the functional configuration for executing the history recording process and the structured data generation process among the functional configurations of the list terminal 10 configured by the information processing apparatus 100 of Figure 2. The history recording process is a series of processes for recording the history of activities including the movement history of the user. Further, the structured data generation process is a series of processes for generating structured data for position information output described for displaying position information and content related to the position information based on the recorded history data.

[0031] When the history recording process and the structured data generation process are executed, as shown in FIG. 4, a user interface display control unit (UI display control unit) 51, a history acquisition unit 52, an activity determination unit 53, a video attribute setting unit 54, a parameter correction receiving unit 55, a structured data generation unit 56, and a structured data registration unit 57 function in the CPU 111. In addition, a history data storage unit 71 is set in one area of the storage unit 121. The history data storage unit 71 stores data on the history of activities including the movement history of the user.

[0032] The UI display control unit 51 displays a user interface screen on the wrist terminal 10 for the user to input and output information. The UI display control unit 51 also accepts various operations via the user interface screen. For example, when the user performs an operation to start an activity, the UI display control unit 51 displays a list of multiple types of activity candidates and accepts the user's selection of an activity. When the user inputs an instruction to generate structured data for position information output from data of the activity history, the UI display control unit 51 also accepts input of content to be manually associated with the data of the activity history (such as input of the address of a storage area). The history acquisition unit 52 sequentially acquires the position (latitude, longitude, altitude) and the current time detected by the GPS receiving unit 116, and stores them in the history data storage unit 71 as the user's movement history.

[0033] The activity determination unit 53 determines the type of activity performed by the user. In this embodiment, the activity determination unit 53 determines the type of activity performed by the user based on data on the activity type included in the activity history data. Note that the activity determination unit 53 can also determine the type of activity performed by the user by detecting the type of activity selected by the user from a list of activity candidates via the user interface screen.

[0034] The video attribute setting unit 54 sets video attributes (position of the point of interest (latitude, longitude, altitude), viewpoint movement speed, viewpoint altitude, viewpoint direction (direction of line of sight), viewpoint horizontal angle (depression angle), actions during various events, etc.) according to the type of activity performed by the user. In this embodiment, the type of activity and the video attributes corresponding to that type of activity are set in advance as table-format data (hereinafter referred to as a "video attribute table"). The video attribute setting unit 54 then sets the video attributes by reading the video attributes corresponding to the type of activity from the video attribute table.

[0035] FIG. 5 is a schematic diagram showing an example of a video attribute table. As shown in FIG. 5, the video attribute table stores the types of activities and the attributes of the videos corresponding to the types of activities in association with each other. In this embodiment, mountain climbing, road biking (cycling), skiing / snowboarding, paddleboarding, surfing, and off-road biking are defined as types of activities.

[0036] As a general example, when climbing mountains, the speed is 1-3 km / h, the altitude is 0-8000 m, and the distance traveled is about 15 km. When riding a road bike, the speed is 10-60 km / h, and the distance traveled is long (about 20 km or more). When skiing / snowboarding, the speed is 20-50 km / h, and the distance traveled per trip is short (about 10 km or less). When paddleboarding, the speed is 2-10 km / h, and the distance traveled is indefinite. When surfing, the speed is 10-40 km / h, and the distance traveled per trip is about 100 m or less. When riding an off-road bike, the speed is 20-100 km / h, and the distance traveled is indefinite.

[0037] In view of these features, in this embodiment, as attributes of the video, the position of the point of interest (latitude, longitude, altitude), the moving speed of the viewpoint, the altitude of the viewpoint, the azimuth of the viewpoint (the direction in which the line of sight points), the horizontal angle of the viewpoint (depression angle), the actions at various events (for example, whether to stop temporarily at a specific location, zoom up or plummet in the case of a temporary stop, or the stop time in the case of a temporary stop), etc. are included, and numerical values and actions corresponding to the types of activities are defined.

[0038] For example, regarding the position of the point of interest, as the shooting target position of the virtual camera in the video, each position in the movement history of the activity or a selected specific position can be defined. Regarding the moving speed of the viewpoint, a moving speed that is a constant multiple of the actual moving speed in the activity or a selected specific speed can be defined, or it can be defined as the speed of moving the entire process in a preset time (for example, 3 minutes, etc.). Regarding the altitude of the viewpoint, in the case of mountain climbing or trekking, the height of the viewpoint that can overlook a wide range of the movement range can be defined, and in the case of road cycling (cycling), the height of the viewpoint that can express the sense of speed of the bicycle (that is, a lower viewpoint height than mountain climbing or trekking) can be defined.

[0039] Regarding the azimuth of the viewpoint, according to the type of activity, the azimuth in which the line of sight (the shooting direction of the virtual camera in the video) points when displaying the point of interest can be defined. Regarding the horizontal angle of the viewpoint, a depression angle corresponding to the type of activity can be defined. For example, in the case of mountain climbing or trekking, a relatively large depression angle (an angle close to looking straight down) can be defined, and in the case of road cycling (cycling), a relatively small depression angle (an angle close to looking in the traveling direction) can be defined.

[0040] Regarding the operations during various events, for example, the operations at the location where a photo is taken: if it is mountain climbing or trekking, the movement is temporarily stopped for a relatively long time (e.g., 3 seconds, etc.), and after a temporary zoom-in operation at the shooting location, the image of the taken photo is displayed in a pop-up manner; if it is a road bike, the movement is temporarily stopped for a relatively short time (e.g., 1.5 seconds), and the image of the taken photo can be displayed in a pop-up manner without zooming in at the shooting location. Also, for example, regarding the operations at the location where the user inputs speech (inputs voice to the list terminal 10): if it is mountain climbing or trekking, the movement speed is decreased for a relatively long time (e.g., 3 seconds, etc.), and the text data of the spoken voice is displayed in a balloon form; if it is a road bike, the movement speed is decreased for a relatively short time (e.g., 1.5 seconds), and the text data of the spoken voice can be displayed in a balloon form. In addition, regarding the start point, the goal point, or comments added by the user to specific locations of the history data after manually operating to end the activity, etc., a text box including a marker and comments regarding that location can be displayed in a pop-up manner, and the display form can be made different according to the type of activity.

[0041] The parameter correction reception unit 55 receives the user's correction operation for the parameters representing the attributes of the video set by the video attribute setting unit 54. For example, the parameter correction reception unit 55 changes the parameters according to the user's operation for the movement speed, the altitude of the viewpoint, or the horizontal angle of the viewpoint, etc. set by the video attribute setting unit 54.

[0042] When an instruction to generate structured data for outputting location information is input for data of a user's activity history, the structured data generation unit 56 generates structured data for outputting location information for displaying data of the movement history (location information) of the activity and content related to the movement history, based on the attributes of the video set in the data of the activity history. In this embodiment, the content related to the movement history is content that the user has manually associated by inputting the address of a storage area, etc.

[0043] FIG. 6 is a schematic diagram showing an example of structured data for outputting position information. As shown in Figure 6, structured data for outputting location information is configured as a structured document whose structure is defined by tags, and various tags are defined for outputting location information. Such a structured document can be written in, for example, XML (eXtensible Markup Language) format, and various tags that are effective for outputting location information can be defined, such as tags that set the viewpoint and tags that define actions at the time of an event. In this embodiment, the structured data for outputting location information is made up of multiple blocks grouped by function, as shown in Fig. 6. As an example, the structured data for outputting location information includes a block that defines various styles, a block that defines the drawing of trajectory lines, a block that defines the drawing of markers, a block that defines the display of videos, etc.

[0044] The blocks that define various styles define the styles of various elements in the display of the video, such as the URL where the image of the marker as an element in the display of the video is stored, the size of the marker, or the transparency of the marker. The block that defines the drawing of the trajectory line defines the drawing of the trajectory displayed in the video, and defines multiple coordinates that represent the user's movement history consecutively in chronological order of the movement. The block that defines the drawing of the trajectory line also defines the line width and color of the trajectory.

[0045] The block that defines the drawing of markers defines the drawing of markers to be displayed on the trajectory, and defines the markers to be added to the trajectory, text data of the spoken voice, information about the photo, information about the start position and goal position, etc., the display position on the movement trajectory and the corresponding time (time the photo was taken, etc.), the style to be applied, the URL where the actual content to be drawn (photo, etc.) is stored, etc.

[0046] The block that defines the video display defines the video presentation, and defines the operation of displaying or erasing markers in the video (operations including various presentations), the position of the point of interest (latitude, longitude, altitude), the viewpoint movement speed, viewpoint altitude, viewpoint horizontal angle, and altitude reference (altitude above sea level, etc.). When displaying the video, the view (map information) of the position of the point of interest as seen from the defined viewpoint altitude, horizontal angle, etc. is displayed while moving sequentially to each defined viewpoint position at the defined viewpoint speed.

[0047] The structured data registration unit 57 uploads the structured data for outputting location information generated by the structured data generation unit 56 to the cloud storage server 50 via the smartphone 30.

[0048] FIG. 7 is a functional block diagram showing the functional configuration for executing the structured data analysis process, among the functional configurations of the map information providing server 40 configured by the information processing device 200 of FIG. Structured data analysis processing is a series of processes that interpret structured data for outputting location information and display map information and related content, etc., while moving the viewpoint position described in the structured data for outputting location information in chronological order.

[0049] When the structured data analysis process is executed, as shown in FIG. 7, in CPU 211, structured data receiving section 151, structured data executing section 152, and content obtaining section 153 function.

[0050] The structured data receiving unit 151 receives structured data for outputting position information to be processed from the PC 60. The structured data execution unit 152 interprets the structured data for outputting position information received by the structured data receiving unit 151, and identifies the addresses of related content (such as the URL (Uniform Resource Locator) of the storage area in the cloud storage server 50). The structured data execution unit 152 acquires the related content from the identified addresses via the content acquisition unit 153. Then, the structured data execution unit 152 executes the structured data for outputting position information, and generates data of a video for displaying map information, related content, etc. while moving the position of the viewpoint described in the structured data for outputting position information in time series. At this time, since the structured data for outputting position information describes the attributes of the video according to the type of activity, the generated video data becomes a video having an appropriate display form according to the activity. Then, the structured data execution unit 152 transmits the generated video data to the PC 60 that transmitted the structured data for outputting position information. The content acquisition unit 153 acquires various contents from the addresses of the related content identified by the structured data execution unit 152 interpreting the structured data for outputting position information.

[0051] FIG. 8 is a functional block diagram showing a functional configuration for executing storage service processing among the functional configurations of the cloud storage server 50 configured by the information processing apparatus 200 in FIG. 3. The storage service processing is a series of processes that associate and store various data uploaded by the user, such as structured data for outputting position information transmitted from the list terminal 10 and image data transmitted from the imaging device 20, with the user, and provide the stored various data in response to requests from other devices such as the map information providing server 40.

[0052] When the storage service process is executed, as shown in FIG. 8, in the CPU 211, a structured data receiving unit 251, a structured data transmitting unit 252, a content receiving unit 253, a content transmitting unit 254, and a data management unit 255 function. Also, in an area of the storage unit 218, a structured data storage unit 271 and a related content storage unit 272 are set. In the structured data storage unit 271, the structured data for position information output uploaded by the user is stored in association with the identification information of the user. In the related content storage unit 272, the data of various contents uploaded by the user is stored in association with the identification information of the user.

[0053] The structured data receiving unit 251 receives the structured data for position information output transmitted from another device such as the smartphone 30, and outputs the received structured data for position information output to the data management unit 255 together with the identification information of the transmitting user. When the structured data transmitting unit 252 receives a transmission request for the structured data for position information output from another device such as the smartphone 30, it outputs the received transmission request to the data management unit 255. The transmission request for the structured data includes the identification information of the user and the identification information of the structured data for which transmission is requested. Then, when the structured data for position information output corresponding to the transmission request is input from the data management unit 255, the structured data transmitting unit 252 transmits the input structured data for position information output to the device that requested the transmission.

[0054] The content receiving unit 253 receives the content transmitted from another device such as the smartphone 30, and outputs the received content to the data management unit 255 together with the identification information of the transmitting user. When the content transmission unit 254 receives a content transmission request from another device such as the PC 60, it outputs the received transmission request to the data management unit 255. The content transmission request includes the user's identification information and the identification information of the content for which transmission is requested. Then, when the structured data transmission unit 252 receives the content corresponding to the transmission request from the data management unit 255, it transmits the input content to the device that is the source of the transmission request.

[0055] When the data management unit 255 receives the structured data for position information output and the identification information of the transmitted user from the structured data reception unit 251, it associates the structured data for position information output with the user's identification information and stores it in the structured data storage unit 271. Also, when the data management unit 255 receives a transmission request for the structured data for position information output from the structured data transmission unit 252, it reads out the structured data for position information output associated with the identification information of the user indicated in the transmission request from the structured data storage unit 271 and outputs it to the structured data transmission unit 252.

[0056] Also, when the data management unit 255 receives the content and the identification information of the transmitted user from the content reception unit 253, it associates the content with the user's identification information and stores it in the related content storage unit 272. Also, when the data management unit 255 receives a content transmission request from the content transmission unit 254, it reads out the content associated with the identification information of the user indicated in the transmission request from the related content storage unit 272 and outputs it to the content transmission unit 254.

[0057] FIG. 9 is a functional block diagram showing the functional configuration for executing video playback processing among the functional configurations of the PC 60 constituted by the information processing apparatus 200 of FIG. 3. The video playback processing is a series of processes for playing back a video based on the structured data for position information output in response to a user's instruction.

[0058] When video playback processing is executed, as shown in FIG. 9, in the CPU 211, the structured data acquisition unit 351, the video playback request unit 352, and the video playback unit 353 function. Also, a structured data storage unit 371 is set in an area of the storage unit 218. The structured data storage unit 371 stores structured data for outputting position information for the video to be played back.

[0059] The structured data acquisition unit 351 transmits a transmission request for the structured data for outputting position information, which is instructed by the user to play back the video, to the cloud storage server 50. Then, the structured data acquisition unit 351 receives the structured data for outputting position information corresponding to the transmission request from the cloud storage server 50, and stores the received structured data for outputting position information in the structured data storage unit 371.

[0060] The video playback request unit 352 transmits a video playback request to the map information providing server 40 together with the structured data for outputting position information, which is instructed by the user to play back the video. The video playback unit 353 receives the video data based on the structured data for outputting position information transmitted from the map information providing server 40, and plays it back on the display of the output unit 217.

[0061] [Specific display example] FIG. 10 is a schematic diagram showing an example of the data of the activity history. In the example shown in FIG. 10, the entire data of the history when mountain climbing is performed as an activity is shown, and various markers are added to the positions P1 to P6 in the trajectory of the movement history. Note that the thick dashed line in FIG. 10 represents the trajectory in the movement history (the same applies to FIGS. 11 to 13). For example, the marker at position P1 represents the starting position, and displays the text "Start of climbing 10:40:00." The marker at position P2 represents the text of a comment added by the user, and displays the text "Comment 10:46:45." The marker at position P3 represents the text obtained by speech recognition of a comment input by the user, and displays the text "Speech 12:36:54." The marker at position P4 represents the position where a photo was taken, and displays the text "Photo 13:04:22." The marker at position P5 represents the position where a photo was taken, and displays the text "Photo 13:48:04." The marker at position P6 represents the finish position, and displays the text "Finish of climbing 14:28:47."

[0062] Such activity history data is converted by the wrist terminal 10 into structured data for outputting location information, in which video attributes are set according to the type of activity, and is interpreted and executed by the map information providing server 40 to produce a video in an appropriate display format according to the type of activity.

[0063] FIG. 11 is a schematic diagram showing a display example in which the data of the activity history shown in FIG. 10 is converted into a moving image. In Figure 11, the dotted circle indicates the position of the point of interest. Note that the circle indicating the point of interest is shown to indicate the position of the point of interest schematically, but is not displayed in the actual video in this embodiment. However, the position of the point of interest may be explicitly indicated by displaying a moving icon or the like.

[0064] As shown in Figure 11, when the data of the mountain climbing history shown in Figure 10 is made into a video, the position of the focus point is moved sequentially from the starting position, and the display format is such that a viewpoint set at a high altitude that overlooks a relatively wide range traces the movement history. Then, when the focus point reaches the position where the event in the movement history trajectory is set, the operation of the set event is executed.

[0065] FIG. 12 is a schematic diagram showing the operation of the event of taking a photo at the position P4 where the photo was taken. In FIG. 12, when the focus point reaches the position P4 where the photo was taken, the movement of the viewpoint and the focus point is temporarily stopped for a relatively long time (for example, 3 seconds, etc.). Then, after temporarily zooming in on the shooting location (position P4), the image of the taken photo is displayed in a pop-up manner.

[0066] FIG. 13 is a schematic diagram showing a state where the image of the taken photo is displayed in a pop-up manner at the position P4. In addition, in the example shown in FIG. 13, text indicating the comment input by the user is displayed together with the taken photo.

[0067] As shown in FIG. 13, when the focus point reaches the position P4 where the photo was taken, after the movement of the viewpoint and the focus point is temporarily stopped for a relatively long time (for example, 3 seconds, etc.), it is zoomed in on the shooting location, and an enlarged image of the vicinity of the shooting location is displayed. Then, a pop-up including the photo associated with the position P4 is displayed for a predetermined time. After that, the display of the pop-up is erased, and it returns to the position of the original viewpoint from the zoomed-in state, and the display of the video continues.

[0068] In the example shown in FIG. 13, the pop-up including the photo is displayed so as not to overlap with the movement history trajectory. Therefore, it is possible to easily grasp which position in the movement history trajectory the corresponding photo is. In addition, the photo associated with the position P4 may be displayed in a larger size or the entire screen. In this case, the photo associated with the position P4 can be displayed in a more visible form.

[0069] [Operation] Next, the operation of the information processing system 1 will be described. [History Recording Process] Figure 14 is a flowchart for explaining the flow of the history recording process executed by the list terminal 10 having the functional configuration shown in FIG. 4. The history recording process starts when an operation for instructing the start of the history recording process is performed via the input unit 119.

[0070] In step S1, the UI display control unit 51 displays a screen of a user interface for the user to input and output information. In step S2, the UI display control unit 51 accepts input of the type of activity that is the target of the history recording process. At this time, the UI display control unit 51 displays a list of candidates for a plurality of types of activities and accepts selection of an activity by the user.

[0071] In step S3, the history acquisition unit 52 acquires the position (latitude, longitude, altitude) detected by the GPS reception unit 116 and the current time. In step S4, the history acquisition unit 52 determines whether or not the end of the history recording process has been instructed. If the end of the history recording process has not been instructed, it is determined as NO in step S4, and the process proceeds to step S3. On the other hand, if the end of the history recording process has been instructed, it is determined as YES in step S4, and the process proceeds to step S5.

[0072] In step S5, the history acquisition unit 52 associates the acquired series of position data with the time when it was acquired, and stores it in the history data storage unit 71 as the movement history of the user. Note that the movement history of the user stored at this time includes data on the type of activity received in step S2. After step S5, the history recording process ends.

[0073] [Structured Data Generation Process] Figure 15 is a flowchart for explaining the flow of the structured data generation process executed by the list terminal 10 having the functional configuration shown in FIG. 4. The structured data generation process is started when an operation is performed via input unit 119 to instruct the start of the structured data generation process.

[0074] In step S11, the UI display control unit 51 accepts the selection of data on the history of the activity that is the target of the structured data generation process. In step S12, the activity determination unit 53 determines the type of activity performed by the user based on the data on the type of activity included in the activity history data.

[0075] In step S13, the video attribute setting unit 54 sets video attributes (position of the point of interest (latitude, longitude, altitude), viewpoint movement speed, viewpoint altitude, viewpoint direction (direction of line of sight), viewpoint horizontal angle (depression angle), actions during various events, etc.) according to the type of activity performed by the user. At this time, the video attribute setting unit 54 sets the video attributes by reading the video attributes corresponding to the activity type from the video attribute table. In step S14, the parameter correction receiving unit 55 receives a user's correction operation for the parameters representing the attributes of the moving image set by the moving image attribute setting unit .

[0076] In step S15, the structured data generation unit 56 generates structured data for outputting location information based on the video attributes set in the activity history data, for displaying data on the movement history (location information) of the activity and content related to that movement history. In step S16, the structured data registration unit 57 uploads the structured data for outputting location information generated by the structured data generation unit 56 to the cloud storage server 50 via the smartphone 30. After step S16, the structured data analysis process ends.

[0077] [Structured data analysis processing] FIG. 16 is a flowchart illustrating the flow of structured data analysis processing executed by the map information providing server 40 having the functional configuration of FIG. The structured data analysis process is started when an operation to instruct the start of the structured data analysis process is performed via the input unit 216 or the communication unit 219 .

[0078] In step S21, the structured data receiving unit 151 receives the structured data for outputting position information to be processed from the PC 60 together with a request to play back a moving image. In step S22, the structured data execution unit 152 interprets the structured data for outputting location information received by the structured data reception unit 151. As a result, the address of the related content (such as the URL of the storage area in the cloud storage server 50) is identified.

[0079] In step S23, the content acquisition unit 153 acquires various contents from the addresses of the related contents identified by the structured data execution unit 152 by interpreting the structured data for outputting location information, and outputs the acquired contents to the structured data execution unit 152. In step S24, the structured data execution unit 152 executes the structured data for outputting location information, thereby generating video data for displaying map information and related content while moving the viewpoint position described in the structured data for outputting location information in chronological order.

[0080] In step S25, the structured data execution unit 152 transmits the generated video data to the PC 60, which is the source of the structured data for outputting position information. When the structured data execution unit 152 transmits video data to the PC 60, it is possible to transmit the entire generated video data all at once, or to divide the video data into small partial data and transmit them. After step S25, the structured data analysis process is repeated.

[0081] [Storage service processing] FIG. 17 is a flowchart illustrating the flow of storage service processing executed by the cloud storage server 50 having the functional configuration of FIG. The storage service process is started when an operation to instruct the start of the storage service process is performed via the input unit 216 or the communication unit 219 .

[0082] In step S31, the structured data receiving unit 251 determines whether or not structured data for outputting position information has been received. If the structured data for outputting position information has not been received, the determination in step S31 is NO, and the process proceeds to step S33. On the other hand, if structured data for outputting position information has been received, the determination in step S31 is YES, and the process proceeds to step S32.

[0083] In step S32, the data management unit 255 stores the structured data for position information output received by the structured data receiving unit 251 in the structured data storage unit 271 in association with the identification information of the user who sent it. In step S33, the structured data transmission unit 252 determines whether or not a transmission request for structured data for outputting location information has been received. The transmission request for structured data for outputting location information includes the user's identification information and the identification information of the structured data for outputting location information that is requested to be transmitted.

[0084] If a request for transmitting structured data for outputting position information has not been received, the result of step S33 is determined to be NO, and the process proceeds to step S36. If a request to transmit structured data for outputting position information has been received, the determination in step S33 is YES, and the process proceeds to step S34. In step S34, the data management unit 255 reads out, from the structured data storage unit 271, structured data for outputting location information that corresponds to the identification information indicated in the transmission request from among the structured data for outputting location information that is associated with the user's identification information indicated in the transmission request.

[0085] In step S35, the structured data transmission unit 252 transmits the read structured data for outputting position information to the device that is the sender of the transmission request. In step S36, the content receiving unit 253 determines whether or not the content has been received. If the content has not been received, the determination in step S36 is NO, and the process proceeds to step S38. On the other hand, if the content has been received, the determination in step S36 is YES, and the process proceeds to step S37.

[0086] In step S37, the data management unit 255 stores the content received by the content receiving unit 253 in the related content storage unit 272 in association with the identification information of the user who sent the content. In step S38, the content transmission unit 254 determines whether or not a content transmission request has been received. The content transmission request includes the user identification information and the identification information of the content that is requested to be transmitted. If a content transmission request has not been received, the determination in step S38 is NO, and the storage service processing is repeated. If a content transmission request has been received, the determination in step S38 is YES, and the process proceeds to step S39.

[0087] In step S39, the data management unit 255 reads from the related content storage unit 272 the content corresponding to the identification information indicated in the transmission request from the content associated with the user identification information indicated in the transmission request. In step S40, the content transmission unit 254 transmits the read content to the device that is the source of the transmission request. After step S40, the storage service process is repeated.

[0088] [Video playback processing] FIG. 18 is a flowchart illustrating the flow of a video playback process executed by the PC 60 having the functional configuration of FIG. The video playback process is started when an operation is performed via input unit 119 to instruct the start of video playback process.

[0089] In step S51, the structured data acquisition unit 351 acquires the structured data for outputting location information by transmitting a request to transmit the structured data for outputting location information for which a video playback has been instructed by the user to the cloud storage server 50. The acquired structured data for outputting location information is stored in the structured data storage unit 371. In step S52, the video playback request unit 352 transmits a request to play the video to the map information providing server 40 together with the structured data for position information output for which playback of the video has been instructed by the user.

[0090] In step S53, the video playback unit 353 receives the video data based on the structured data for position information output transmitted from the map information providing server 40. In step S54, the video playback unit 353 plays the received video data on the display of the output unit 217. After step S54, the video playback process ends.

[0091] By this process, in the information processing system 1, structured data for outputting position information is automatically generated from the data of the history of the activity, based on the attributes of the video set according to the type of activity. Then, based on the structured data for outputting location information, the map information providing server 40 generates video data that traces the user's movement trajectory and moves on the map. The video data generated at this time includes content related to each position in the movement history, such as markers, photos, and comments, and these contents and videos are displayed in a display format suitable for the type of activity. Therefore, although it is extremely cumbersome for a user to manually set the attributes of a video for the movement history data to generate structured data for position information output, it is possible to easily set the effects of the video following the movement trajectory. That is, it becomes possible to perform a display with various pieces of information added when playing back the movement trajectory of the video.

[0092] [Modification Example 1] In the above-described embodiment, when setting the attributes of a video for the structured data for position information output, although the attributes of the video preset corresponding to the type of activity are automatically set, the present invention is not limited to this. For example, the attributes of the video may be set based on the size, degree of change, or rate of change of predetermined information in the movement trajectory. As an example, according to the stay time in the movement history, the number of captured images, the size (capacity) of the captured images, the number of comments, the size, degree of change, or rate of change of sensor information (the highest temperature, the highest altitude, the highest wind speed, the rate of change of heart rate, etc.), the attributes of the video at the positions corresponding to them can be set. Thereby, characteristic positions in the user's movement history can be displayed in a distinguishable manner in the video.

[0093] [Modification Example 2] In the above-described embodiment, when the user manually sets the attributes of a video for the structured data for position information output, the attributes of the video may be set by setting a category (evaluation information) for the position to be set. In this case, for example, the category is set as a rank representing the importance of the position, and the attributes of the video corresponding to each category are preset. Then, when any category is set for a predetermined position, the attributes of the video corresponding to that category are set as the attributes of the video at that predetermined position. Thereby, even when the user manually sets the attributes of the video, it becomes possible to set the attributes of the video with a simpler operation.

[0094] [Modification Example 3] In the above-described embodiment, the content related to the movement history has been described as the content manually associated by the user, such as by inputting the address of the storage area, but it is not limited to this. That is, when associating content with the movement history, various forms can be taken as the association method. As an example, the list terminal 10 can function as a remote controller of the imaging device 20. In this case, the list terminal 10 can receive and display a live view image from the imaging device 20. Then, when an instruction to save an image is given on the list terminal 10 or an instruction to take a picture is given to the imaging device 20 by the user, the list terminal 10 records an image (such as a thumbnail image) based on the live view image. Then, based on the recording date and time of the image based on the live view image, it becomes possible to associate a specific position in the movement history with the recorded live view image.

[0095] Also, as another example, when an instruction to take a picture is given to the imaging device 20, by transmitting information indicating the execution of shooting from the imaging device 20 to the list terminal 10, etc., the list terminal 10 can detect the execution of shooting in the imaging device 20. The list terminal 10 records the execution date and time of shooting in association with the movement history. Then, when the structured data for outputting position information is generated, the list terminal 10 collates the shooting date and time of the image stored in a predetermined storage area or the like with the execution date and time of shooting associated with the movement history, so that the image can be specified and associated.

[0096] Also, as yet another example, the list terminal 10 collates the date and time and position of the movement history with the shooting date and time and shooting position of the photos stored in a predetermined storage area or the like, and can automatically associate the content whose date and time and position match with the movement history. Note that content other than photos can also be associated with the movement history in these forms. Thereby, the user can easily associate appropriate content with the movement history without manually associating the movement history with the content.

[0097] [Modification Example 4] In the above-described embodiment, when a video tracing the movement history is displayed, content related to the data of the activity history is displayed based on the attributes of the videos set for each content. However, the present invention is not limited to this. For example, when a plurality of contents are associated with the same position or a proximity position (for example, within about several tens of meters), correction may be made to the attributes of the set videos so that more appropriate display is performed for the entire plurality of contents. As an example, in mountain climbing, when a plurality of photos are associated with the same position (for example, the mountaintop), any one of the photos may be selected and displayed, or the other photos may not be displayed, or the display time of each photo may be shortened for display. In addition, in the case where a marker is automatically set at a predetermined position according to the staying time at the position, if the action radius per unit time is less than the threshold value, it is possible to control to avoid setting the marker at that position.

[0098] The list terminal 10 configured as described above includes a structured data generation unit 56. The structured data generation unit 56 generates structured data that defines the display of a video tracing the movement trajectory within the map display provided as map information based on the related information related to a predetermined position in the movement trajectory, the display control information of the video tracing the movement trajectory, and the movement trajectory information representing the movement trajectory. The movement trajectory is a movement trajectory corresponding to the history of a predetermined action of the user. The structured data generation unit 56 generates structured data so as to display a video in the display control information according to the content of a predetermined action of the user. Thereby, by executing the structured data, it becomes possible to display a video tracing the movement trajectory corresponding to the action history while displaying the related information with respect to the movement trajectory according to the display control information. Therefore, it becomes possible to perform display with various information added during the reproduction of the movement trajectory.

[0099] The structured data generation unit 56 associates the related information related to a predetermined position in the movement trajectory with the predetermined position in the movement trajectory information, and generates structured data that defines the display of the related information at the predetermined position in the video following the movement trajectory in a display form based on the display control information. Thereby, while performing an effect of displaying the information related to the position at the predetermined position in the movement trajectory according to the predetermined display control, it is possible to display the video following the movement trajectory.

[0100] The structured data defines at least any one of the position of the point of interest, information regarding the viewpoint, and the operation at the time of a preset event. Thereby, it is possible to define videos in various forms following the movement trajectory.

[0101] The structured data generation unit 56 generates structured data so as to display a video according to the display control information according to the type of a predetermined action of the user. Thereby, it becomes possible to display a video following the movement trajectory in an appropriate display form according to the type of action performed by the user.

[0102] The structured data generation unit 56 changes the display control information according to the content of a predetermined action of the user and generates structured data. Thereby, it becomes possible to change the display control information so that a video following the movement trajectory is in an appropriate display form according to the content of the action performed by the user.

[0103] The display control information includes information for controlling at least any one of the display time of the related information, the change in the display size, and the expression method in the display. Thereby, it becomes possible to display the related information in various display forms.

[0104] In addition, the list terminal 10 includes a parameter correction reception unit 55. The parameter correction reception unit 55 corrects the display control information. This makes it possible to correct the set display control information according to the user's request or the like.

[0105] The structured data generation unit 56 automatically acquires the preset display control information or acquires the display control information input by the user. This makes it possible to set the display control information omitting the user's input labor or to flexibly set the display control information according to the user's request.

[0106] The structured data generation unit 56 sets the display control information based on at least any one of the size, degree of change, or rate of change of predetermined information in the movement trajectory. This makes it possible to display the characteristic positions in the movement trajectory in a distinguishable manner in the video.

[0107] The display control information is classified into predetermined categories, and the control content is determined for each classified category. The structured data generation unit 56 sets the display form of the video following the movement trajectory based on the category set for the movement trajectory. This makes it possible to set with a simpler operation even when the user manually sets the display control information.

[0108] The structured data generation unit 56 generates structured data for displaying related information so as not to overlap with the movement trajectory. This makes it possible to easily grasp which position in the movement trajectory the related information corresponds to.

[0109] The related information includes the captured image, the image acquired from another device, the comment, the marker, and the information from the sensor. This makes it possible to display various related information in association with the movement trajectory.

[0110] Further, the map information providing server 40 configured as described above includes a structured data execution unit 152. The structured data execution unit 152 executes structured data that defines the display of a video that follows the movement trajectory within a map display provided as map information, which is created based on related information related to a specific position on the movement trajectory, display control information for a video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory. The movement trajectory corresponds to the history of a predetermined action of the user. The structured data execution unit 152 executes the video in accordance with the display control information in response to the content of the predetermined action of the user. This makes it possible to display a video that traces a movement trajectory corresponding to a behavior history while displaying information related to the movement trajectory in accordance with the display control information. Therefore, when the movement trajectory is played back, it is possible to display the movement trajectory with various information added thereto.

[0111] The PC 60 configured as above also includes a video playback unit 353 . The video playback unit 353 acquires and displays video data resulting from the execution of structured data that defines the display of a video that follows the movement trajectory within a map display provided as map information, based on related information related to a specific position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory. The movement trajectory corresponds to the history of a predetermined action of the user. The video playback unit 353 displays the video that has been executed in accordance with the display control information in accordance with the content of the predetermined action of the user. This makes it possible to display a video that traces a movement trajectory corresponding to a behavior history while displaying information related to the movement trajectory in accordance with the display control information. Therefore, when the movement trajectory is played back, it is possible to display the movement trajectory with various information added thereto.

[0112] The present invention is not limited to the above-described embodiment, and any modifications and improvements that can achieve the object of the present invention are included in the present invention.

[0113] In the above-described embodiment, when the imaging device 20 performs imaging during an activity, the data of the captured image is uploaded to the cloud storage server 50 via the smartphone 30. However, the present invention is not limited to this. That is, if the data of the image captured by the imaging device 20 can be used for video playback processing on the PC 60, the captured image may be stored in a device other than the cloud storage server 50.

[0114] In the above-described embodiment, the list terminal 10 acquires the data of the movement history and generates the structured data for position information output, the map information providing server 40 interprets and executes the structured data for position information output, and the PC 60 plays back the video. However, the present invention is not limited to this. That is, these processes can be executed by a single information processing device or arbitrarily shared and executed by a plurality of information processing devices.

[0115] In the above-described embodiment, the content related to the movement history can be various information such as the captured image (photo), the image acquired from another device via the network 70 or the like, the comment, the marker, and the information detected by the sensor (temperature, weather (humidity or atmospheric pressure, etc.), altitude, wind speed, biometric information of the user (body temperature, heart rate)).

[0116] In the above-described embodiment, as the presentation form of the content, the display time of the content, the change in the display size, and the presentation method in the display (rotation, blinking, semi-transparent display, etc.) can be defined in the attributes of the video.

[0117] In the above-described embodiment, the structured data for position information output generated can be output as data in the KLM (Keyhole Markup Language) format, for example.

[0118] In addition, in the above-described embodiment, the information processing devices 100, 200 to which the present invention is applied have been described as examples of a wrist terminal 10, a photographing device 20, a smartphone 30, a map information providing server 40, a cloud storage server 50, or a PC 60, but are not limited to these. For example, the present invention can be applied to electronic devices in general that have a function for generating and processing structured data, a function for interpreting and executing structured data, or a function for playing video based on structured data. Specifically, the present invention can be applied to notebook personal computers, printers, television receivers, video cameras, portable navigation devices, mobile phones, smartphones, portable game consoles, etc.

[0119] The above-described series of processes can be executed by hardware or software. 4, 7, and 9 are merely examples and are not particularly limited. That is, it is sufficient for the information processing devices 100 and 200 to be provided with a function that can execute the above-described series of processes as a whole, and the functional blocks used to realize this function are not particularly limited to the examples in FIGS. Furthermore, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination thereof. The functional configuration in this embodiment is realized by a processor that executes arithmetic processing, and processors that can be used in this embodiment include those that are composed of various processing devices alone, such as single processors, multiprocessors, and multicore processors, as well as those that combine these various processing devices with processing circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays).

[0120] When a series of processes is executed by software, the programs that make up the software are installed into a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. Alternatively, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

[0121] A recording medium containing such a program is not only constituted by the removable media 131 and 231 in FIGS. 2 and 3 distributed separately from the apparatus main body to provide the program to the user, but also constituted by a recording medium or the like provided to the user in a state pre-installed in the apparatus main body. The removable media 131 and 231 are constituted by, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk is constituted by, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), a Blu-ray (registered trademark) Disc, or the like. The magneto-optical disk is constituted by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state pre-installed in the apparatus main body is constituted by, for example, the ROMs 112 and 212 in FIGS. 2 and 3 in which the program is recorded, or a hard disk included in the storage units 121 and 218 in FIGS. 2 and 3.

[0122] Note that, in this specification, the steps of describing the program recorded on the recording medium include not only the processes performed in chronological order along the order, but also the processes executed in parallel or individually even if they are not necessarily processed in chronological order. Also, in this specification, the term "system" shall mean the entire apparatus constituted by a plurality of devices, a plurality of means, or the like.

[0123] Although several embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and their modifications are included within the scope and spirit of the invention described in this specification, etc., and are also included in the invention described in the claims and their equivalents.

[0124] The inventions described in the claims of the present application as originally filed are set forth below. [Appendix 1] a structured data generating means for generating structured data that defines a display of the video that follows the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information for a video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory; the movement trajectory corresponds to a history of a predetermined action of the user, The information processing device is characterized in that the structured data generation means generates the structured data so that the video is displayed in the display control information in accordance with the content of the predetermined action of the user. [Appendix 2] The information processing device described in Appendix 1 is characterized in that the structured data generation means associates related information relating to a predetermined position in the movement trajectory with the predetermined position in the movement trajectory information, and generates the structured data that defines the display of the related information at the predetermined position in the video that follows the movement trajectory in a display form based on the display control information. [Appendix 3] The information processing device according to claim 1 or 2, characterized in that the structured data defines at least one of the position of a point of interest, information about a viewpoint, and an action at the time of a preset event. [Appendix 4] The structured data generation means generates the structured data so as to cause the display control information to display the video according to the type of the predetermined action of the user, the information processing apparatus according to any one of Appendices 1 to 3. [Appendix 5] The structured data generation means generates the structured data by changing the display control information according to the content of the predetermined action of the user, the information processing apparatus according to any one of Appendices 1 to 4. [Appendix 6] The display control information includes information for controlling at least any one of the display time of the related information, the change in the display size, and the expression method in the display, the information processing apparatus according to any one of Appendices 1 to 5. [Appendix 7] The information processing apparatus according to any one of Appendices 1 to 6, further comprising correction means for correcting the display control information. [Appendix 8] The structured data generation means automatically acquires the preset display control information or acquires the display control information input by the user, the information processing apparatus according to any one of Appendices 1 to 7. [Appendix 9] The structured data generation means sets the display control information based on at least any one of the size, the degree of change, or the rate of change of predetermined information in the movement locus, the information processing apparatus according to any one of Appendices 1 to 8. [Appendix 10] The display control information is classified into predetermined categories, and the control content is determined for each classified category. The structured data generation means sets the display form of the video following the movement locus based on the category set for the movement locus, the information processing apparatus according to any one of Appendices 1 to 9. [Appendix 11] The structured data generation means generates the structured data for displaying the related information so as not to overlap with the movement locus, and the information processing apparatus according to any one of Appendices 1 to 10. [Appendix 12] The related information includes an image taken, an image acquired from another device, a comment, a marker, and information from a sensor, and the information processing apparatus according to any one of Appendices 1 to 11. [Appendix 13] Structured data execution means for executing structured data in which the display of the moving image along the movement locus is defined in the map display provided as map information, based on the related information related to a predetermined position in the movement locus, the display control information of the moving image along the movement locus, and the movement locus information representing the movement locus. The movement locus is a movement locus corresponding to the history of a predetermined action of the user. The structured data execution means executes the moving image according to the display control information according to the content of the predetermined action of the user, and the information processing apparatus. [Appendix 14] Video display means for acquiring and displaying video data as a result of executing structured data in which the display of the moving image along the movement locus is defined in the map display provided as map information, based on the related information related to a predetermined position in the movement locus, the display control information of the moving image along the movement locus, and the movement locus information representing the movement locus. The movement locus is a movement locus corresponding to the history of a predetermined action of the user. The video display means displays the moving image executed according to the display control information according to the content of the predetermined action of the user, and the information processing apparatus. [Appendix 15] A structured data generation step including generating structured data for defining the display of the moving image along the movement locus in the map display provided as map information, based on the related information related to a predetermined position in the movement locus, the display control information of the moving image along the movement locus, and the movement locus information representing the movement locus. The movement trajectory is a movement trajectory corresponding to the history of a predetermined action of the user, In the structured data generation step, the structured data is generated so that the video is displayed in the display control information according to the content of the predetermined action of the user. An information processing method characterized by this. [Appendix 16] Including a structured data execution step of executing structured data in which the display of the video following the movement trajectory is defined in a map display provided as map information based on the related information related to a predetermined position in the movement trajectory, the display control information of the video following the movement trajectory, and the movement trajectory information representing the movement trajectory, The movement trajectory is a movement trajectory corresponding to the history of a predetermined action of the user, In the structured data execution step, the video is executed according to the display control information according to the content of the predetermined action of the user. An information processing method characterized by this. [Appendix 17] Including a video display step of acquiring and displaying video data as a result of executing structured data in which the display of the video following the movement trajectory is defined in a map display provided as map information based on the related information related to a predetermined position in the movement trajectory, the display control information of the video following the movement trajectory, and the movement trajectory information representing the movement trajectory, The movement trajectory is a movement trajectory corresponding to the history of a predetermined action of the user, In the video display step, the video executed according to the display control information is displayed according to the content of the predetermined action of the user. An information processing method characterized by this. [Appendix 18] On the computer, Realize a structured data generation function that generates structured data that defines the display of the video following the movement trajectory in a map display provided as map information based on the related information related to a predetermined position in the movement trajectory, the display control information of the video following the movement trajectory, and the movement trajectory information representing the movement trajectory, the movement trajectory corresponds to a history of a predetermined action of the user, The program is characterized in that the structured data generation function generates the structured data so that the video is displayed in the display control information according to the content of the specified user behavior. [Appendix 19] On the computer, A structured data execution function is realized that executes structured data that defines the display of the video that follows the movement trajectory within a map display provided as map information, the structured data being created based on related information related to a predetermined position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory, the movement trajectory corresponds to a history of a predetermined action of the user, The structured data execution function executes the video in accordance with the display control information in response to the content of the predetermined user action. [Appendix 20] On the computer, A video display function is realized that acquires and displays video data resulting from execution of structured data that defines the display of the video that follows the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory, the movement trajectory corresponds to a history of a predetermined action of the user, The video display function displays the video executed in accordance with the display control information in accordance with the content of the predetermined action of the user. [Explanation of symbols]

[0125] 1 ···information processing system, 10 ···wrist terminal, 20 ···photography device, 30 ···smartphone, 40 ···map information providing server, 50 ···cloud storage server, 60 ···PC, 70 ···network, 111, 211 ··CPU, 112, 212 ···ROM, 113, 213 ··RAM, 114, 214 ···bus, 115, 215 ···input / output interface, 116 ···GPS unit, 117 ···photography unit, 118 ···sensor unit, 119, 216 ···input unit, 120, 217 ···output unit, 121, 218 ···storage unit, 122, 219 ···communication unit, 123, 220 ···drive, 131, 231 ···removable media, 51 ···user interface Interface display control unit (UI display control unit), 52... history acquisition unit, 53... activity determination unit, 54... video attribute setting unit, 55... parameter correction reception unit, 56... structured data generation unit, 57... structured data registration unit, 71... history data storage unit, 151, 251... structured data reception unit, 152... structured data execution unit, 153... content acquisition unit, 252... structured data transmission unit, 253... content reception unit, 254... content transmission unit, 255... data management unit, 271, 371... structured data storage unit, 272... related content storage unit, 351... structured data acquisition unit, 352... video playback request unit, 353... video playback unit

Claims

1. Structured data generation means for generating structured data for displaying, within a map display provided as map information based on the type of the activity, a trajectory of the movement history and content related to the movement history together with data of a history of activities including the movement history of a user; wherein the structured data has an orientation defined at which a line of sight is directed when displaying a point of interest along the trajectory of the movement history, according to the type of the activity; wherein the structured data generation means determines the type of activity performed by the user based on data of the type of activity included in the data of the history of activities, sets an attribute of a video according to the type of the activity, and generates the structured data based on the attribute of the video according to the type of the activity; An information processing apparatus characterized by the above.

2. The information processing apparatus according to claim 1, further comprising activity reception means for receiving an input of the type of the activity.

3. The information processing apparatus according to claim 1, wherein the attribute of the video has at least either a position of a point of interest along the trajectory of the movement history or information regarding a viewpoint defined.

4. The information processing apparatus according to claim 3, wherein, as the information regarding the viewpoint, a movement speed that is a constant multiple of an actual movement speed in the activity, or a selected specific speed, or a speed at which the entire process moves in a preset time is defined.

5. The information processing apparatus according to claim 3, wherein, as the information regarding the viewpoint, an altitude of the viewpoint according to the type of the activity is defined.

6. A structured data generation step for generating structured data for displaying, within a map display provided as map information based on the type of the activity, a trajectory of the movement history and content related to the movement history together with data of a history of activities including the movement history of a user; wherein the structured data has an orientation set at which a line of sight is directed when displaying a point of interest along the trajectory of the movement history. The structured data generation step determines the type of activity performed by the user based on the data of the type of activity included in the data of the activity history, sets the attributes of the video according to the type of the activity, and generates the structured data based on the attributes of the video according to the type of the activity. An information processing method characterized by this.

7. On a computer, Based on the data of the activity history including the movement history of the user and the type of the activity, a structured data generation function for generating structured data for displaying the trajectory of the movement history and the content related to the movement history in a map display provided as map information is realized. In the structured data, the direction in which the line of sight is directed when displaying the attention point following the trajectory of the movement history is set. The structured data generation function determines the type of activity performed by the user based on the data of the type of activity included in the data of the activity history, sets the attributes of the video according to the type of the activity, and generates the structured data based on the attributes of the video according to the type of the activity. A program characterized by this.

Images