Mobile terminal, information processing device, and information processing method

Abstract

[Problem] Video filmed by a mobile terminal that moves indoors could not be suitably used in the prior art. [Solution] Video filmed by a mobile terminal that moves indoors can be suitably used by a mobile terminal 5 comprising: a position acquisition unit 231 that acquires relative position information identifying the relative position of a mobile terminal 5 indoors; a filming unit 232 that films video; a composition unit 234 that associates, with the video, the relative position information or absolute position information which has been acquired by using the relative position information, and acquires an additional video; and a video output unit 251 that outputs the additional video.

Description

Mobile terminal, information processing device, and information processing method

[0001] The present invention relates to a mobile terminal or the like that captures video while moving indoors.

[0002] 2. Description of the Related Art Conventionally, there has been a technique for providing a user with image data of an in-vehicle camera taken at a location and environment desired by the user (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-83034

[0004] However, in the prior art, it has not been possible to appropriately utilize videos captured by a mobile terminal moving indoors.

[0005] The mobile terminal of the first invention is a mobile terminal that includes a position acquisition unit that acquires relative position information that identifies the relative position of the mobile terminal indoors, a shooting unit that shoots a video, a component that associates the relative position information or absolute position information acquired using the relative position information with the video, acquires additional video, and a video output unit that outputs the additional video.

[0006] With this configuration, it is possible to provide a mobile terminal that can capture moving images and can be used appropriately while moving around indoors.

[0007] In addition, the mobile terminal of the second invention, compared to the first invention, further includes an attribute value acquisition unit that acquires one or more camera attribute values ​​including directional information indicating the direction in which the shooting unit takes pictures, and the configuration unit is a mobile terminal that acquires an additional video in which the one or more camera attribute values ​​acquired by the attribute value acquisition unit are also associated with the video.

[0008] This configuration allows for proper provision of video footage captured while moving around indoors.

[0009] Furthermore, in the mobile terminal of the third invention, compared to the first or second invention, the position acquisition unit is a mobile terminal equipped with absolute position acquisition means for acquiring absolute position information of the mobile terminal, and relative position acquisition means for acquiring relative position information when the absolute position information can no longer be acquired.

[0010] This configuration allows for proper provision of video footage captured while moving around indoors.

[0011] In addition, the mobile terminal of the fourth invention is a mobile terminal that, compared to any one of the first to third inventions, further comprises a position transmission unit that transmits position information, which is relative position information or absolute position information obtained using the relative position information, to an information processing device, a movement instruction receiving unit that receives a transmission instruction from the information processing device, and a movement video transmission unit that transmits a video in response to the transmission instruction.

[0012] This configuration allows for proper provision of video footage captured while moving around indoors.

[0013] In addition, the mobile terminal of the fifth invention is a mobile terminal that, compared to any one of the first to third inventions, further comprises a mobile inquiry receiving unit that receives an inquiry having location identification information that identifies an indoor location, a mobile determination unit that determines whether relative location information or absolute location information obtained using the relative location information matches the inquiry, and a mobile video sending unit that sends a video when the mobile determination unit determines that the information matches the inquiry.

[0014] This configuration allows for proper provision of video footage captured while moving around indoors.

[0015] In addition, the information processing device of the sixth invention is an information processing device that includes an inquiry receiving unit that receives an inquiry having location identification information that identifies an indoor location from a user terminal, a video acquisition unit that acquires a video that matches the inquiry and is transmitted from a mobile terminal that activates indoors, and a video transmission unit that transmits the video to the user terminal.

[0016] This configuration makes it possible to provide a platform that allows appropriate use of videos captured by mobile terminals moving indoors.

[0017] In addition, the information processing device of the seventh invention, compared to the sixth invention, further comprises a terminal management unit in which terminal information including location information that identifies the location of the mobile terminal is stored, associated with each of one or more mobile terminals; a location receiving unit that receives location information from each of one or more mobile terminals, which is relative location information or absolute location information obtained using the relative location information; a location accumulation unit that associates the location information with the mobile terminal and accumulates it in the terminal management unit; and a terminal determination unit that determines the mobile terminal corresponding to the location information that corresponds to the location identified by the location identification information included in the query, and the video acquisition unit is an information processing device that acquires video taken by the mobile terminal determined by the terminal determination unit.

[0018] This configuration makes it possible to provide a platform that allows appropriate use of videos captured by mobile terminals moving indoors.

[0019] Furthermore, the information processing device of the eighth invention is an information processing device that, compared to the sixth invention, further includes a terminal management unit in which terminal information, which is information about the mobile terminal, is stored in correspondence with each of one or more mobile terminals, and an inquiry sending unit that sends an inquiry to each of one or more mobile terminals, and the video acquisition unit receives video from the mobile terminal in response to the sending of the inquiry.

[0020] This configuration makes it possible to provide a platform that allows appropriate use of videos captured by mobile terminals moving indoors.

[0021] Furthermore, the information processing device of the ninth invention is an information processing device in which, compared to any one of the sixth to eighth inventions, the video acquisition unit acquires videos taken by multiple mobile terminals, and further includes a video composition unit that temporally combines the multiple videos acquired by the video acquisition unit or spatially fuses the multiple videos acquired by the video acquisition unit to compose a single video, and the video transmission unit transmits the video composed by the video composition unit to the user terminal.

[0022] This configuration makes it possible to provide a platform that allows appropriate use of videos captured by mobile terminals moving indoors.

[0023] In addition, the information processing device of the tenth invention is an information processing device in which, compared to any one of the sixth to ninth inventions, the video acquired by the video acquisition unit is associated with a rights holder identifier that identifies the rights holder of the video, and further includes a rights holder processing unit that performs rights holder processing, which is processing related to the rights holder identified by the rights holder identifier associated with the video.

[0024] This configuration allows appropriate processing regarding the rights holder of the video.

[0025] In addition, the information processing device of the eleventh invention is an information processing device in which, compared to the tenth invention, the rights holder processing unit is equipped with a reward means that performs reward processing, which is a process of providing rewards to each rights holder identified by a rights holder identifier.

[0026] With this configuration, it is possible to give a reward to the rights holder of the video captured by the mobile terminal.

[0027] In addition, the information processing device of the twelfth invention is an information processing device in which, compared to the eleventh invention, the reward means performs reward processing, which is a process of determining a reward to the previous rights holder using a video attribute value corresponding to the video and awarding the reward.

[0028] With this configuration, appropriate compensation can be given to the rights holder of the video captured by the mobile terminal.

[0029] In addition, the information processing device of the thirteenth invention is an information processing device in which, compared to the tenth invention, the rights holder processing unit is equipped with a first security means that performs a first security processing that associates the video with an attribute value set corresponding to the video and stores it.

[0030] With this configuration, the video can be preserved.

[0031] In addition, the information processing device of the fourteenth invention is an information processing device in which, compared to the tenth invention, the rights holder processing unit is equipped with a second security means that performs a second security process to store the video in association with a rights holder identifier corresponding to the video.

[0032] With this configuration, the rights holder of the video can be the appropriate rights holder.

[0033] In addition, the information processing device of the fifteenth invention is an information processing device in which, compared to the tenth invention, the rights holder processing unit is equipped with a third security means for storing the video in association with a rights holder identifier that identifies the user of the user terminal.

[0034] With this configuration, the rights holder of the video can become the rights holder who needs the video.

[0035] In addition, the information processing device of the sixteenth invention is an information processing device in which, compared to any one of the thirteenth to fifteenth inventions, the rights holder processing unit is equipped with a fourth security means that performs a fourth security process that accumulates security information including access information for accessing the video in a blockchain.

[0036] With this configuration, it is possible to preserve management information for videos that need to be preserved.

[0037] According to the mobile terminal of the present invention, it is possible to appropriately provide videos captured while moving around indoors.

[0038] FIG. 1 is a conceptual diagram of an information system A in a first embodiment; FIG. 2 is a block diagram of the information system A; FIG. 3 is a block diagram of the information processing device 1; FIG. 4 is a flowchart illustrating an example of the operation of the information processing device 1; FIG. 5 is a flowchart illustrating an example of the terminal determination process; FIG. 6 is a flowchart illustrating an example of the video fusion process; FIG. 7 is a flowchart illustrating an example of the image fusion process;

[0039] Hereinafter, embodiments of a mobile terminal and the like will be described with reference to the drawings. Note that components with the same reference numerals in the embodiments perform similar operations, and therefore repeated description may be omitted.

[0040] (First embodiment) <Outline of first embodiment>

[0041] In this embodiment, a mobile terminal that acquires relative position information of the mobile terminal indoors and outputs a captured video in association with the relative position information will be described.

[0042] In this embodiment, a mobile terminal will be described that also outputs one or more camera attribute values ​​including direction information indicating the shooting direction in association with a video.

[0043] In this embodiment, a mobile terminal will be described that, when it becomes unable to acquire absolute position information, acquires relative position information and outputs the relative position information in association with a captured video.

[0044] In this embodiment, a mobile terminal that transmits acquired location information to an information processing device and transmits a captured video to the information processing device in response to a transmission instruction received from the information processing device will be described.

[0045] In addition, in this embodiment, we will describe an information processing device that uses location information received from a mobile terminal to determine a mobile terminal that shoots a video that matches an inquiry received from a user terminal, instructs the mobile terminal to send the video, receives the video from the mobile terminal, and sends the video to the user terminal that sent the inquiry.

[0046] In this embodiment, an information processing device that sequentially transmits moving images received from a plurality of mobile terminals 2 to a user terminal will be described.

[0047] In addition, in this embodiment, we will describe an information processing device that temporally combines videos received from multiple mobile terminals 2 to form a single video, or spatially fuses them to form a single video, and transmits the single video to a user terminal.

[0048] Furthermore, in this embodiment, an information processing device that performs right holder processing, which is processing related to the right holder of the output video, will be described. The right holder processing includes, for example, a compensation processing described later and various security processing described later.

[0049] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc. <Outline of Information System A>

[0050] 1 is a conceptual diagram of an information system A according to this embodiment. The information system A comprises an information processing device 1, one or more mobile terminals 2, and one or more user terminals 3.

[0051] The information processing device 1 is a server that provides videos transmitted from one or more mobile terminals 2 to a user terminal 3. The information processing device 1 is, for example, a so-called server, such as a cloud server or an ASP server, but the type does not matter. The information processing device 1 may also be a device that constitutes a blockchain.

[0052] It should be noted that the interval between the capture times of the still images included in the moving image in this specification does not matter. The moving image may be, for example, a moving image at 60 frames per second or 30 frames per second, but may also be a collection of still images captured at intervals of a predetermined time (for example, one minute) or more, or may be a collection of still images captured when predetermined conditions are met.

[0053] The mobile terminal 2 is a terminal that can move. The mobile terminal 2 is a terminal that is installed on a moving object and that captures video. The mobile terminal 2 is, for example, a smartphone, a tablet terminal, a camera with a communication function, camera-equipped glasses, or a smart watch with a camera. Furthermore, "installed" usually means that it is fixed, but it may also be considered to include being in contact with or held by something. The mobile object is a moving object, such as a piece of equipment that moves underground, or a living thing. The living thing is usually a person, but it may also be a dog, cat, etc.

[0054] The user terminal 3 is a terminal used by a user. A user is someone who watches videos or needs videos. The user terminal 3 may have the functions of a mobile terminal 2. In other words, the user terminal 3 may be a terminal of a user who provides videos. The user terminal 3 may be, for example, a smartphone, a tablet terminal, a so-called personal computer, a navigation terminal, or the like, and its type does not matter.

[0055] The information processing device 1 and one or more mobile terminals 2, and the information processing device 1 and one or more user terminals 3 are generally capable of communicating via a network such as the Internet.

[0056] 2 is a block diagram of an information system A according to the present embodiment, and FIG. 3 is a block diagram of an information processing device 1. As shown in FIG.

[0057] The information processing device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit 14. The storage unit 11 includes a terminal management unit 111. The receiving unit 12 includes a location receiving unit 121 and an inquiry receiving unit 122. The processing unit 13 includes a location accumulation unit 131, a terminal determination unit 132, a video acquisition unit 133, a video composition unit 134, and a right holder processing unit 135. The right holder processing unit 135 includes a first security means 1351, a second security means 1352, a third security means 1353, a fourth security means 1354, and a remuneration means 1355. The transmitting unit 14 includes a video sending unit 141.

[0058] The mobile terminal 2 includes a movement storage unit 21, a movement receiving unit 22, a movement processing unit 23, a movement transmitting unit 24, and a movement output unit 25. The movement receiving unit 22 includes a movement instruction receiving unit 221. The movement processing unit 23 includes a position acquisition unit 231, a photographing unit 232, an attribute value acquisition unit 233, and a configuration unit 234. The position acquisition unit 231 includes absolute position acquisition means 2311 and relative position acquisition means 2312. The movement transmitting unit 24 includes a position transmission unit 241 and a movement video transmission unit 242. The movement output unit 25 includes a video output unit 251.

[0059] The user terminal 3 includes a user storage unit 31, a user reception unit 32, a user processing unit 33, a user transmission unit 34, a user reception unit 35, and a user output unit 36. <Details of Components of Information Processing Device 1>

[0060] Various types of information are stored in the storage unit 11. The various types of information include, for example, terminal information (to be described later), videos, and video attribute values ​​associated with the videos.

[0061] Video attribute values ​​are attribute values ​​of a video. Examples of video attribute values ​​include environmental information. Environmental information is information about the environment in which a video was shot. Examples of environmental information include location information, time information, weather information, temperature information, and seasonal information, which will be described later. Time information is information that identifies the time when a video was shot. The time when a video was shot may be around the time the video was shot, and accuracy is not required. Examples of time information include the time of day, year / month / date / time, year / month / date / time / minute, year / month / date / time / minute / second, year / month / date, and month / day. In other words, the granularity of the time information is not important. Weather information is information that identifies the weather at the location (or region) where the video was shot. Examples of weather information include "sunny," "rain," "snow," and "cloudy." Temperature information is information that identifies the temperature at the location where the video was shot. Examples of temperature information include "25 degrees" and "above 30 degrees." Seasonal information is information that identifies the season at the location where the video was shot. The season information is, for example, "spring," "summer," "early summer," and "winter."

[0062] The terminal management unit 111 stores one or more pieces of terminal information. The terminal information is associated with the mobile terminal 2. The terminal information is information relating to the mobile terminal 2. The terminal information here typically includes a terminal identifier that is an identifier of the mobile terminal 2, location information, and terminal communication information. The mobile terminal 2 here is typically a terminal that can transmit video.

[0063] The terminal identifier is information that identifies the mobile terminal 2. The terminal identifier may be a rights holder identifier that identifies the rights holder who is the user of the mobile terminal 2. The terminal identifier is, for example, the ID of the mobile terminal 2, the user identifier of the user of the mobile terminal 2, the name of the mobile terminal 2, the IP address of the mobile terminal 2, or the MAC address of the mobile terminal 2.

[0064] A rights holder is a person who has some right to a video shot by the mobile terminal 2. A rights holder is, for example, the owner of the video, the copyright holder of the video, the owner of the mobile terminal 2 that shot the video, or the person with the right to dispose of the video. A rights holder is, for example, the initial rights holder of the video. The rights holder is usually the owner of the mobile terminal 2, but it may be any person who has the rights to the video shot by the mobile terminal 2.

[0065] The right holder identifier may be a terminal identifier. The right holder identifier is, for example, the right holder's ID, name, email address, or telephone number. The right holder's ID is, for example, a user identifier.

[0066] The position information is information that specifies the position of the mobile terminal 2. The position information is, for example, absolute position information of the mobile terminal 2 or relative position information of the mobile terminal 2. The absolute position information is, for example, (latitude, longitude) or (movement, longitude, altitude). The relative position information is information that specifies a position relative to a reference point. The relative position information is, for example, information that specifies a relative position indoors. The relative position information is, for example, a relative position indoors, or a relative position indoors between an area identifier. The relative position indoors is, for example, (x coordinate value, y coordinate value) relative to the reference point (0,0), or (x coordinate value, y coordinate value, h coordinate value) relative to the reference point (0,0,0). The h coordinate value is the altitude (height).

[0067] The area identifier is information that identifies an area corresponding to the relative position information. An area usually has a size, but it may also be a single location. The area identifier is the ID of an indoor area or the name of the indoor area (hereinafter referred to as the area name, as appropriate). The area name is, for example, the name of an underground mall, the name of a station, the name of a building, or the name of a street.

[0068] The terminal communication information is information for communicating with the mobile terminal 2. The terminal communication information is, for example, the IP address of the mobile terminal 2, the MAC address of the mobile terminal 2, and the ID of the communication application installed in the mobile terminal 2.

[0069] The receiving unit 12 receives various types of information and instructions from the mobile terminal 2 or the user terminal 3. The various types of information and instructions include, for example, location information, inquiries, videos, and additional videos, which will be described later.

[0070] The receiving unit 12 may receive video from one or more mobile terminals 2 in pairs with location information. In such a case, it is preferable for the receiving unit 12 to receive video continuously. Continuous reception means, for example, receiving periodically or receiving constantly. The intervals between periods may vary slightly.

[0071] The position receiving unit 121 receives position information from one or more mobile terminals 2. The position information is information that identifies the position of the mobile terminal 2. The position information received by the position receiving unit 121 is, for example, relative position information, a region identifier and relative position information, or absolute position information obtained using relative position information. The position receiving unit 121 usually receives position information in association with a terminal identifier.

[0072] It is preferable that the location receiving unit 121 continuously receives the latest location information from one or more mobile terminals 2 .

[0073] The inquiry receiving unit 122 receives inquiries from one or more user terminals 3. An inquiry can be considered an instruction to transmit a video. The inquiry includes location-specific information. The inquiry may also include absolute location information. The inquiry may also include one or more dynamic attribute values. Note that the one or more dynamic attribute values ​​included in the inquiry typically constitute a condition linked with AND.

[0074] The location identification information is information that identifies an indoor location. The location identification information is, for example, relative position information of an indoor store and the name of the indoor store. The location identification information has, for example, a location name. The location identification information may be, for example, WHAT3WORDS (registered trademark), which allows a location to be identified with any three words, and may also include a floor identifier that identifies a floor of a building. The floor identifier is information that identifies a floor (level) of a building, underground mall, etc. The location name is the name of the location. The location name is, for example, the name of a store, a place used for meeting up, the name of a specific item, or a street name. The location identification information has, for example, a floor identifier. The floor identifier is information that identifies a floor. For example, floor identifiers are "3rd floor," "basement," "2nd basement floor," and "rooftop."

[0075] The processing unit 13 performs various processes, such as processes performed by a location accumulation unit 131, a terminal determination unit 132, a video acquisition unit 133, a video composition unit 134, and a rights holder processing unit 135.

[0076] The processing unit 13 may store the videos continuously received by the receiving unit 12 in association with the location information and the terminal identifier.

[0077] The location accumulation unit 131 accumulates the location information received by the location receiving unit 121 from the mobile terminal 2 in the terminal management unit 111 in association with the mobile terminal 2. The location accumulation unit 131 accumulates the location information in the terminal management unit 111, for example, in pairs with a terminal identifier.

[0078] The terminal determination unit 132 determines one or more mobile terminals 2 corresponding to the location information corresponding to the location specified by the location specifying information included in the received inquiry.

[0079] The terminal determination unit 132 may determine one or more mobile terminals 2 corresponding to the location information corresponding to the absolute location information included in the received inquiry.

[0080] The terminal determination unit 132 determines, for example, one or more mobile terminals 2 corresponding to the location information included in the inquiry and that satisfies the approximation condition. Preferably, such location information is the latest location information of each mobile terminal 2 in the terminal management unit 111. The approximation condition is that the two locations are close to each other. For example, the approximation condition is that the location specified by the location information is close to the location indicated by the location information. For example, the approximation condition is that the distance between the location specified by the location information and the location indicated by the location information is within a threshold or is smaller than a threshold. Note that the distance is usually a physical distance, but it may also be travel time.

[0081] Depending on the structure or content of the location identification information, the terminal determination unit 132 determines the mobile terminal 2 as follows, for example: (1) When the location identification information includes a location name

[0082] The terminal determination unit 132, for example, refers to a location name database and acquires relative location information corresponding to the location name included in the query. Next, the terminal determination unit 132 determines, for example, one or more mobile terminals 2 corresponding to relative location information that satisfies the approximate condition for the location indicated by the relative location information.

[0083] In such a case, a location name database is stored in the storage unit 11 or an external device. In addition, in such a case, the location information received by the location receiving unit 121 is relative location information. The location name database has one or more pairs of location names and relative location information. The location name database may have one or more pairs of area identifiers, location names, and relative location information. The location name database may have one or more pairs of area identifiers, floor identifiers, location names, and relative location information. (2) When location identification information includes area identifiers and relative location information

[0084] The terminal determination unit 132, for example, determines one or more mobile terminals 2 that correspond to location information that matches the area identifier included in the query and satisfies the approximation condition with the relative location information included in the query. In such a case, the location information received by the location receiving unit 121 is an area identifier and relative location information. (3) When the location identification information includes an area identifier, a floor identifier, and a location name The terminal determination unit 132, for example, refers to a location name database and acquires relative location information that corresponds to the location name included in the query. Note that this relative location information is information that specifies a relative location on a floor identified by a floor identifier in an area identified by the area identifier. Then, the terminal determination unit 132, for example, determines one or more mobile terminals 2 that correspond to location information that matches the area identifier included in the query, matches the floor identifier included in the query, and satisfies the approximation condition with the acquired relative location information. In such a case, the location information received by the location receiving unit 121 is an area identifier, a floor identifier, and relative location information. (4) When the location identification information includes absolute location information

[0085] For example, the position accumulation unit 131 acquires absolute position information for the received relative position information from the relative position information received by the position receiving unit 121, using absolute position information of a reference point for the relative position information. Then, the position accumulation unit 131 accumulates the acquired absolute position information in the terminal management unit 111, paired with the terminal identifier of the mobile terminal 2. Through this processing, absolute position information indicating the latest position of each of one or more mobile terminals 2 is stored in the terminal management unit 111.

[0086] Then, the terminal determination unit 132, for example, refers to the terminal management unit 111 and determines one or more mobile terminals 2 that correspond to the absolute location information included in the inquiry and that satisfy the approximation condition.

[0087] The video acquisition unit 133 acquires a video that matches the inquiry and is transmitted from the mobile terminal 2 that activates the indoor function.

[0088] The video acquisition unit 133 may acquire videos taken by each of a plurality of mobile terminals 2. For example, the video acquisition unit 133 receives a video that matches the query from one mobile terminal 2. Then, when the one mobile terminal 2 moves and no longer matches the query, the video acquisition unit 133 receives a video that matches the query from another mobile terminal 2.

[0089] The video acquisition unit 133 acquires, for example, a video captured by the mobile terminal 2 determined by the terminal determination unit 132. The video acquisition unit 133 acquires, for example, terminal communication information corresponding to each of the one or more mobile terminals 2 determined by the terminal determination unit 132 from the terminal management unit 111. Next, the video acquisition unit 133 transmits, for example, a transmission instruction to the mobile terminal 2 corresponding to each of the one or more pieces of terminal communication information. Next, the video acquisition unit 133 receives the video from each of the one or more mobile terminals 2 in response to the transmission of the transmission instruction. Note that the transmission instruction may be transmitted by the transmitting unit 14. The video may be received by the receiving unit 12. Also, the transmission instruction is an instruction to transmit the captured video.

[0090] The video acquisition unit 133 acquires, for example, a video paired with the terminal identifier of the mobile terminal 2 determined by the terminal determination unit 132 from the storage unit 11. In this case, the mobile terminal 2 continuously transmits the captured video to the information processing device 1.

[0091] The video composition unit 134, for example, temporally combines the multiple videos acquired by the video acquisition unit 133 to compose a single video. Note that a single video combined in time is a single video created by connecting multiple original videos. When the multiple original videos that form a single video combined in time are shot, it is preferable that they are consecutive in time, but they may also be separated in time. For example, the time when one original video was shot may be, for example, 8:20, and the day when another original video was shot may be, for example, 8:25. A single video combined in time is appropriately referred to as a combined video.

[0092] Furthermore, the video composition unit 134, for example, spatially fuses the multiple videos acquired by the video acquisition unit 133 to compose a single video. A single spatially fused video is a video constructed by constructing a frame using some or all of the frames that make up each of the multiple original videos, and then connecting the multiple frames in time. Note that at least one of the frames that make up the single video is a frame that contains some or all of the frames of each of the multiple original videos. A single spatially fused video is appropriately referred to as a fused video.

[0093] The moving image composing unit 134 may perform processing up to passing one moving image acquired by the moving image acquiring unit 133 to the moving image transmitting unit 141. Such processing may also be considered to be processing for composing one moving image. The processing up to passing to the moving image transmitting unit 141 is usually processing for sequentially arranging the moving images in a buffer for transmitting the moving images.

[0094] The video composition unit 134, for example, combines multiple videos acquired by the video acquisition unit 133 in the order of time information corresponding to the videos to compose a single video. The video composition unit 134, for example, acquires portions of multiple videos acquired by the video acquisition unit 133 and combines the portions of the videos in order to compose a single video. Combining multiple videos in time typically means combining portions of videos taken by multiple mobile terminals 2 in the order in which they were taken. Combining portions of videos in order may also mean passing portions of the videos in order to the video transmission unit 141. Furthermore, combining videos in the order of time corresponding to the videos means combining the videos in the order of time indicated by the time information corresponding to the videos, or combining the videos in the order in which they were received. Combining multiple videos in time to compose a single video is sufficient as long as the user sees a single video as a result.

[0095] The moving image composition unit 134 composes one moving image, for example, by spatially fusing parts of each of the multiple moving images that have different position information associated with each of the multiple moving images acquired by the moving image acquisition unit 133. The moving image composition unit 134 composes one frame using some or all of the frames that compose each of the multiple moving images acquired by the moving image acquisition unit 133, and connects the frames in time to compose one moving image.

[0096] The process of spatially fusing frames included in each of a plurality of moving images is, for example, the following processes (a) and (b): (a) A method based on image processing

[0097] The video composition unit 134, for example, performs a process of matching the direction and scale of each of the multiple frames that are the target of the spatial connection process. Next, the video composition unit 134, for example, detects identical areas in each of the multiple frames. Next, the video composition unit 134, for example, performs a process of overlapping multiple frames that have identical areas to compose one wide-area frame. Note that it is possible to detect identical areas in each of the multiple frames using publicly known technology. (b) Method based on machine learning

[0098] The video composition unit 134, for example, provides a plurality of frames and a learning model to a module that performs machine learning prediction processing, executes the module, and acquires one wide-area frame.

[0099] The learning model is obtained by providing multiple training data sets, each with multiple frames as explanatory variables and a single wide-area frame composed of the multiple frames as the objective variable, to a module that performs machine learning learning processing, and then executing the module.

[0100] The learning model may also be referred to as a learner, a classifier, a classification model, etc. Deep learning is preferable as a machine learning algorithm, but random forests, etc., are also acceptable. For machine learning, various machine learning functions, such as the TensorFlow library and the random forest module of the R language, and various existing libraries can be used.

[0101] The rights holder processing unit 135 performs rights holder processing. The rights holder processing is processing related to a rights holder identified by a rights holder identifier associated with a video. The rights holder processing unit 135 performs, for example, processing in response to the video transmission unit 141 transmitting a video, and is processing related to a rights holder identified by a rights holder identifier associated with the video. The rights holder processing is, for example, a first security processing described below, a second security processing described below, a third security processing described below, a fourth security processing described below, and a compensation processing described below.

[0102] The rights holder identifier associated with a video may be, for example, the rights holder identifier associated with one or more videos that are the basis for the video, or the identifier of the user who requested the video. The user who requested the video is the user of the user terminal 3 that sent the inquiry.

[0103] The first security means 1351 performs a first security process of storing a moving image composed by the moving image composition unit 134 in association with a set of attribute values ​​corresponding to one or more moving images that are the basis of the moving image. The first security means 1351 may also perform a first security process of storing a moving image received from the mobile terminal 2 in association with a set of attribute values ​​corresponding to the moving image.

[0104] The second security means 1352 performs a second security process of storing a video created by the video composition unit 134 in association with a rights holder identifier corresponding to one or more videos that are the basis of the video.

[0105] In addition, the first security means 1351 or the second security means 1352 may store a video constructed by the video construction unit 134 in association with a set of attribute values ​​corresponding to one or more videos that formed the basis of the video, and with a rights holder identifier corresponding to one or more videos that formed the basis of the video.

[0106] The third security means 1353 stores one video composed by the video composition unit 134 in association with a rights holder identifier that identifies the user of the user terminal 3. The user of the user terminal 3 here is the person who watches the one video. The user terminal 3 here is, for example, the terminal that sent the inquiry.

[0107] Furthermore, the destination for storing one video is, for example, the storage unit 11, but it may also be another device, another device that constitutes a blockchain, etc.

[0108] The fourth security means 1354 performs a fourth security process to store security information. The security information includes access information for accessing one of the stored videos. The order of storing the video and performing the fourth security process on the security information corresponding to the video does not matter.

[0109] The fourth security means 1354 performs a fourth security process, for example, of storing security information including access information for accessing one video that has been composed and stored by the video composition unit 134 in the blockchain.

[0110] It is preferable that the fourth security means 1354 stores the security information in a blockchain. That is, it is preferable that the fourth security means 1354 stores the security information in a distributed ledger of the blockchain. It is preferable that the fourth security means 1354 registers the security information as an NFT (non-fungible token). It is preferable that the fourth security means 1354 registers the security information in a distributed file system in an IPFS (Inter Planetary File System) network.

[0111] The preservation information is information for ensuring the originality of a video. The preservation information can be said to be header information for the video. The preservation information is, for example, access information and one or more video attribute values. It is preferable that the preservation information has, for example, one or more rights holder identifiers. When the preservation information has multiple rights holder identifiers, the rights holder of the video may be shared, and the multiple rights holder identifiers may be rights holder history information. The rights holder history information is a collection of rights holder identifiers and is information that indicates the history of the rights holder. The fourth preservation process can ensure the originality of the preservation information of the registered video. By ensuring the originality of the preservation information, the originality of the video corresponding to the preservation information is also guaranteed. The access information is information for accessing the video. The access information is information that identifies the storage location of the video. The access information is, for example, a URL or a URI.

[0112] The security information preferably includes information (which may be called a flag) indicating whether the video can be provided to a third party. The flag may be, for example, information indicating that the video can be viewed by a third party, information indicating that the video can be sold, or information indicating that neither viewing nor selling is permitted.

[0113] The remuneration means 1355 performs remuneration processing for each right holder identified by a right holder identifier associated with one or more videos that are the basis of one video composed by the video composition unit 134 .

[0114] The reward process is a process of providing a reward. The reward process is, for example, a process of increasing points managed in pairs with one or more rights holder identifiers associated with a video. The reward process is, for example, a process of depositing money to a rights holder identified by one or more rights holder identifiers associated with a video. The reward process is, for example, a process of transmitting a video or other content to a user terminal 3 of a rights holder identified by one or more rights holder identifiers associated with a video. The reward process is not limited to the type of merit provided to the rights holder identified by one or more rights holder identifiers associated with the video. The reward may be, for example, money, points, goods, content, or the like, and the type of merit is not limited.

[0115] It is preferable that the reward means 1355 acquires one or more video attribute values ​​corresponding to one or more videos that are the basis for a video sent by the video sending unit 141, and performs reward processing, which is a process of determining rewards for each of multiple rights holders using the one or more video attribute values ​​and awarding the rewards.

[0116] Here, the one or more moving image attribute values ​​are, for example, the amount of moving image data, the length of moving image, the number of frames of moving image, and the resolution of moving image.

[0117] The reward means 1355 preferably acquires a reward amount corresponding to a service identifier that identifies a service provided for a target video, and performs a reward process that awards a reward corresponding to the reward amount. The service identifier may be, for example, "view" or "purchase." In this case, the storage unit 11 stores, for example, the reward amount corresponding to the service identifier, or information for determining the reward amount corresponding to the service identifier.

[0118] The reward means 1355 performs reward processing, which is a process of obtaining a reward amount using, for example, one or more pieces of information from one or more video attribute values ​​and one or more service identifiers, and awarding a reward corresponding to the reward amount. In such a case, for example, an arithmetic expression or a correspondence table corresponding to each of the multiple service identifiers is stored in the storage unit 11. The arithmetic expression is a formula for calculating the reward amount using one or more video attribute values ​​as parameters. The correspondence table is a table having multiple pieces of correspondence information for managing reward amounts corresponding to one or more video attribute values.

[0119] The reward means 1355 typically performs a process of charging a user who has enjoyed a service related to the target video with a reward. The process of charging a user with a reward is, for example, a process of charging the user with the amount of reward that has been acquired. The process of charging a user with a reward is, for example, a process of charging the user with the amount of reward that has been acquired and the amount of profit that the operator of the information processing device 1 will gain. The process of charging a user with a reward is, for example, a process of deducting points corresponding to the user who has enjoyed the service, a settlement process using the user's credit card number, etc.

[0120] The transmitter 14 transmits various types of information or instructions to the mobile terminal 2 or the user terminal 3. The various types of information or instructions include, for example, a video, a transmission instruction, and a stop instruction. Note that the stop instruction is an instruction to stop transmission of the video.

[0121] The video sending unit 141 transmits the video composed by the video composition unit 134 to the user terminal 3. The video composed by the video composition unit 134 may be the video itself acquired by the video acquisition unit 133. <Details of Components of the Mobile Terminal 2>

[0122] Various types of information are stored in the moving storage unit 21. The various types of information include, for example, a video, an attribute value set, a rights holder identifier, and a camera attribute value. The camera attribute value is an attribute value of a camera. The camera attribute value is, for example, an angle of view, direction information, and resolution.

[0123] For example, one or more moving image attribute values ​​constituting an attribute value set correspond to one or more still images (which may also be called fields or frames) that make up a moving image. The one or more moving image attribute values ​​may correspond to all of the still images, some of the still images, or multiple still images.

[0124] The mobile receiving unit 22 receives various types of information. The various types of information include, for example, a transmission instruction and a stop instruction. The mobile instruction receiving unit 221 receives a transmission instruction from the information processing device 1.

[0125] The movement processing unit 23 performs various processes, such as processes performed by a position acquisition unit 231, a photographing unit 232, an attribute value acquisition unit 233, and a configuration unit 234.

[0126] The position acquisition unit 231 acquires relative position information that specifies the indoor relative position of the mobile terminal 2. The position acquisition unit 231 acquires absolute position information that indicates the outdoor position of the mobile terminal 2.

[0127] The absolute position acquisition unit 2311 acquires absolute position information of the mobile terminal 2. The absolute position acquisition unit 2311 acquires the absolute position information by, for example, a GPS receiver. It is preferable that the absolute position acquisition unit 2311 stores the acquired latest absolute position information in the movement storage unit 21.

[0128] The relative position acquisition unit 2312 acquires relative position information of the mobile terminal 2. It is preferable that the relative position acquisition unit 2312 stores the acquired latest relative position information in the movement storage unit 21.

[0129] It is preferable that the relative position acquisition unit 2312 acquires relative position information when the absolute position acquisition unit 2311 is no longer able to acquire absolute position information. For example, the relative position acquisition unit 2312 acquires relative position information when the mobile terminal 2 moves from outdoors to indoors.

[0130] The relative position acquisition means 2312 acquires relative position information of the mobile terminal 2, for example, by LiDAR. The relative position acquisition means 2312 receives signals from, for example, three or more signal transmitters and calculates relative position information using the strength of each of the three or more signals and the position information of each of the three or more signal transmitters. The relative position acquisition means 2312 receives relative position information from, for example, a signal transmitter (e.g., an RFID tag or a beacon) that transmits relative position information. However, the means and method by which the relative position acquisition means 2312 acquires the relative position information are not important. For example, the relative position acquisition means 2312 receives an area identifier from a signal transmitter installed indoors. For example, the relative position acquisition means 2312 receives an area identifier and a floor identifier from a signal transmitter installed indoors. For example, the relative position acquisition means 2312 receives a location name from a signal transmitter installed indoors. Note that the technology by which the relative position acquisition means 2312 acquires relative position information that identifies an indoor location is not important.

[0131] The photographing unit 232 photographs a video. For example, the photographing unit 232 photographs a video while the mobile terminal 2 is moving. It is preferable that the photographed video be associated with the location information acquired by the location acquisition unit 231.

[0132] It is preferable that the imaging unit 231 stores the captured video in the mobile storage unit 21. When the imaging unit 231 stores video in the mobile storage unit 21, which has a limited storage capacity, it is preferable that the imaging unit 231 overwrites new video in the area where old video is stored. In other words, it is preferable that the structure of the mobile storage unit 21 is a ring buffer structure.

[0133] The attribute value acquisition unit 233 acquires one or more camera attribute values ​​including direction information indicating the direction in which the image is captured by the image capture unit 232. It is preferable that the attribute value acquisition unit 233 acquires one or more video attribute values ​​including the camera attribute value.

[0134] The attribute value acquisition unit 233 acquires, for example, one or more video attribute values. One or more video attribute values ​​may be referred to as an attribute value set. The attribute value acquisition unit 233 acquires time information from a clock (not shown), for example, during shooting. The attribute value acquisition unit 233 acquires time information, for example, constantly, at predetermined intervals, or when acquisition conditions are met. The attribute value acquisition unit 233 acquires time information from a clock (not shown), for example, during shooting, and acquires seasonal information corresponding to the time information. The attribute value acquisition unit 233 acquires weather information, for example, during shooting. Note that even if the shooting location is indoors, the weather information may be information about the weather in a region similar to the indoor location. The movement processing unit 23 acquires weather information corresponding to the location information, for example, from a server (not shown). The movement processing unit 23 acquires weather information, for example, constantly, at predetermined intervals, or when acquisition conditions are met. The attribute value acquisition unit 233 acquires temperature information, for example, during shooting. The movement processing unit 23 acquires temperature information corresponding to the location information, for example, from a server (not shown). The movement processing unit 23 acquires temperature information, for example, from a temperature sensor installed on the moving object. The movement processing unit 23 acquires temperature information, for example, constantly, at predetermined intervals, or when acquisition conditions are met. Note that the acquisition conditions may be, for example, when the analysis results of the video satisfy a predetermined condition.

[0135] The composition unit 234 associates the acquired position information with the video and acquires the additional video. The composition unit 234, for example, associates relative position information or absolute position information acquired using the relative position information with the video and acquires the additional video. The composition unit 234, for example, associates absolute position information acquired by the absolute position acquisition unit 2311 with the video and acquires the additional video.

[0136] The composition unit 234 acquires, for example, an additional video in which one or more camera attribute values ​​acquired by the attribute value acquisition unit 233 are also associated with the video. The composition unit 234 acquires, for example, an additional video in which one or more video attribute values ​​acquired by the attribute value acquisition unit 233 are also associated with the video.

[0137] The mobile transmission unit 24 transmits various types of information to the information processing device 1. The various types of information include, for example, location information, moving images, and additional moving images. The mobile transmission unit 24 may continuously transmit moving images acquired by the shooting unit 232 to the information processing device 1. In such cases, the moving images are usually associated with location information.

[0138] It is preferable that the mobile transmitter 24 stops transmitting the video when a stop instruction is received.

[0139] The position transmission unit 241 transmits the position information acquired by the position acquisition unit 231 to the information processing device 1. The position information is, for example, relative position information or absolute position information acquired using the relative position information.

[0140] It is preferable that the location transmission unit 241 continuously transmits the location information to the information processing device 1. "Continuously" may include "always" or "periodically." "Periodic" means that the transmission intervals may vary slightly. The frequency and intervals at which the location transmission unit 241 transmits the location information are not important.

[0141] The moving video transmitting unit 242 transmits the video in response to the reception of the transmission instruction by the movement instruction receiving unit 221. It is preferable that the moving video transmitting unit 242 transmits an additional video.

[0142] The movement output unit 25 outputs various types of information, such as a moving image and a set of attribute values.

[0143] Output is a concept that includes displaying on a display, projecting using a projector, printing on a printer, outputting sound, transmitting to an external device, storing on a recording medium, and handing over processing results to other processing devices or other programs.

[0144] The video output unit 251 outputs a video. It is preferable that the video output unit 251 outputs an additional video. The output here may be storage in the mobile storage unit 21. <Details of Components of the User Terminal 3>

[0145] Various types of information are stored in the user storage unit 31. The various types of information include, for example, a user identifier, a video, and an inquiry.

[0146] The user receiving unit 32 receives various instructions and information. The various instructions and information include, for example, inquiries and purchase instructions. Note that inquiries may include purchase instructions.

[0147] It should be noted that a purchase instruction is an instruction to purchase a video. The purchase instruction is associated with a user identifier. The purchase instruction typically includes information that identifies the video. The purchase instruction has, for example, a video identifier. The purchase instruction includes, for example, an inquiry. The purchase instruction includes, for example, purchase conditions. The purchase conditions are, for example, a purchase price. The purchase conditions include, for example, information that identifies the rights period.

[0148] The means for inputting various instructions and information may be any means, such as a touch panel, keyboard, mouse, or menu screen.

[0149] The user processing unit 33 performs various processes. For example, the various processes are processes for creating a data structure for transmitting various instructions and information received by the user receiving unit 32. For example, the various processes are processes for creating a structure for transmitting information received by the user receiving unit 35.

[0150] The user transmission unit 34 transmits various instructions and information to the information processing device 1. The various instructions and information are, for example, inquiries and purchase instructions.

[0151] The user receiving unit 35 receives various information and instructions. The various information and instructions are, for example, moving images and additional moving images. Note that the moving images may be considered to be a concept that includes the additional moving images.

[0152] The user output unit 36 ​​outputs various types of information, such as moving images and additional moving images.

[0153] Here, output is a concept that includes displaying on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and delivery of processing results to other processing devices or other programs.

[0154] The storage unit 11, the terminal management unit 111, the mobile storage unit 21, and the user storage unit 31 are preferably non-volatile recording media, but may also be realized by volatile recording media.

[0155] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0156] The receiving unit 12, the location receiving unit 121, the inquiry receiving unit 122, the movement receiving unit 22, the movement instruction receiving unit 221, and the user receiving unit 35 are typically realized by wireless or wired communication means, but may also be realized by means of receiving broadcasts.

[0157] The processing unit 13, position accumulation unit 131, terminal determination unit 132, video acquisition unit 133, video composition unit 134, rights holder processing unit 135, first security means 1351, second security means 1352, third security means 1353, fourth security means 1354, reward means 1355, movement processing unit 23, position acquisition unit 231, attribute value acquisition unit 233, composition unit 234, and user processing unit 33 can typically be realized by a processor, memory, etc. The processing procedures of the processing unit 13, etc. are typically realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type does not matter.

[0158] The transmitting unit 14, video transmitting unit 141, mobile transmitting unit 24, location transmitting unit 241, mobile video transmitting unit 242, and user transmitting unit 34 are typically realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0159] The absolute position acquisition means 2311 is realized by, for example, a GPS receiver.

[0160] The relative position acquisition unit 2312 is realized by, for example, LiDAR. The relative position acquisition unit 2312 is realized by, for example, a communication unit and software.

[0161] The image capturing unit 232 includes a camera capable of capturing video. Note that the type of camera is not important.

[0162] The movement output unit 25 and the video output unit 251 can be realized, for example, by an output device such as a display or a speaker, or by a combination of driver software for an output device, etc. The movement output unit 25 and the video output unit 251 can be realized, for example, by a processor, a memory, etc.

[0163] The user reception unit 32 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0164] The user output unit 36 ​​may or may not include an output device such as a display or speaker. The user output unit 36 ​​may be realized by driver software for an output device, or by a combination of driver software for an output device and the output device. <Operation>

[0165] Next, an example of the operation of the information system A will be described. First, an example of the operation of the information processing device 1 will be described using the flowchart in Fig. 4. Note that "S" in each flowchart used in the following description indicates a step.

[0166] (S401) The location receiving unit 121 determines whether or not location information has been received from the mobile terminal 2. If location information has been received, the process proceeds to S402, and if location information has not been received, the process proceeds to S403. The received location information is associated with a terminal identifier.

[0167] (S402) The location accumulation unit 131 associates the location information received in S401 with the terminal identifier and accumulates it in the terminal management unit 111. The process returns to S401.

[0168] In addition, when the position accumulation unit 131 receives relative position information, it may obtain absolute position information from the absolute position information of the reference point and the relative position information, associate the absolute position information with the terminal identifier, and accumulate it in the terminal management unit 111.

[0169] (S403) The inquiry receiving unit 122 determines whether or not an inquiry has been received from the user terminal 3. If an inquiry has been received, the process proceeds to S404, and if an inquiry has not been received, the process returns to S401. Note that an inquiry is usually associated with the user identifier of the user who sent the inquiry.

[0170] (S404) The terminal determination unit 132 determines one or more mobile terminals 2 that satisfy the approximation condition for the point identification information included in the query. An example of such terminal determination processing will be described using the flowchart in Fig. 5. Note that the mobile terminals 2 that satisfy the approximation condition are mobile terminals 2 that correspond to location information that satisfies the approximation condition.

[0171] (S405) The moving image acquiring unit 133 assigns 1 to a counter i.

[0172] (S406) The video acquisition unit 133 determines whether the i-th mobile terminal 2 is present among the mobile terminals 2 determined in S404. If the i-th mobile terminal 2 is present, the process proceeds to S407; if not, the process proceeds to S409.

[0173] (S407) Video acquisition unit 133 acquires terminal communication information of i-th mobile terminal 2 from terminal management unit 111. Video acquisition unit 133 transmits a video transmission instruction to i-th mobile terminal 2 using the terminal communication information.

[0174] (S408) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S406.

[0175] (S409) The receiving unit 12 determines whether or not video has been received from one or more mobile terminals 2. If video has been received, the process proceeds to S410, and if video has not been received, the process returns to S409. Note that it is preferable to wait here until video has been received from all mobile terminals 2 to which transmission instructions have been sent.

[0176] (S410) The moving image composition unit 134 determines whether or not multiple moving images have been received in S409. If multiple moving images have been received, the process proceeds to S411, and if one moving image has been received, the process proceeds to S412.

[0177] (S411) The moving image composition unit 134 performs processing to spatially blend multiple moving images. An example of such moving image blending processing will be described with reference to the flowchart in FIG.

[0178] (S412) The video composition unit 134 places the acquired video in a video transmission buffer. The video transmission unit 141 transmits the video from the buffer to the user terminal 3 that has sent the inquiry.

[0179] (S413) The terminal determination unit 132 acquires from the terminal management unit 111 the latest location information of each of the one or more mobile terminals 2 determined in S404.

[0180] It is assumed that the location receiving unit 121 normally receives location information continuously and at all times from one or more mobile terminals 2, and the location storage unit 131 stores the latest location information in the terminal management unit 111 in association with the terminal identifier.

[0181] (S414) The terminal determination unit 132 determines whether the location specifying information included in the inquiry and the latest location information of one or more mobile terminals 2 satisfy the approximation condition. If the approximation condition is satisfied, the process proceeds to S409; if the approximation condition is not satisfied, the process proceeds to S415.

[0182] That is, as long as the latest location information of all the mobile terminals 2 determined in S404 satisfies the approximation condition, video is received from the mobile terminals 2. Then, video reception from the mobile terminals 2 whose latest location information no longer satisfies the approximation condition is terminated.

[0183] (S415) The transmitter 14 transmits a stop instruction to the mobile terminal 2 whose latest location information no longer satisfies the approximation condition.

[0184] (S416) The rights holder processing unit 135 performs a security process. An example of the security process will be described with reference to the flowchart in FIG.

[0185] (S417) The processing unit 13 determines whether or not to end the process of transmitting moving images to the user terminal 3. If the process of transmitting moving images is to be ended, the process returns to S401; if not, the process returns to S404.

[0186] In the flowchart of FIG. 4, the process ends when the power is turned off or an interrupt occurs to end the process.

[0187] Next, an example of the terminal determination process in S404 will be described with reference to the flowchart in FIG.

[0188] (S501) The terminal determination unit 132 acquires location identification information from the received inquiry.

[0189] (S502) The terminal determination unit 132 assigns 1 to a counter i.

[0190] (S503) The terminal determination unit 132 determines whether or not the i-th terminal information exists in the terminal management unit 111. If the i-th terminal information exists, the process proceeds to S504; if not, the process proceeds to S509.

[0191] (S504) The terminal determination unit 132 acquires the latest location information contained in the i-th terminal information from the terminal management unit 111.

[0192] (S505) The terminal determination unit 132 determines whether the location specification information acquired in S501 and the location information acquired in S504 satisfy the approximation condition. If the approximation condition is satisfied, the process proceeds to S506. If the approximation condition is not satisfied, the process proceeds to S508.

[0193] (S506) The terminal determination unit 132 acquires the terminal communication information included in the i-th terminal information from the terminal management unit 111, and stores the information in a buffer (not shown).

[0194] (S507) The terminal determination unit 132 calculates the distance between the position indicated by the point identification information acquired in S501 and the position indicated by the position information acquired in S504.

[0195] (S508) The terminal determination unit 132 increments the counter i by 1. The process returns to S503.

[0196] (S509) The terminal determination unit 132 sorts the mobile terminals 2 using the distance as a key. The process returns to the upper level process. Note that sorting the mobile terminals 2 is equivalent to sorting the terminal communication information.

[0197] 5, the terminal determination unit 132 may determine only a predetermined number of mobile terminals 2 or less. The predetermined number is, for example, "1" or "2." The terminal determination unit 132 may determine only the predetermined number of mobile terminals 2, for example, in ascending order of the distance acquired in S507.

[0198] Next, an example of the moving image fusion process in S411 will be described with reference to the flowchart in FIG.

[0199] (S601) The video composition unit 134 acquires reference location information. The reference location information is, for example, location information indicated by the location identification information included in the received inquiry. The reference location information is, for example, location information of the mobile terminal 2 that is closest to the location indicated by the location identification information.

[0200] (S602) The moving image composition unit 134 assigns 1 to a counter i.

[0201] (S603) The moving image composition unit 134 determines whether the i-th moment (i-th frame) in the moving image exists. If the i-th moment exists, the process proceeds to S604; if not, the process proceeds to S610.

[0202] (S604) The video composition unit 134 acquires a reference image. The reference image is an image in the video transmitted by the mobile terminal 2 that is closest to the position indicated by the point identification information. The first reference image is the first frame in the video transmitted by the mobile terminal 2.

[0203] (S605) The moving image composition unit 134 assigns 1 to a counter j.

[0204] (S606) The video composition unit 134 determines whether or not the jth other image exists. If the jth other image exists, the process proceeds to S607; if not, the process proceeds to S609. The other image is a frame in a video transmitted from another mobile terminal 2. Normally, the reference image and the other image are synchronized.

[0205] (S607) The moving image composition unit 134 performs image fusion processing. An example of the image fusion processing will be described with reference to the flowchart in FIG.

[0206] (S608) The moving image composition unit 134 increments the counter j by 1. The process returns to S604.

[0207] (S609) The moving image composition unit 134 increments the counter i by 1. The process returns to S603.

[0208] (S610) The moving image constructing unit 134 temporally combines the multiple fusion images fused in S607 to construct a fusion moving image, and temporarily stores the fusion moving image in a buffer (not shown). The process then returns to the upper level process.

[0209] Next, an example of the image fusion process in S607 will be described with reference to the flowchart in FIG.

[0210] (S701) The moving image composition unit 134 acquires a base image. The initial base image is the reference image acquired in S602. The base image is also the base image updated in S607 in the previous loop from S606 to S608.

[0211] (S702) The moving image composition unit 134 acquires area information that identifies the area covered by the base image acquired in S701. The area information is, for example, a set of position information of vertices that form a polygon, or a set of position information of two diagonal corners that form a rectangle.

[0212] (S703) The moving image composition unit 134 acquires the j-th other image in S606.

[0213] (S704) The moving image composing unit 134 acquires area information that identifies the area covered by the other images acquired in S703.

[0214] (S705) The video composition unit 134 determines an area in the base image where the other image will be placed, using the area information of the base image and the area information of the other image, places the other image in that area, and obtains an updated base image. The process then returns to the upper-level process.

[0215] Next, an example of the security process in S416 will be described with reference to the flowchart in FIG.

[0216] (S801) The rights holder processing unit 135 stores the transmitted moving image in association with a set of attribute values ​​corresponding to one or more moving images that are the basis of the transmitted moving image.

[0217] It is preferable that the rights holder processing unit 135 stores the video in pairs with a rights holder identifier that identifies one or more rights holders. The rights holder identifier here is, for example, one or more rights holder identifiers of the videos that are the source of the stored video. The rights holder identifier here is, for example, a single rights holder identifier that identifies the user who made the inquiry.

[0218] Furthermore, the rights holder processing unit 135 stores the video in, for example, the storage unit 11 or a device other than the information processing device 1. The device other than the information processing device 1 may be a device that constitutes a blockchain.

[0219] (S802) The fourth maintenance means 1354 performs the fourth maintenance process. An example of the fourth maintenance process has been described with reference to the flowchart in FIG.

[0220] (S803) The rights holder processing unit 135 assigns 1 to the counter i.

[0221] (S804) The rights holder processing unit 135 determines whether the i-th video that was the source of the stored video exists. If the i-th video exists, the process proceeds to S805; if not, the process returns to the upper process.

[0222] (S805) Reward unit 1355 performs reward processing. An example of the reward processing will be described with reference to the flowchart in Fig. 10. The reward processing here is a process of providing reward to the rights holder of the i-th video that is the basis for the accumulated video.

[0223] (S806) The rights holder processing unit 135 determines whether or not to change the rights holder of the i-th video that is the source of the stored video. If the rights holder is to be changed, the process proceeds to S807. If the rights holder is not to be changed, the process proceeds to S808.

[0224] Whether or not to change the rights holder may be based on a flag corresponding to the i-th video, may be predetermined, or the rights holder may be changed if the inquiry includes "information indicating a request to change the rights holder."

[0225] (S807) The rights holder processing unit 135 acquires the user identifier of the user terminal 3. This user identifier becomes the new rights holder identifier.

[0226] (S808) The first maintenance unit 1351 stores the i-th moving image that was the source of the stored moving images.

[0227] (S809) The fourth maintenance unit 1354 performs the fourth maintenance process on the i-th video that was the source of the stored video. An example of the fourth maintenance process was described using the flowchart in FIG.

[0228] (S810) The rights holder processing unit 135 increments the counter i by 1. The process returns to S804.

[0229] In the flowchart of Figure 8, when a combined video is constructed, a process (S808) of storing the multiple videos that formed the combined video and a fourth conservation process (S809) are performed, but such processes do not have to be performed.

[0230] Next, an example of the fourth maintenance process in steps S802 and S809 will be described with reference to the flowchart in FIG.

[0231] (S901) The fourth security unit 1354 acquires access information that identifies the storage location of a moving image, such as a fused moving image or a combined moving image.

[0232] (S902) The fourth preservation unit 1354 acquires a set of attribute values ​​corresponding to the stored video. If the stored video is composed of multiple original videos, the set of attribute values ​​corresponding to the video is the set of attribute values ​​of each of the multiple original videos.

[0233] (S903) The fourth security means 1354 constructs security information including the access information acquired in S901, the attribute value set acquired in S902, and the rights holder identifier of the video. If a new rights holder identifier has been acquired, the fourth security means 1354 constructs security information including, for example, the new rights holder identifier and the original rights holder identifier. (S904) The fourth security means 1354 stores the security information constructed in S903. The process returns to the upper level process.

[0234] In S904, if preservation information for a video corresponding to the preservation information to be stored has been stored, the preservation information configured in S903 is overwritten with the preservation information. This allows, for example, the transition of rights holders of the video to be managed. Furthermore, the fourth preservation unit 1354 stores the preservation information in a blockchain, for example.

[0235] Next, an example of the reward process in S805 will be described with reference to the flowchart in FIG.

[0236] (S1001) The reward unit 1355 acquires one or more right holder identifiers of the target video. The reward unit 1355 may also acquire the right holder identifiers of the previous right holders of the target video.

[0237] (S1002) The reward means 1355 acquires a set of attribute values ​​of the target video.

[0238] (S1003) The reward unit 1355 acquires a service identifier that identifies the service provided for the target video. The service identifier is, for example, "view" or "purchase."

[0239] (S1004) The reward unit 1355 acquires a reward amount using one or more pieces of information from the attribute value set acquired in S1002 and the service identifier acquired in S1003.

[0240] If multiple right holder identifiers are acquired, the remuneration unit 1355 acquires a remuneration amount for each right holder identifier. If history information of right holders including multiple right holder identifiers is acquired, the remuneration unit 1355 may acquire a remuneration amount for each right holder identifier.

[0241] Furthermore, it is preferable that the remuneration means 1355, for example, acquires a video attribute value corresponding to each of the multiple videos that were the basis for the video transmitted by the video transmission unit 141, and uses the video attribute value to determine a remuneration amount for each of the multiple rights holders. It is preferable that the remuneration means 1355, for example, determines a larger remuneration amount the greater the data amount, video duration, or number of frames of the original video used in the video transmitted by the video transmission unit 141. It is preferable that the remuneration means 1355, for example, determines a larger remuneration amount the greater the resolution of the original video used in the video transmitted by the video transmission unit 141.

[0242] (S1005) The reward unit 1355 performs processing to give a reward equal to the reward amount acquired in S1004 to the right holder identified by the right holder identifier acquired in S1001.

[0243] (S1006) The remuneration means 1355 performs a process of charging the user who has enjoyed the service related to the target video with a remuneration. The process returns to the upper level process. Note that the target video is usually a video transmitted to the user terminal 3.

[0244] In the flowchart of FIG. 10, the profits obtained by the operator of the information processing device 1 may be acquired and accumulated.

[0245] Next, an example of the operation of the mobile terminal 2 will be described with reference to the flowchart of FIG.

[0246] (S1101) The mobile terminal 2 determines whether or not a start instruction has been received. If a start instruction has been received, the process proceeds to S1102; if not, the process proceeds to S1111. The start instruction may be, for example, an instruction to launch a video transmission application or to power on the mobile terminal 2.

[0247] (S1102) The location acquisition unit 231 performs location acquisition processing. An example of the location acquisition processing will be described with reference to the flowchart in Fig. 12. The location acquisition processing is processing for acquiring current location information of the mobile terminal 2.

[0248] (S1103) The attribute value acquisition unit 233 acquires one or more moving image attribute values ​​excluding the position information.

[0249] (S1104) The configuration unit 234 acquires a terminal identifier from the movement storage unit 21.

[0250] (S1105) Composition unit 234 composes information to be transmitted, which is information having location information, one or more video attribute values, and a terminal identifier. Location transmission unit 241 transmits the information composed in S1105 to information processing device 1.

[0251] (S1106) The image capturing unit 232 captures and acquires video.

[0252] (S1107) Composition unit 234 composes an additional moving image having a moving image, location information, one or more moving image attribute values, and a terminal identifier. Composition unit 234 stores the additional moving image in transfer storage unit 21.

[0253] (S1108) The moving video transmission unit 242 determines whether or not to transmit the video to the information processing device 1. If the video is to be transmitted, the process proceeds to S1109; if not, the process proceeds to S1110. Note that when transmitting a video, for example, the transmission flag may be "ON." The initial value of the transmission flag is "OFF."

[0254] (S1109) The moving video sending unit 242 sends the additional image accumulated in S1107 to the information processing device 1. Note that there is no limit to the amount of video sent at one time.

[0255] (S1110) The movement processing unit 23 determines whether or not to end the transmission of the position information, etc. If the transmission is to be ended, the process returns to S1101, and if the transmission is not to be ended, the process returns to S1102. The transmission of the position information, etc. is ended, for example, when an end instruction is received from the user.

[0256] (S1111) The movement instruction receiving unit 221 determines whether or not a video transmission instruction has been received from the information processing device 1. If a transmission instruction has been received, the process proceeds to S1102, and if not, the process returns to S1101.

[0257] (S1112) The movement processing unit 23 sets the transmission flag to “ON.” The process returns to S1102.

[0258] (S1113) The mobile receiving unit 22 determines whether or not a stop instruction has been received from the information processing device 1. If a stop instruction has been received, the process proceeds to S1114, and if not, the process returns to S1101.

[0259] (S1114) The movement processing unit 23 turns the transmission flag “OFF.” The process returns to S1102.

[0260] In the flowchart of FIG. 11, the process ends when the power is turned off or an interrupt occurs to end the process.

[0261] Next, an example of the position acquisition process in S1102 will be described with reference to the flowchart in FIG.

[0262] (S1201) The absolute position acquisition unit 2311 acquires absolute position information and attempts to store the absolute position information in a buffer (not shown).

[0263] (S1202) The position acquisition unit 231 determines whether or not absolute position information was acquired in S1201. If absolute position information was acquired, the process returns to the upper processing, and if it was not acquired, the process proceeds to S1203. Note that if absolute position information was acquired in S1201 and the relative position acquisition means 2312 has been activated, it is preferable that the movement processing unit 23 stop the processing of the relative position acquisition means 2312.

[0264] (S1203) The position acquisition unit 231 activates the relative position acquisition means 2312. Note that if the relative position acquisition means 2312 has already been activated, the position acquisition unit 231 does nothing here.

[0265] (S1204) Relative position acquiring unit 2312 acquires relative position information and stores the relative position information in a buffer (not shown).

[0266] (S1205) The position acquisition unit 231 uses the relative position information acquired in S1204 to determine whether or not to acquire absolute position information. If absolute position information is to be acquired, the process proceeds to S1206, and if absolute position information is not to be acquired, the process returns to the upper level processing. Note that absolute position information is acquired, for example, when absolute position information that serves as a reference for the relative position information exists in the movement storage unit 21 or a buffer (not shown).

[0267] (S1206) The position acquisition unit 231 acquires absolute position information serving as a reference, calculates absolute position information corresponding to the relative position information using the absolute position information and the acquired relative position information, and stores the absolute position information in a buffer (not shown). The process then returns to the upper level process.

[0268] Next, an example of the operation of the user terminal 3 will be described with reference to the flowchart of FIG.

[0269] (S1301) The user reception unit 32 determines whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S1302; if not, the process proceeds to S1307. Note that the reception of an inquiry is, for example, the reception of an input by a user. The reception of an inquiry is, for example, the input of an instruction by the user about a specific location on the indoor map output to the mobile terminal 2.

[0270] (S1302) The user processing unit 33 composes a query to be transmitted. Next, the user transmitting unit 34 transmits the query to the information processing device 1 in association with the user identifier.

[0271] (S1303) The user receiving unit 35 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S1304, and if information has not been received, the process returns to S1303.

[0272] (S1304) The user processing unit 33 determines whether or not the information received in S1303 includes a moving image. If a moving image is included, the process proceeds to S1305, and if a moving image is not included, the process proceeds to S1306.

[0273] (S1305) The user output unit 36 ​​outputs the received video. Return to S1301.

[0274] (S1306) The user output unit 36 ​​outputs the other received information. Return to S1301.

[0275] (S1307) The user receiving unit 32 determines whether or not a purchase instruction has been received from the user. If a purchase instruction has been received, the process proceeds to S1308, and if not, the process returns to S1301. The purchase instruction may include, for example, an inquiry.

[0276] (S1308) The user processing unit 33 composes a purchase instruction to be transmitted. Next, the user transmitting unit 34 transmits the purchase instruction to the information processing device 1 in association with the user identifier.

[0277] (S1309) The user receiving unit 35 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S1310, and if information has not been received, the process returns to S1309. The information may be, for example, a video, information indicating that the transfer of rights holders has been completed, and a set of attribute values ​​of the video.

[0278] (S1310) The user processing unit 33 uses the received information to compose information to be output. The user output unit 36 ​​outputs the information. The process returns to S1301.

[0279] In the flowchart of Fig. 13, the process ends when the power is turned off or an interrupt occurs to end the process.

[0280] A specific example of the operation of the information system A in this embodiment will be described below.

[0281] Currently, the terminal management unit 111 of the information processing device 1 stores a terminal management table having the structure shown in FIG. 14. The terminal management table is a table for managing terminal information. The terminal management table manages one or more records each having an "ID," "terminal identifier," "location information," and "terminal communication information." The "location information" has an "area identifier" and "location details." The "area identifier" is an identifier of an area corresponding to relative location information. If the area identifier is "-", the "location details" is absolute location information. If the area identifier is information other than "-", the "location details" is relative location information. The "terminal communication information" is information used when communicating with the mobile terminal 2.

[0282] A large number of mobile terminals 2 capturing moving images perform the following operations and continuously transmit the latest location information to the information processing device 1.

[0283] That is, the GPS receiver included in the absolute position acquisition means 2311 of the mobile terminal 2 acquires absolute position information and attempts to store the absolute position information in a buffer (not shown). Next, the position acquisition unit 231 determines whether or not the absolute position information has been acquired. If the absolute position information has not been acquired, the relative position acquisition means 2312 is activated. Then, the communication module included in the relative position acquisition means 2312 receives an area identifier from a signal transmitter present in the area. Furthermore, the LiDAR included in the relative position acquisition means 2312 acquires relative position information in the area. Next, the position acquisition unit 231 generates position information including the area identifier and the relative position information. Note that if the absolute position acquisition means 2311 of the mobile terminal 2 has acquired the absolute position information of the mobile terminal 2, the relative position acquisition means 2312 is not activated. The position acquisition unit 231 repeatedly acquires such position information.

[0284] Next, the location transmission unit 241 of the mobile terminal 2 successively transmits the latest location information to the information processing device 1 in association with the terminal identifier.

[0285] Next, the location receiving unit 121 of the information processing device 1 continuously receives the latest location information associated with the terminal identifiers from a large number of mobile terminals 2. Next, the location accumulation unit 131 associates the location information with the terminal identifiers and accumulates it in the terminal management table. Through the above processing, the latest location information of a large number of mobile terminals 2 is managed in the terminal management table. In the above situation, the following two specific examples will be described.

[0286] (Specific Example 1) The user sends a query to the user terminal 3, "<Area Identifier> Underground Mall A Relative Position Information (x ａ , y ａ It is assumed that the user inputs the following command: "(location)" (location)." It is assumed that the user inputs the following command by specifying a point on the underground map of underground mall A displayed on the user terminal 3, for example.

[0287] Next, the user reception unit 32 of the user terminal 3 responds to the query “<area identifier> underground mall A relative position information (x ａ , y ａNext, the user processing unit 33 composes a query to be transmitted. Next, the user transmitting unit 34 transmits the query to the information processing device 1 in association with the user identifier.

[0288] Next, the inquiry receiving unit 122 of the information processing device 1 receives an inquiry from the user terminal 3. Next, the terminal determining unit 132 refers to the terminal management table ( FIG. 14 ) and determines the location specifying information “<area identifier> underground mall A relative position information (x ａ , y ａ In other words, the terminal determination unit 132 determines the mobile terminal 2 that satisfies the approximation condition for the area identifier "underground mall A" and the relative position information (x ａ , y ａ ) and the distance to the location details (x １ , y １ ) satisfies the approximation condition. Next, the terminal determination unit 132 acquires the terminal communication information "communication destination 1" from the terminal management table (FIG. 14). Note that the terminal determination unit 132 determines that the area identifier "underground mall A" matches, but the relative position information (x ａ , y ａ ) is greater than the threshold value. ２ , y ２ ) does not satisfy the approximation condition. That is, the terminal determination unit 132 does not acquire the terminal communication information "notification destination 2" from the terminal management table (FIG. 14).

[0289] Next, the video acquisition unit 133 transmits a video transmission instruction to the mobile terminal 2 with the terminal identifier "T001" that can communicate using the terminal communication information "communication destination 1."

[0290] Next, the movement instruction receiving unit 221 of the mobile terminal 2 receives a video transmission instruction from the information processing device 1. Next, the position acquisition unit 231 performs position acquisition processing. Furthermore, the attribute value acquisition unit 233 acquires one or more video attribute values ​​(e.g., time, temperature, etc.). Furthermore, the configuration unit 234 acquires a terminal identifier from the movement storage unit 21. The configuration unit 234 configures information to be transmitted, which is information having location information, one or more video attribute values, and a terminal identifier. The position transmission unit 241 transmits the configured information to the information processing device 1. The configuration unit 234 configures an additional video having a video, location information, one or more video attribute values, and a terminal identifier. The configuration unit 234 accumulates the additional video in the movement storage unit 21. The movement video transmission unit 242 transmits the additional image to the information processing device 1. This transmission of the video continues for a predetermined time or more.

[0291] Next, the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2. Next, the video composition unit 134 places the acquired video in a buffer for transmitting videos. The video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has transmitted the inquiry. The transmission of the video continues for a predetermined time or more.

[0292] Next, the user terminal 3 receives and outputs the video from the information processing device 1. The output of this video continues for a predetermined time or more. This allows the user to watch a video of a predetermined spot in underground mall A (for example, a store that the user wants to visit). By watching this video, the user can find out, for example, whether there is a line in front of a store.

[0293] Furthermore, it is assumed that due to the movement of the mobile terminal 2, the terminal determination unit 132 determines that the location information of the mobile terminal 2 no longer satisfies the approximation condition for the point specifying information included in the inquiry from the user.

[0294] Next, the rights holder processing unit 135 performs the preservation process described using the flowchart in Fig. 8. As a result, the user of the user terminal 3 that sent the video can receive a reward, and the video is properly preserved.

[0295] Next, it is assumed that, due to the movement of the user terminal 3 identified by the terminal identifier "T002", the location information of the user terminal 3 comes to satisfy the approximation condition for the point specifying information contained in the query.

[0296] Then, the terminal determination unit 132 of the information processing device 1 determines that the area identifier "underground mall A" matches the latest location information paired with the terminal identifier "T002", and the relative location information (x ａ , y ａ ) is within the threshold value, the detailed location information satisfies the approximation condition. Next, the terminal determination unit 132 acquires the terminal communication information "communication destination 2" from the terminal management table (FIG. 14).

[0297] Next, the video acquisition unit 133 transmits a video transmission instruction to the mobile terminal 2 with the terminal identifier "T002" that can communicate using the terminal communication information "communication destination 2".

[0298] Next, the movement instruction receiving unit 221 of the mobile terminal 2 receives a video transmission instruction from the information processing device 1. Next, the position acquisition unit 231 performs position acquisition processing. Furthermore, the attribute value acquisition unit 233 acquires one or more video attribute values ​​(e.g., time, temperature, etc.). Furthermore, the configuration unit 234 acquires a terminal identifier from the movement storage unit 21. The configuration unit 234 configures information to be transmitted, which is information having location information, one or more video attribute values, and a terminal identifier. The position transmission unit 241 transmits the configured information to the information processing device 1. The configuration unit 234 configures an additional video having a video, location information, one or more video attribute values, and a terminal identifier. The configuration unit 234 accumulates the additional video in the movement storage unit 21. The movement video transmission unit 242 transmits the additional image to the information processing device 1. This transmission of the video continues for a predetermined time or more.

[0299] Next, the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2. Next, the video composition unit 134 places the acquired video in a buffer for transmitting videos. The video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has transmitted the inquiry. The transmission of the video continues for a predetermined time or more.

[0300] Next, the user terminal 3 receives and outputs the video from the information processing device 1. The output of this video continues for a predetermined time or more. This allows the user to sequentially view the videos transmitted from the multiple mobile terminals 2. By viewing this video, the user can, for example, know the situation in front of the store for a long period of time.

[0301] As described above, according to this embodiment, it is possible to provide a mobile terminal that can shoot videos and can be used appropriately while moving around indoors.

[0302] Furthermore, according to this embodiment, it is possible to provide a platform that allows appropriate use of videos captured by a mobile terminal moving indoors.

[0303] The processing of each device in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may also be recorded on a recording medium such as a CD-ROM and distributed. This also applies to the other embodiments in this specification.

[0304] (Second embodiment) <Outline of second embodiment>

[0305] In the first embodiment, the information processing device 1 determines the mobile terminal 2 to which a video that matches the user's inquiry can be transmitted. In the present embodiment, the mobile terminal determines whether or not to transmit a video in response to the user's inquiry. <Outline of Information System A>

[0306] The conceptual diagram of the information system B in this embodiment is the same as FIG. 1 except for the reference numerals.

[0307] 15 is a block diagram of an information system B according to this embodiment. FIG. 16 is a block diagram of an information processing device 4.

[0308] The information system B includes an information processing device 4 , one or more mobile terminals 5 , and one or more user terminals 3 .

[0309] The information processing device 4 is a server that provides videos transmitted from one or more mobile terminals 5 to the user terminal 3. The information processing device 4 is, for example, a so-called server, such as a cloud server or an ASP server, but the type does not matter. The information processing device 4 may also be a device that constitutes a blockchain.

[0310] The mobile terminal 5 is a terminal installed on a moving object and capable of capturing video. The mobile terminal 5 is, for example, a smartphone, a tablet terminal, a camera with a communication function, camera-equipped glasses, or a smart watch with a camera.

[0311] The information processing device 4 and one or more mobile terminals 5, and the information processing device 4 and one or more user terminals 3 are generally capable of communicating via a network such as the Internet.

[0312] The information processing device 4 includes a storage unit 41, a receiving unit 42, a processing unit 43, and a transmitting unit 14. The storage unit 41 includes a terminal management unit 411. The receiving unit 42 includes an inquiry receiving unit 122. The processing unit 43 includes a video acquisition unit 433, a video composition unit 134, and a rights holder processing unit 135.

[0313] The mobile terminal 5 includes a movement storage unit 21, a movement receiving unit 52, a movement processing unit 53, a movement transmitting unit 54, and a movement output unit 25. The movement processing unit 53 includes a position acquisition unit 231, a shooting unit 232, an attribute value acquisition unit 233, a configuration unit 234, and a movement determination unit 531. The movement transmitting unit 54 includes a movement video transmitting unit 542. <Details of components of information processing device 4>

[0314] Various types of information are stored in the storage unit 41. The various types of information include, for example, terminal information (to be described later), videos, and video attribute values ​​associated with the videos.

[0315] The terminal management unit 411 stores one or more pieces of terminal information. The terminal information is associated with the mobile terminal 5. The terminal information is information relating to the mobile terminal 5. The terminal information here typically includes a terminal identifier, which is an identifier of the mobile terminal 2, and terminal communication information.

[0316] The receiving unit 42 receives various information and instructions from the mobile terminal 2 or the user terminal 3. The various information and instructions are, for example, inquiries, videos, and additional videos.

[0317] The processing unit 43 performs various processes, such as processes performed by the video acquisition unit 433, the video composition unit 134, and the rights holder processing unit 135.

[0318] The video acquisition unit 433 acquires a video that is transmitted from the mobile terminal 5 that activates the indoors and that matches the inquiry.

[0319] The video acquisition unit 433 normally receives video from the mobile terminal 5 in response to a transmitted inquiry. For example, the video acquisition unit 433 receives video captured by each of the plurality of mobile terminals 5 from the mobile terminal 5. The mobile terminal 5 that transmits the video is the mobile terminal 5 that corresponds to the location information that matches the inquiry. <Details of the components of the mobile terminal 5>

[0320] The movement receiving unit 52 receives various types of information. The various types of information are, for example, inquiries. The movement instruction receiving unit 221 receives the inquiries from the information processing device 1.

[0321] The movement processing unit 53 performs various processes, such as processes performed by the position acquisition unit 231, the photographing unit 232, the attribute value acquisition unit 233, the composition unit 234, and the movement determination unit 531.

[0322] The movement determination unit 531 determines whether or not the location information acquired by the location acquisition unit 231 matches the inquiry received by the mobile receiving unit 52. More specifically, the movement determination unit 531 determines whether or not the location information acquired by the location acquisition unit 231 satisfies an approximation condition for the point specifying information included in the inquiry received by the mobile receiving unit 52. Note that the location information acquired by the location acquisition unit 231 is, for example, relative location information or absolute location information acquired using the relative location information.

[0323] The mobile transmission unit 54 transmits various types of information to the information processing device 1. The various types of information include, for example, an attribute value set, a moving image, and an additional moving image.

[0324] The moving video transmitting unit 542 transmits the video when the movement determining unit 531 determines that the video matches the inquiry. Note that the video here may be an additional video.

[0325] The storage unit 41 and the terminal management unit 411 are preferably non-volatile recording media, but can also be realized by volatile recording media.

[0326] There is no restriction on the process by which information is stored in the storage unit 41 etc. For example, information may be stored in the storage unit 41 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 41 etc., or information input via an input device may be stored in the storage unit 41 etc.

[0327] The receiving unit 42 and the mobile receiving unit 52 are usually realized by wireless or wired communication means, but may also be realized by means for receiving broadcasts.

[0328] The processing unit 43, movement processing unit 53, and movement determination unit 531 can usually be realized by a processor, memory, etc. The processing procedures of the processing unit 43, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type does not matter.

[0329] The video acquisition unit 433 is usually realized by a wireless or wired communication means, but may also be realized by a means for receiving broadcasts.

[0330] The mobile transmitting unit 54 and the mobile video transmitting unit 542 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0331] Next, an example of the operation of the information system B will be described. First, an example of the operation of the information processing device 4 will be described using the flowchart of Fig. 17. In the flowchart of Fig. 17, the description of the same steps as in the flowchart of Fig. 4 will be omitted.

[0332] (S1701) The inquiry receiving unit 122 determines whether or not an inquiry has been received from the user terminal 3. If an inquiry has been received, the process proceeds to S1702, and if an inquiry has not been received, the process returns to S1701.

[0333] (S1702) The moving image obtaining unit 433 assigns 1 to a counter i.

[0334] (S1703) The video acquisition unit 433 determines whether or not the i-th terminal information exists in the terminal management unit 411. If the i-th terminal information exists, the process proceeds to S1704, and if not, the process proceeds to S1706. The existence of the terminal information means that there is a mobile terminal 5 to which video can potentially be transmitted.

[0335] (S1704) The video acquisition unit 433 sends a query to the mobile terminal 5 corresponding to the i-th terminal information.

[0336] (S1705) The moving image obtaining unit 433 increments the counter i by 1. The process returns to S1703.

[0337] (S1706) Receiving unit 42 determines whether or not additional moving images have been received from one or more mobile terminals 5. If additional moving images have been received, the process proceeds to S1707, and if additional moving images have not been received, the process proceeds to S1708.

[0338] (S1707) The processing unit 43 stores the received one or more additional moving images in a buffer (not shown). Then, the process returns to S1706.

[0339] (S1708) The processing unit 43 determines whether a timeout has occurred. If a timeout has occurred, the process proceeds to S1709, and if not, the process returns to S1706. Note that a timeout occurs when a time equal to or greater than a threshold has elapsed since the transmission of the inquiry. The threshold time is the time to wait for reception of video.

[0340] (S1709) Receiving unit 42 determines whether or not additional moving images have been received from one or more mobile terminals 5. If additional moving images have been received within the time until the timeout, the process proceeds to S410; if not, the process proceeds to S1710.

[0341] (S1710) The transmission unit 14 transmits an error message to the user terminal 3. The error message is information to the effect that a video corresponding to the inquiry could not be acquired.

[0342] In the flowchart of FIG. 17, the process ends when the power is turned off or an interrupt occurs to end the process.

[0343] Next, an example of the operation of the mobile terminal 5 will be described with reference to the flowchart of Fig. 18. In the flowchart of Fig. 18, the description of the same steps as in the flowchart of Fig. 11 will be omitted.

[0344] (S1801) The mobile receiving unit 52 determines whether or not an inquiry has been received from the information processing device 4. If an inquiry has been received, the process proceeds to S1102, and if an inquiry has not been received, the process returns to S1101.

[0345] (S1802) The location acquisition unit 231 acquires location information. An example of such location acquisition processing has been described with reference to the flowchart in FIG.

[0346] (S1803) The movement determining unit 531 acquires the location specifying information included in the inquiry.

[0347] (S1804) The movement determination unit 531 determines whether the location information acquired in S1102 and the point identification information acquired in S1803 satisfy the approximation condition. If the approximation condition is satisfied, the process proceeds to S1102, and if the approximation condition is not satisfied, the process proceeds to S1101.

[0348] (S1805) The movement processing unit 53 determines whether or not to end the process. If the process is to be ended, the process returns to S1101, and if the process is not to be ended, the process returns to S1102.

[0349] In the flowchart of Fig. 18, the process ends when the power is turned off or an interrupt occurs to end the process.

[0350] A specific example of the operation of the information system B in this embodiment will be described below.

[0351] It is assumed that the terminal management unit 411 of the information processing device 4 stores a terminal management table having attributes of "ID," "terminal identifier," and "terminal communication information" from the terminal management table shown in FIG.

[0352] In this situation, the user sends a query to the user terminal 3, "<area identifier> underground mall A relative position information (x ａ , y ａ ) is input.

[0353] Next, the user reception unit 32 of the user terminal 3 responds to the query “<area identifier> underground mall A relative position information (x ａ , y ａ Next, the user processing unit 33 composes a query to be transmitted. Next, the user transmitting unit 34 transmits the query to the information processing device 1 in association with the user identifier.

[0354] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry from the user terminal 3. Next, the video acquiring unit 433 transmits the inquiry to a large number of mobile terminals 5 in the terminal management table (FIG. 14).

[0355] Next, the mobile receiving unit 52 of each mobile terminal 5 receives the inquiry from the information processing device 4. Next, the location acquisition unit 231 of each mobile terminal 5 acquires location information by the process described using the flowchart in FIG.

[0356] Next, the movement determination unit 531 of each mobile terminal 5 acquires the location specification information included in the inquiry. Furthermore, the movement determination unit 531 determines whether the acquired location information and the acquired location specification information satisfy the approximation condition.

[0357] Next, the position acquisition unit 231 of the mobile terminal 5 that satisfies the approximation conditions performs a position acquisition process. The attribute value acquisition unit 233 of the mobile terminal 5 acquires one or more video attribute values ​​(e.g., time, temperature, etc.). The configuration unit 234 of the mobile terminal 5 acquires a terminal identifier from the mobile storage unit 21. The configuration unit 234 of the mobile terminal 5 configures information to be transmitted, which includes location information, one or more video attribute values, and a terminal identifier. The configuration unit 234 configures an additional video that includes a video, location information, one or more video attribute values, and a terminal identifier. The configuration unit 234 accumulates the additional video in the mobile storage unit 21. The mobile video transmission unit 542 of the mobile terminal 5 transmits the additional image to the information processing device 1. This video transmission continues for a predetermined period of time or longer.

[0358] Next, the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2. Next, the video composition unit 134 places the acquired video in a buffer for transmitting videos. The video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has transmitted the inquiry. The transmission of the video continues for a predetermined time or more.

[0359] Next, the user terminal 3 receives and outputs the video from the information processing device 1. The output of this video continues for a predetermined time or more. This allows the user to watch a video of a predetermined spot in underground mall A (for example, a store that the user wants to visit). By watching this video, the user can find out, for example, whether there is a line in front of a store.

[0360] Furthermore, it is assumed that, due to the movement of the mobile terminal 5, the movement determination unit 531 determines that the location information of the mobile terminal 5 no longer satisfies the approximation condition for the point specifying information included in the inquiry. Next, the mobile terminal 5 stops pausing the video.

[0361] Next, when the movement determination unit 531 determines that the approximation condition has been satisfied due to the movement of another mobile terminal 5, the position acquisition unit 231 of the mobile terminal 5 performs position acquisition processing. The attribute value acquisition unit 233 of the mobile terminal 5 acquires one or more video attribute values ​​(e.g., time, temperature, etc.). The configuration unit 234 of the mobile terminal 5 acquires a terminal identifier from the movement storage unit 21. The configuration unit 234 of the mobile terminal 5 configures information to be transmitted, which includes location information, one or more video attribute values, and a terminal identifier. The configuration unit 234 configures an additional video including a video, location information, one or more video attribute values, and a terminal identifier. The configuration unit 234 accumulates the additional video in the movement storage unit 21. The movement video transmission unit 542 of the mobile terminal 5 transmits the additional image to the information processing device 1. This video transmission continues for a predetermined time or longer.

[0362] Next, the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2. Next, the video composition unit 134 places the acquired video in a buffer for transmitting videos. The video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has transmitted the inquiry. The transmission of the video continues for a predetermined time or more.

[0363] Next, the user terminal 3 receives and outputs the video from the information processing device 1. The output of this video continues for a predetermined time or more. This allows the user to watch a video of a predetermined point in underground mall A for a long period of time. (Specific Example 2) Assume that a user different from the user in Specific Example 1 inputs a query "<area identifier> Building B <location name> Store S" to his / her user terminal 3. Next, the user reception unit 32 of the user terminal 3 receives the query "<area identifier> Building B <location name> Store S". Next, the user processing unit 33 composes the query to be sent. Next, the user transmission unit 34 transmits the query to the information processing device 1 in association with the user identifier. Next, the query reception unit 122 of the information processing device 1 receives the query from the user terminal 3. Next, the terminal determination unit 132 refers to a location name database (not shown) and selects relative position information (x Ｓ , y ＳNext, the terminal determination unit 132 refers to the terminal management table (FIG. 14) and acquires the location specification information "<area identifier> Building B relative position information (x Ｓ , y Ｓ In other words, the terminal determination unit 132 determines the mobile terminal 2 that satisfies the approximation condition for the area identifier "Building B" and the relative position information (x Ｓ , y Ｓ ) and the distance to the location details (x ４ , y ４ ) satisfies the approximation condition. Next, the terminal determination unit 132 acquires the terminal communication information "communication destination 4" from the terminal management table (FIG. 14). Note that the terminal determination unit 132 determines that the area identifier "Building B" matches, but the relative position information (x Ｓ , y Ｓ ) is greater than the threshold value. ３ , y ３) does not satisfy the approximation condition. In other words, the terminal determination unit 132 does not acquire the terminal communication information "notification destination 3" from the terminal management table ( FIG. 14 ). Next, the video acquisition unit 133 transmits a video transmission instruction to the mobile terminal 2 with the terminal identifier "T004" that can communicate using the terminal communication information "communication destination 4". Next, the movement instruction receiving unit 221 of the mobile terminal 2 receives the video transmission instruction from the information processing device 1. Next, the position acquisition unit 231 performs a position acquisition process. Furthermore, the attribute value acquisition unit 233 acquires one or more video attribute values ​​(e.g., time, temperature, etc.). Furthermore, the configuration unit 234 acquires a terminal identifier from the movement storage unit 21. The configuration unit 234 configures information to be transmitted, which is information having location information, one or more video attribute values, and a terminal identifier. The position transmission unit 241 transmits the configured information to the information processing device 1. The configuration unit 234 configures an additional video having a video, location information, one or more video attribute values, and a terminal identifier. The composition unit 234 accumulates the added video in the moving storage unit 21. The moving video transmission unit 242 transmits the added image to the information processing device 1. This transmission of the video continues for a predetermined time or more. Next, the receiving unit 12 of the information processing device 1 receives the added video from the mobile terminal 2. Next, the video composition unit 134 places the acquired video in a buffer for transmitting videos. The video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has sent the inquiry. This transmission of the video continues for a predetermined time or more. Next, the user terminal 3 receives and outputs the video from the information processing device 1. This output of the video continues for a predetermined time or more. In this way, the user can watch a video of the area around store S in building B. By watching this video, the user can, for example, find out whether there is a line in front of store S.

[0364] As described above, according to this embodiment, it is possible to provide a mobile terminal that can shoot videos and can be used appropriately while moving around indoors.

[0365] Furthermore, according to this embodiment, it is possible to provide a platform that allows appropriate use of videos captured by a mobile terminal moving indoors.

[0366] 19 shows the appearance of a computer that executes the programs described herein to realize the information processing device 1, mobile terminal 2, user terminal 3, information processing device 4, and mobile terminal 5 of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 19 is an overview of this computer system 300, and FIG. 20 is a block diagram of system 300.

[0367] In FIG. 19, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0368] 20, the computer 301 includes, in addition to a CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012 etc., a ROM 3015 for storing programs such as a boot-up program, a RAM 3016 connected to the MPU 3013 for temporarily storing instructions for application programs and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card for providing connection to a LAN.

[0369] A program that causes computer system 300 to execute the functions of information processing device 1 and the like of the above-described embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 when executed. The program may also be loaded directly from CD-ROM 3101 or the network.

[0370] The program does not necessarily have to include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the information processing device 1 of the above-described embodiment. The program only needs to include instructions that call appropriate functions (modules) in a controlled manner and achieve the desired results. How the computer system 300 operates is well known, and a detailed description thereof will be omitted.

[0371] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0372] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0373] Furthermore, in each of the above embodiments, it goes without saying that a plurality of communication means present in one device may be physically realized by one medium.

[0374] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0375] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention.

[0376] As described above, the mobile terminal according to the present invention has the effect of enabling appropriate use of videos captured by the mobile terminal moving indoors, and is useful as a terminal for providing videos, etc.

Claims

1. A position acquisition unit comprising: absolute position acquisition means for acquiring absolute position information indicating a current position outdoors using a GPS receiver; and indoor position acquisition means for receiving a signal including an area identifier that identifies a specific area indoors from a signal transmitter installed indoors when the absolute position acquisition means is no longer able to acquire the absolute position information due to moving from outdoors to indoors, and acquiring relative position information that specifies a relative position indicating the current position indoors; a mobile inquiry receiving unit that receives an inquiry having point identification information that includes the area identifier and the relative position information and identifies the indoor point from an information processing device; a movement determination unit that determines whether or not the area identifier and the relative position information acquired by the position acquisition unit satisfy an approximation condition with respect to the point identification information; An image capturing unit that captures an image; an additional image configuration unit that acquires an additional image in which the area identifier, the relative position information, and one or more camera attribute values ​​including direction information indicating a direction in which the image is captured by the image capture unit are associated with the image; a transmission unit that transmits the additional image that satisfies the approximation condition to the information processing device when the movement determination unit determines that the approximation condition is satisfied; A mobile terminal comprising:

2. In an information system including a plurality of mobile terminals according to claim 1, an information processing device, and a user terminal, The transmission unit of each of the mobile terminals transmitting the absolute location information, the area identifier, and the relative location information together with a terminal identifier for identifying each of the mobile terminals to the information processing device; The information processing device includes: a receiving unit for receiving the absolute location information, the area identifier, and the relative location information together with the terminal identifier from each of the mobile terminals; a terminal management unit that stores the received absolute position information, the area identifier, and the relative position information in association with each of the terminal identifiers; an inquiry receiving unit that receives the inquiry from the user terminal; a determination unit that determines whether or not a position indicated by the point identification information satisfies a similarity condition with a position indicated by the relative position information associated with any of the terminal identifiers stored in the terminal management unit; an inquiry sending unit that sends the inquiry to the mobile terminal identified by one or more terminal identifiers when the one or more terminal identifiers are associated with the location that is determined by the determining unit to satisfy the approximate condition; an additional image acquisition unit that receives and acquires the additional images transmitted from the mobile terminal in response to the inquiry; a configuration unit that temporally combines the acquired multiple additional images or spatially fuses the acquired multiple additional images to construct a single image; an image transmission unit that transmits the one image to the user terminal; Information processing system.

3. An information processing method for use in a mobile terminal, comprising: Obtaining absolute position information indicating a current position by a GPS receiver outdoors; receiving a signal including an area identifier that identifies a predetermined area in the room from a signal transmitter installed in the room, when the absolute position acquisition means is unable to acquire the absolute position information due to movement from outdoors to indoors, and acquiring relative position information that specifies a relative position that indicates a current position in the room; receiving an inquiry from an information processing device, the inquiry including point identification information that identifies the indoor point and that includes the area identifier and the relative position information; a determination step of determining whether or not the acquired area identifier and the acquired relative position information satisfy an approximation condition with respect to the point identification information; Taking an image; acquiring an additional image in which the area identifier, the relative position information, and one or more camera attribute values ​​including direction information indicating a direction in which the image capturing unit captures images are associated with the image; a step of transmitting the additional image that satisfies the approximation condition to the information processing device when it is determined in the determination step that the approximation condition is satisfied; An information processing method comprising: