Information processing device and information processing method

Abstract

[Problem] Conventionally, it has been impossible to use a moving image captured by a mobile terminal and a moving image captured by a fixed camera to form one useful moving image. [Solution] It is possible to use a moving image captured by a mobile terminal and a moving image captured by a fixed camera to form and provide one useful moving image by using an information processing device 1 comprising: a moving image acquisition unit 133 that acquires a mobile moving image which is a moving image captured by a mobile terminal 2, and that acquires a fixed moving image which is a moving image captured by a fixed camera; a moving image formation unit 134 that forms one moving image by temporally connecting the mobile moving image and the fixed moving image, or that forms one moving image which includes a plurality of frames obtained by spatially fusing at least some of the frames of the mobile moving image and at least some of the frames of the fixed moving image; and a moving image transmission unit 141 that transmits the one moving image formed by the moving image formation unit 134.

Description

[Technical field]

[0001] The present invention relates to an information processing device or the like that processes video captured by a mobile terminal and video captured by a fixed camera. [Background technology]

[0002] Conventionally, there have been image display systems that enable a user to view scenes other than the user's own field of view (scenes seen from moving objects other than the user's own field of view). In such image display systems, multiple moving images captured and transmitted by multiple moving objects can be output on a single screen (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-154192 A (paragraph

[0140] , Figure 24, etc.) Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, it was only possible to view videos taken by multiple moving objects on a single screen, and it was not possible to create and provide a single useful video by using videos taken by mobile terminals and videos taken by fixed cameras. [Means for solving the problem]

[0005] The information processing device of the first invention is an information processing device comprising: a video acquisition unit that acquires mobile videos, which are videos taken by a mobile terminal and correspond to an attribute value set including one or more pieces of environmental information including location information identifying the shooting location or time information identifying the time of shooting, and transmitted from a mobile terminal; and a fixed video, which is a video taken by a fixed camera and corresponds to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with a fixed camera whose shooting position is fixed; a video composition unit that temporally combines the mobile videos and the fixed videos to compose a single video, or composes a single video having multiple frames by spatially fusing at least a portion of the frames of the mobile video and at least a portion of the frames of the fixed video; and a video transmission unit that transmits the single video composed by the video composition unit.

[0006] With this configuration, a useful video can be constructed by using videos captured by a mobile terminal and videos captured by a fixed camera.

[0007] In addition, the information processing device of the second invention is an information processing device in which, compared to the first invention, the video acquisition unit is an information processing device that acquires a video corresponding to a priority type indicating a priority type of video among the fixed video and the moving video when there are fixed video and moving video corresponding to location information indicating positions close enough to each other to satisfy the first location condition, and the video acquisition unit is a plurality of videos that satisfy adoption conditions, which are conditions for the original videos that constitute a single video.

[0008] With this configuration, it is possible to select an appropriate video from among videos captured by a mobile terminal and videos captured by a fixed camera, and to compose a single useful video.

[0009] In addition, the information processing device of the third invention, compared to the first or second invention, further includes a receiving unit that receives moving videos from a mobile terminal and fixed videos from a fixed terminal equipped with a fixed camera, and the video acquisition unit acquires the moving videos and fixed videos received by the receiving unit, and the receiving unit receives the moving videos from a mobile terminal that has accepted the user's permission in accordance with a usage mode flag for the videos stored in the mobile terminal, is an information processing device.

[0010] With this configuration, a useful single video can be constructed by using moving videos and fixed videos based on the intention of the user of the mobile terminal.

[0011] Furthermore, the information processing device of the fourth invention is an information processing device in which, compared to the third invention, the usage mode flag is information indicating whether or not there is a "desire for non-provisional actual use" for the moving video sent from the mobile terminal, and the receiving unit receives the moving video from the mobile terminal only if the mobile terminal accepts the user's permission when the usage mode flag stored in the mobile terminal is information indicating "there is a desire for non-provisional actual use."

[0012] With this configuration, a useful single video can be constructed by using moving videos and fixed videos based on the intention of the user of the mobile terminal.

[0013] In addition, the information processing device of the fifth invention is an information processing device relative to any one of the first to fourth inventions, further comprising a rights holder processing unit that corresponds to a rights holder identifier that identifies the rights holder of the moving video or fixed video, and performs rights holder processing relating to the rights holder identified by the rights holder identifier that corresponds to a video composed by the video composition unit.

[0014] With this configuration, appropriate processing regarding the rights holder of the video can be carried out.

[0015] In addition, the information processing device of the sixth invention is an information processing device in which, compared to the fifth invention, the rights holder identifier corresponding to one video is a rights holder identifier corresponding to each of the multiple videos that were the basis for the one video, and the rights holder processing unit is an information processing device equipped with a reward means that performs reward processing to give rewards to each rights holder identified by the rights holder identifier corresponding to each of the multiple videos that were the basis for the one video.

[0016] With this configuration, it is possible to give a reward to the rights holder of the video captured by the mobile terminal.

[0017] In addition, the information processing device of the seventh invention is an information processing device in which, compared to the sixth invention, the reward means acquires video attribute values ​​corresponding to each of multiple videos that were the basis of a single video transmitted by a video transmission unit, uses the video attribute values ​​to determine rewards for each of multiple rights holders, and performs reward processing to award the rewards.

[0018] With this configuration, appropriate compensation can be given to the rights holder of the video captured by the mobile terminal.

[0019] In addition, the information processing device of the eighth invention is an information processing device in which, compared to the fifth invention, the rights holder processing unit is equipped with a first security means that performs a first security processing in which a single video composed by a video composition unit is associated with a set of attribute values ​​corresponding to each of a plurality of videos that were the basis of the single video and stored.

[0020] With this configuration, it is possible to preserve a single video made up of a plurality of images.

[0021] In addition, the information processing device of the ninth invention is an information processing device in which, compared to the fifth invention, the rights holder processing unit is equipped with a second security means that performs a second security process in which a single video composed by the video composition unit is stored in association with a rights holder identifier corresponding to each of the multiple videos that were the basis for the single video.

[0022] With this configuration, the rights holder of a video made up of a plurality of images can be determined to be the appropriate rights holder.

[0023] In addition, the information processing device of the tenth invention is an information processing device according to the fifth invention, which further includes a query receiving unit that receives from a user terminal a query regarding environmental information relating to the environment in which the mobile terminal shot the video, the video acquisition unit acquires a plurality of videos corresponding to the query received by the query receiving unit, and the rights holder processing unit includes a third security means that stores a single video composed by the video composition unit in association with a rights holder identifier that identifies the user of the user terminal.

[0024] With this configuration, the rights holder of one video can be identified from multiple images as the rights holder who needs that video.

[0025] In addition, the information processing device of the eleventh invention is an information processing device in which, compared to the eighth invention, the rights holder processing unit is equipped with a fourth security means that performs a fourth security process of storing security information including access information for accessing a video in a blockchain.

[0026] With this configuration, it is possible to preserve management information for moving images that need to be preserved.

[0027] Another information processing device of the present invention is an information processing device in which, compared to the first or second invention, the video acquisition unit sequentially acquires multiple videos that meet one or more of the following conditions: a location condition related to location information corresponding to the video, and a time condition related to time information corresponding to the video.

[0028] With this configuration, videos captured by a mobile terminal and videos captured by a fixed camera can be appropriately used to create a single useful video. Effect of the Invention

[0029] According to the present invention, a video captured by a mobile terminal and a video captured by a fixed camera can be used to construct and provide a single useful video. [Brief description of the drawings]

[0030] [Figure 1] Conceptual diagram of information system A in embodiment 1. [Diagram 2]Block diagram of Information System A [Diagram 3] Block diagram of the information processing device 1 [Figure 4] Block diagram of fixed terminal 3 [Diagram 5] A flowchart illustrating an example of the operation of the information processing device 1. [Figure 6] A flowchart for explaining an example of a process for acquiring a set of attribute values [Figure 7] A flowchart for explaining an example of the moving object attribute value tag acquisition process. [Figure 8] A flowchart illustrating an example of the video tag acquisition process. [Figure 9] A flowchart illustrating an example of the fourth maintenance process. [Figure 10] A flowchart illustrating an example of the moving image fusion process. [Figure 11] A flowchart illustrating an example of the image fusion process. [Figure 12] A flowchart illustrating an example of the video combining process. [Figure 13] A flowchart illustrating an example of the registered video search process. [Figure 14] A flowchart for explaining a first example of the unregistered video search process. [Figure 15] A flowchart for explaining a second example of the unregistered video search process. [Figure 16] A flowchart illustrating an example of the maintenance process. [Figure 17] A flowchart illustrating an example of the reward process. [Figure 18] A flowchart for explaining an example of the three-dimensional processing [Figure 19] A flowchart for explaining an example of the operation of the mobile terminal 2 [Figure 20] A flowchart illustrating an example of the video transmission process. [Figure 21] A flowchart illustrating an example of the terminal image acquisition process [Figure 22]10 is a flowchart for explaining an example of the operation of the fixed terminal 3. [Figure 23] A flowchart illustrating an example of the fixed video acquisition process. [Figure 24] A flowchart for explaining an example of the operation of the user terminal 4 [Diagram 25] A flowchart for explaining an example of a query generation process [Figure 26] FIG. 2 shows the mobile terminal management table. [Figure 27] A diagram showing the fixed terminal management table [Figure 28] A diagram showing the registered video management table [Figure 29] Overview of the computer system [Diagram 30] Block diagram of the computer system DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Hereinafter, an embodiment of an information processing device and the like will be described with reference to the drawings. Note that, in the embodiments, components with the same reference numerals perform similar operations, and therefore repeated description may be omitted.

[0032] (Embodiment 1) <Outline of First Embodiment>

[0033] In this embodiment, an information processing device is described that uses videos taken by one or more fixed cameras and videos acquired by cameras possessed by one or more mobile terminals to construct one video that is combined in time or that is integrated in space. The inquiry for selecting multiple videos is, for example, an inquiry using location information specified by a user terminal, an inquiry using one or more location information received from a target terminal to be watched, an inquiry using a destination set in a navigation terminal, or an inquiry using route information. The adoption conditions for the original videos that constitute one video include, for example, conditions related to one or more of the location information and time information.

[0034] In addition, a video that is linked in time is a video that is a combination of multiple original videos. When multiple original videos that are linked in time are shot, it is preferable that they are continuous in time, but they may be separated in time. For example, the original video may be shot on August 20th, and the original video may be shot on October 21st.

[0035] A spatially fused moving image is a moving image that is constructed by constructing a frame using some or all of the frames constituting each of a plurality of original moving images for each of the plurality of frames constituting each of the plurality of original moving images, and then connecting the constructed frames in time. Note that at least one of the frames constituting the single moving image is a frame that includes some or all of the frames of each of the plurality of original moving images.

[0036] In addition, in this embodiment, a prioritized type of video is determined between fixed videos captured by a fixed camera and moving videos captured by a camera of a mobile terminal, and the information processing device is described which preferentially adopts the prioritized type of video.

[0037] In the present embodiment, an information processing device that performs post-processing to convert one moving image into a stereoscopic 3D moving image and provides the stereoscopic 3D moving image will be described. Note that the one moving image to be post-processed may be only a part of the moving image that satisfies the processing conditions.

[0038] In addition, in this embodiment, a case will be described in which the usage method of a video differs according to a usage mode flag set by the rights holder of the original video of a video. Note that the usage mode flag is, for example, a flag indicating the presence or absence of a "non-provisional request for permanent use", and if the request is a "non-provisional request for permanent use", a process of inquiring of the rights holder is performed before transmitting the video.

[0039] In addition, in this embodiment, an information processing device that performs right holder processing, which is processing related to the right holder of one output video, will be described. The right holder processing is, for example, a compensation processing to be described later and various security processing to be described later.

[0040] In this embodiment, an information processing device that automatically receives and stores videos that meet the preservation conditions from a mobile terminal will be described.

[0041] In addition, in this embodiment, an information processing device that handles videos to which one or more tags obtained by analyzing the videos, etc. are added will be described. Note that the process of adding tags is performed, for example, by the information processing device, a mobile terminal, or a fixed terminal, but may be performed by other devices.

[0042] In addition, in this embodiment, an information processing device that performs processing for a user of a mobile terminal or a user who needs a video when a video desired by the user exists but cannot be transmitted will be described. Note that a case where a video cannot be transmitted is, for example, when the power of the mobile terminal is OFF.

[0043] In this embodiment, an information processing device that receives and utilizes movement information related to the start of movement from a mobile terminal will be described.

[0044] In this embodiment, a mobile terminal that transmits movement information to an information processing device will be described.

[0045] In this embodiment, a mobile terminal that transmits the latest set of attribute values ​​for the captured video to the information processing device when the mobile terminal finishes moving will be described.

[0046] Furthermore, in this embodiment, a mobile terminal and a fixed terminal will be described that acquire one or more tags by analyzing a moving image or the like and associates the tags with the moving image.

[0047] In this embodiment, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc. <Outline of Information System A>

[0048] 1 is a conceptual diagram of an information system A in this embodiment. The information system A includes an information processing device 1, one or more mobile terminals 2, one or more fixed terminals 3, one or more user terminals 4, one or more target terminals 5, and one or more navigation terminals 6.

[0049] The information processing device 1 is a server that provides a video to a user terminal 4 using videos transmitted from one or more mobile terminals 2 and one or more fixed terminals 3. The information processing device 1 is, for example, a cloud server or an ASP server, but the type is not important. The information processing device 1 may also be a device that constitutes a blockchain.

[0050] The video here refers to images captured by the mobile terminal 2 or the fixed terminal 3. The time interval between captures of the still images contained in the video does not matter. The video may be, for example, a video of 60 frames / second or a video of 30 frames / second, but it may also be a collection of still images captured with an interval of a predetermined time (for example, one minute) or more, or a collection of still images captured when a predetermined condition is met.

[0051] The mobile terminal 2 is a terminal installed on a moving object and is a terminal that shoots video. The mobile terminal 2 is, for example, a drive recorder, a smartphone, a tablet terminal, or a camera with a communication function. Installation usually means being fixed, but it may also be considered to include being in contact with or held by something. The mobile terminal 2 may also have a driving means such as an engine or a moving means such as wheels.

[0052] A mobile body is an object that moves, such as a terrestrial mobile device, a marine mobile device, an undersea mobile device, an aerial mobile device, a space mobile device, or a living being.

[0053] Examples of ground vehicles include automobiles, bicycles, rickshaws, toys, and other vehicles that are moved by human power, trains, steam locomotives, and other railroad vehicles, rides in amusement parks, and commercial vehicles in factories and other facilities. Ground vehicles may also be non-human vehicles, such as various robots for commercial or playground use, radio-controlled toys, and other such toys. Examples of motor vehicles include passenger cars, trucks, buses, taxis, and motorcycles.

[0054] Examples of marine vehicles include various types of ships, jet skis, surfboards, rowing boats, swim rings, rafts, and the like.

[0055] The underwater mobile device is, for example, a submarine, a diving robot, or diving equipment such as an aqualung.

[0056] Examples of the aerial vehicle include various airplanes, helicopters, gliders, parachutes, balloons, kites, and the like.

[0057] Space vehicles include, for example, rockets, space probes, and artificial satellites.

[0058] Examples of living things include humans and non-human moving objects. Examples of non-human moving objects include birds, mammals, reptiles, amphibians, fish, insects, and other living things.

[0059] The fixed terminal 3 is a terminal that is installed at a fixed location and has a photographing function. The fixed terminal 3 has a photographing means and a communication means. The photographing means is, for example, a so-called surveillance camera. The fixed terminal 3 has, for example, a surveillance camera installed in a city. The fixed terminal 3 has, for example, a surveillance camera installed in a building such as a private home or a building.

[0060] The user terminal 4 is a terminal used by a user. A user is someone who watches videos or needs videos. The user terminal 4 may have the functions of the mobile terminal 2. In other words, the user terminal 4 may be a terminal of a user who provides videos. The user terminal 4 may also be a navigation terminal 6 described later.

[0061] The target terminal 5 is a terminal that transmits location information. The target terminal 5 is, for example, a terminal provided in a monitoring target. The monitoring target is, for example, a living thing or an object. The living thing is, for example, a person or an animal such as a pet. The person is, for example, a child or an elderly person. The object to be monitored is, for example, an object that should not be stolen, such as a car, a motorcycle, or a bicycle. However, the monitoring target is not limited.

[0062] The navigation terminal 6 is a so-called navigation terminal that provides the user with guidance to the user's current location and the destination to which the user is heading. The navigation terminal 6 is a terminal that is installed in a ground mobile device or held by a person.

[0063] The information processing device 1 and one or more mobile terminals 2 are usually capable of communicating with each other via a network such as the Internet. The information processing device 1 and one or more fixed terminals 3 are usually capable of communicating with each other via a network such as the Internet. The information processing device 1 and one or more user terminals 4 are usually capable of communicating with each other via a network such as the Internet. The information processing device 1 and one or more target terminals 5 are usually capable of communicating with each other via a network such as the Internet. The information processing device 1 and one or more navigation terminals 6 are usually capable of communicating with each other via a network such as the Internet. The user terminal 4 and the target terminal 5, and the user terminal 4 and the navigation terminal 6 are usually capable of communicating with each other via a network such as the Internet.

[0064] Fig. 2 is a block diagram of an information system A in this embodiment. Fig. 3 is a block diagram of an information processing device 1. Fig. 4 is a block diagram of a fixed terminal 3. <Example of each device configuration>

[0065] The information processing device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit 14. The storage unit 11 includes a mobile terminal management unit 111 and a fixed terminal management unit 112. The receiving unit 12 includes a moving information receiving unit 121, an inquiry receiving unit 122, and a collection receiving unit 123. The processing unit 13 includes a moving information accumulation unit 131, a collection accumulation unit 132, a moving image acquisition unit 133, a moving image composition unit 134, a moving image processing unit 135, and a right holder processing unit 136. The moving image acquisition unit 133 includes a moving image acquisition means 1331, and a fixed moving image acquisition means 1332. The right holder processing unit 136 includes a first maintenance means 1361, a second maintenance means 1362, a third maintenance means 1363, a fourth maintenance means 1364, and a reward means 1365. The transmitting unit 14 includes a moving image transmission unit 141, a status transmission unit 142, and a needs transmission unit 143.

[0066] The mobile terminal 2 includes a moving storage unit 21, a moving receiving unit 22, a moving processing unit 23, and a moving transmitting unit 24. The moving processing unit 23 includes a photographing unit 231, a tag acquiring unit 232, and a moving information acquiring unit 233. The moving transmitting unit 24 includes a moving information transmitting unit 241, a moving video transmitting unit 242, and a collective transmitting unit 243.

[0067] The fixed terminal 3 includes a fixed storage unit 31, a fixed processing unit 33, a fixed receiving unit 32, and a fixed transmitting unit . The fixed processing unit 33 includes a fixed camera 331.

[0068] The user terminal 4 includes a user storage unit 41, a user reception unit 42, a user processing unit 43, a user transmission unit 44, a user reception unit 45, and a user output unit 46. <Details of components of information processing device 1>

[0069] Various types of information are stored in the storage unit 11 constituting the information processing device 1. The various types of information are, for example, mobile terminal information (to be described later), fixed terminal information (to be described later), an attribute value set (to be described later), movement information (to be described later), video, and management information (to be described later). The attribute value set is a movement attribute value set or a fixed attribute value set.

[0070] The management information is information used to monitor the person being watched. The management information includes a user identifier for transmitting video to the user terminal 4 of the person watching the person, and a target person identifier for identifying the target terminal 5 of the person being watched.

[0071] Mobile terminal management unit 111 stores one or more pieces of mobile terminal information. The mobile terminal information is information related to mobile terminal 2. The mobile terminal information includes information for identifying a video currently held by mobile terminal 2. The mobile terminal information includes a mobile terminal identifier and a set of movement attribute values. The mobile terminal information may be associated with a video. Furthermore, the mobile terminal identifier may be included in the set of movement attribute values.

[0072] The mobile terminal identifier is information for identifying the mobile terminal 2. The mobile terminal identifier may be a right holder identifier for identifying a right holder who is the user of the mobile terminal 2. The mobile terminal identifier is, for example, the ID of the mobile terminal 2, the user identifier of the user of the mobile terminal 2, the name of the mobile terminal 2, the IP address of the mobile terminal 2, or the MAC address of the mobile terminal 2.

[0073] A rights holder is someone who has some right related to a video. A rights holder is, for example, the owner of the video, the copyright holder of the video, the owner of the mobile terminal 2 that shot the video, or the owner of the fixed terminal 3 that shot the video. A rights holder is the initial rights holder of the video. A rights holder is usually the holder of the mobile terminal 2 or the owner of the fixed terminal 3, but it may be anyone who has the rights to the video shot by the mobile terminal 2 or the video shot by the owner of the fixed terminal 3.

[0074] The right holder identifier is an identifier of the right holder of the video. The right holder identifier may be a terminal identifier. The right holder identifier is, for example, a right holder ID, a right holder name, a right holder email address, or a right holder telephone number. The right holder ID is, for example, a user identifier.

[0075] A moving attribute value set is a set of one or more moving animation attribute values. A moving animation attribute value is an attribute value of a moving animation. A moving animation attribute value is, for example, environmental information. A moving animation attribute value is, for example, a tag. A moving animation attribute value is usually a dynamic attribute value that can change dynamically, but may also be a static attribute value that does not change dynamically.

[0076] The environmental information is information about the environment in which the video was shot. The environmental information is, for example, location information, direction information, camera information, time information, weather information, temperature information, and season information. The location information is information that specifies the shooting location. The shooting location is the location of the camera that shoots. The location information is, for example, (latitude, longitude) or (latitude, longitude, altitude). However, the location information may be an area identifier that specifies an area on a map, an address, a road identifier that specifies a road, a lane identifier that specifies a lane on a road, and the like. The direction information is information that specifies the direction of shooting. The direction information is, for example, an angle from true north. The camera information is information about the camera. The camera information is, for example, an angle of view and a resolution. The time information is information that specifies the time when the video was shot. The time when the video was shot may be a time around the time when the video was shot, and accuracy is not required. The time information is, for example, time, year / month / date / time, year / month / date / time / minute, year / month / date / time / minute / second, year / month / day, and month / day. In other words, the granularity of the time indicated by the time information does not matter. Weather information is information that specifies the weather at the location where the video was shot when it was shot. Examples of weather information are "sunny," "rain," "snow," and "cloudy." Temperature information is information that specifies the outside temperature at the location where the video was shot when it was shot. Examples of temperature information are "25 degrees" and "above 30 degrees." Season information is information that specifies the season at the location where the video was shot when it was shot. Examples of season information are "spring," "summer," "early summer," and "winter."

[0077] A tag is information that identifies the characteristics of a video. A tag is, for example, information resulting from analyzing a video. A tag is, for example, information resulting from analyzing one or more moving object attribute values. A tag is, for example, information resulting from analyzing multiple moving object attribute values ​​in time series.

[0078] A moving object attribute value is an attribute value related to a moving object. A moving object attribute value is, for example, information related to movement that a moving object can acquire while moving. A moving object attribute value is, for example, CAN (Controller Area Network) data, information indicating the use of an airbag. CAN data is, for example, speed, engine RPM, and brake status. A tag is, for example, "accident," "traffic jam," "dangerous driving," "speeding," and the name of an object shown in the video (for example, "person," "bear," or the name of a famous person).

[0079] The moving video attribute values ​​of position information, direction information, time information, weather information, temperature information, and season information are dynamic attribute values, whereas the camera information is a static attribute value.

[0080] The fixed terminal management unit 112 stores one or more pieces of fixed terminal information. The fixed terminal information is information related to the fixed terminal 3. The fixed terminal information includes a fixed terminal identifier and a fixed attribute value set. The fixed terminal information may be associated with a video. Furthermore, the fixed terminal identifier may be included in the fixed attribute value set.

[0081] The fixed terminal identifier is information for identifying the fixed terminal 3. The fixed terminal identifier may be a right holder identifier for identifying a right holder who is a user of the fixed terminal 3. The fixed terminal identifier is, for example, the ID of the fixed terminal 3, the name of the fixed terminal 3, the IP address of the fixed terminal 3, or the MAC address of the fixed terminal 3.

[0082] The fixed attribute value set is a set of one or more fixed video attribute values. A fixed video attribute value is an attribute value of a fixed video. Fixed video attribute values ​​are, for example, environmental information and tags. Fixed video attribute values ​​are dynamic attribute values ​​that can change dynamically, or static attribute values ​​that do not change dynamically. Note that the static attribute values ​​of the fixed terminal 3 are, for example, location information and camera information. The dynamic attribute values ​​of the fixed terminal 3 are, for example, time information, weather information, temperature information, and season information.

[0083] The receiving unit 12 receives various information and instructions from the mobile terminal 2, the fixed terminal 3, the user terminal 4, the target terminal 5, or the navigation terminal 6. The various information and instructions are, for example, movement information, position information, an inquiry, an attribute value set, and a video.

[0084] The receiving unit 12 receives moving images from the mobile terminal 2. Such moving images are called moving images. The receiving unit 12 receives moving images from the mobile terminal 2 for which the user's permission has been accepted, in accordance with the usage mode flag stored in the mobile terminal 2.

[0085] The usage mode flag is information that specifies the stance of the rights holder of the moving video when a third party uses the moving video. The usage mode flag is, for example, information indicating that "there is a desire for permanent use" or information indicating that "temporary use is acceptable." Note that permanent use is not temporary use. The usage mode flag is, for example, information indicating that "permission from the rights holder is obtained for third parties to use the moving video" or information indicating that "third parties may freely use the moving video."

[0086] For example, when the usage mode flag stored in the mobile terminal 2 indicates that there is a desire for non-temporary use, the receiving unit 12 receives the moving video from the mobile terminal 2 only if the mobile terminal 2 accepts the user's permission.

[0087] The receiving unit 12 receives the moving image captured by the fixed camera 331 from the fixed terminal 3. Such a moving image is called a fixed moving image.

[0088] It is preferable that a right holder identifier that identifies the right holder of the moving image is associated with the moving image received by the receiving unit 12. For example, one or more moving image attribute values ​​are associated with the moving image received by the receiving unit 12.

[0089] The video received by the receiving unit 12 is, for example, a video taken of a parking lot. The video received by the receiving unit 12 is, for example, a video showing a child who is the object of monitoring. However, the location from which the video received by the receiving unit 12 is obtained does not matter.

[0090] The movement information receiving unit 121 receives movement information from the mobile terminal 2 when the mobile terminal 2 starts moving. The start time is preferably immediately after the start, but may be after a predetermined time (for example, one minute) has elapsed since the start of the movement.

[0091] The movement information is information specifying that the mobile terminal 2 is moving. The information specifying that the mobile terminal 2 is moving may be information specifying that the mobile terminal 2 has started moving. The movement here may mean that the mobile terminal 2 is about to start moving, or that the mobile terminal 2 is currently moving. The movement information is, for example, a movement start flag and a terminal identifier. The movement start flag is information indicating that the mobile terminal 2 is about to start moving. The movement start means, for example, that the engine has been turned on and that the mobile terminal 2 has started shooting. The terminal identifier is an identifier of the mobile terminal 2 that is starting to move. The terminal identifier may be the same as the right holder identifier.

[0092] The inquiry receiving unit 122 receives an inquiry about environmental information. The inquiry receiving unit 122 usually receives the inquiry from the user terminal 4. The inquiry receiving unit 122 may receive the inquiry from the target terminal 5 or the navigation terminal 6.

[0093] An inquiry about environmental information is an inquiry that includes environmental information as a condition. The environmental information here is information about the environment in which the mobile terminal 2 or the fixed terminal 3 captured a video. The inquiry is a request for a video captured by the mobile terminal 2 or the fixed terminal 3. The query is, for example, SQL, but the format, data structure, etc., of the query are not important.

[0094] The inquiry receiving unit 122 receives an inquiry including, for example, location information. The inquiry receiving unit 122 receives an inquiry including, for example, location information from the user terminal 4.

[0095] The inquiry receiving unit 122 sequentially receives, for example, each of the multiple pieces of location information transmitted from the target terminal 5. The inquiry receiving unit 122 may receive each of the multiple pieces of location information from the target terminal 5, or may receive the information from the user terminal 4 or the like that has been received from the target terminal 5.

[0096] The inquiry receiving unit 122 receives, for example, an inquiry including location information that specifies a destination set in the navigation terminal 6. The inquiry receiving unit 122 may receive the location information from the navigation terminal 6, or may receive the location information from the user terminal 4 that received it from the navigation terminal 6, or may receive the location information from the user terminal 4 that transmitted the location information to the navigation terminal 6 for setting the destination.

[0097] The inquiry receiving unit 122 receives an inquiry including route information including a plurality of pieces of location information, for example. The inquiry receiving unit 122 receives an inquiry including route information from the user terminal 4 or the navigation terminal 6, for example.

[0098] The route information contained in the inquiry is, for example, information specifying the route traveled by the user watching the video, or route information set in the navigation terminal 6 or the user terminal 4 having a navigation function. It is preferable that the route information has time information corresponding to each of the multiple pieces of position information. The time information is, for example, information specifying the time when the user was at the position specified by the position information. It is to be noted that the distance interval between each of the multiple pieces of position information contained in the route information does not matter. The time when the user was at the position specified by the position information is the time when the video was shot at that position.

[0099] The set receiving unit 123 receives the movement attribute value set from the mobile terminal 2. It is preferable that the set receiving unit 123 receives the movement attribute value set from the mobile terminal 2 when the movement of the mobile terminal 2 ends. The movement attribute value set here is information that identifies a video stored in the mobile terminal 2. Note that the movement attribute value set received by the set receiving unit 123 is associated with an identifier of the mobile terminal 2 (e.g., a terminal identifier, a rights holder identifier).

[0100] The set receiving unit 123 receives, for example, a fixed attribute value set associated with an identifier of the fixed terminal 3 from the fixed terminal 3. The set receiving unit 123 receives, for example, dynamic attribute values ​​that can change dynamically from among the fixed video attribute values ​​in the fixed attribute value set. The dynamic attribute values ​​here are, for example, time information, weather information, temperature information, season information, and tags. It is preferable that the set receiving unit 123 does not receive position information and camera information from the fixed terminal 3. The position information and camera information of the fixed terminal 3 are static attribute values, and are preferably stored in the fixed terminal management unit 112 in advance.

[0101] The processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by a movement information storage unit 131, a collection storage unit 132, a moving image acquisition unit 133, a moving image composition unit 134, a moving image processing unit 135, and a right holder processing unit 136.

[0102] The movement information storage unit 131 stores the movement information received by the movement information receiving unit 121 in association with the mobile terminal 2. Association with the mobile terminal 2 means association with, for example, a right holder identifier or a terminal identifier.

[0103] The set storage unit 132 stores the mobile attribute value set received by the set receiving unit 123 in the mobile terminal management unit 111. The set storage unit 132 normally stores the attribute value set in association with the mobile terminal 2 that has transmitted the mobile attribute value set. Association with the mobile terminal 2 means, for example, association with the right holder identifier or terminal identifier of the mobile terminal 2.

[0104] The set storage unit 132 stores the fixed attribute value set received by the set receiving unit 123 in the fixed terminal management unit 112. The set storage unit 132 normally stores the attribute value set in association with the fixed terminal 3 that has transmitted the fixed attribute value set. Association with the fixed terminal 3 means, for example, association with the right holder identifier or terminal identifier of the fixed terminal 3.

[0105] The video acquisition unit 133 acquires moving videos captured by one or more mobile terminals 2. In addition, the video acquisition unit 133 acquires fixed videos captured by one or more fixed terminals 3. The video acquisition unit 133 usually acquires moving videos and fixed videos received by the receiving unit 12.

[0106] A moving video acquired by the video acquisition unit 133 is usually associated with a moving attribute value set. The moving attribute value set includes one or more pieces of environment information. It is preferable that the one or more pieces of environment information include position information that specifies the shooting position or time information that specifies the shooting time. It is also preferable that a right holder identifier or a terminal identifier of the mobile terminal 2 is associated with the moving video.

[0107] A fixed moving image acquired by the moving image acquisition unit 133 is usually associated with a fixed attribute value set. The fixed attribute value set includes one or more pieces of environmental information. It is preferable that the one or more pieces of environmental information include location information that specifies the shooting location or time information that specifies the shooting time. It is also preferable that the fixed moving image is associated with a right holder identifier or a terminal identifier of the fixed terminal 3.

[0108] For example, when there are multiple videos that satisfy the adoption conditions and include a fixed video and a moving video, the video acquisition unit 133 acquires a video that corresponds to a priority type. For example, when there are multiple videos that satisfy the adoption conditions and the fixed video and the moving video correspond to position information that indicates a position close enough to satisfy the first position condition, the video acquisition unit 133 acquires a video that corresponds to a priority type from among the fixed video and the moving video.

[0109] For example, when there are multiple videos that satisfy the adoption conditions, the video acquisition unit 133 may determine the video to be finally adopted by using one or multiple video attribute values ​​of the multiple videos. For example, when there are multiple videos that satisfy the adoption conditions, the video acquisition unit 133 determines the video with the highest resolution.

[0110] The adoption conditions are conditions for a video that is a source of a video. The adoption conditions preferably include one or more of a position condition or a time condition. The position condition is a condition related to the position information associated with the video. For example, the position condition is a video associated with position information whose distance from the position indicated by the position information included in the inquiry received by the inquiry receiving unit 122 is within a threshold or less than a threshold. The time condition is a condition related to the time information associated with the video. For example, the time condition is a video associated with time information indicating the time closest to the current time. For example, the position condition is a first position condition or a second position condition. Note that the one video is a video that is ultimately provided to the user, such as a combined video or a fusion video.

[0111] The first location condition is a condition related to location information. The first location condition is that the location information is in an approximate relationship with respect to a location specified by the target location information. The approximate relationship is closeness, for example, a distance within a threshold, a distance less than a threshold, a travel time within a threshold, or a travel time less than a threshold. The target location information is, for example, location information included in an inquiry, received location information, or location information included in route information included in an inquiry.

[0112] The first position condition is usually a condition that assumes that an image of a position specified by the target position information is shown. In other words, the first position condition may include conditions other than position information. The first position condition may be a condition related to position information and time information. For example, the first position condition is that the position information and time information satisfy an approximation condition for a pair of target position information and time information. The position information and time information that satisfy an approximation condition for a pair of target position information and time information are, for example, position information indicating a position whose distance is within a threshold, whose distance is less than a threshold, whose travel time is within a threshold, or whose travel time is less than a threshold, with respect to a position specified by the target position information, and time information whose time difference is within a threshold or less than a threshold with respect to a time specified by the target time information. The first position condition may be a condition related to position information and a traveling direction of a moving object. The first position condition may be a condition related to position information and direction information of a camera.

[0113] The second position condition is position information that is in an approximate relationship with respect to the reference position or reference area of ​​the comparison target. The approximate relationship here means that the distance from the reference position is within a threshold, the distance from the reference position is less than a threshold, or the position is within the reference area. The reference area is, for example, a specific area such as a parking lot. The comparison target is a location where the video is to be synthesized, for example, a specific location such as a parking lot.

[0114] The priority type is information for specifying a type of moving image that is to be prioritized between a fixed moving image and a moving image. The priority type is, for example, either a "fixed moving image" or a "moving moving image". It is preferable that the priority type is a "fixed moving image".

[0115] The video acquisition unit 133 sequentially acquires a plurality of videos that match one or more of the position condition and the time condition, for example. The plurality of videos includes one or more moving videos and one or more fixed videos.

[0116] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to the inquiries received by the inquiry receiving unit 122 .

[0117] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to the location information that satisfies a first location condition using the location information received by the inquiry receiving unit 122 .

[0118] For example, the video acquisition unit 133 sequentially uses each of the multiple pieces of location information received by the inquiry receiving unit 122 to sequentially acquire multiple videos corresponding to the location information that satisfies the first location condition.

[0119] The video acquisition unit 133 sequentially acquires a plurality of videos corresponding to the location information that satisfies the first location condition using the location information received by the inquiry receiving unit 122, for example.

[0120] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to the position information that satisfies a plurality of first position conditions using each of the plurality of pieces of position information included in the route information.

[0121] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to the position information that satisfies a first position condition using each of a plurality of pieces of position information and time information included in the route information.

[0122] The video acquisition unit 133 acquires a plurality of videos in which, for example, position information associated with each of the plurality of videos satisfies a second position condition, and time information associated with each of the plurality of videos satisfies a time condition. Note that the second position condition is, for example, that the position information is information indicating a position within a predetermined area (for example, a specific parking lot). The time condition is, for example, that the time information is within a threshold value (for example, within 30 seconds) from the current time or a time before the threshold value.

[0123] The multiple moving images acquired by the moving image acquisition unit 133 usually include one or more moving images and one or more fixed images. However, the multiple moving images acquired by the moving image acquisition unit 133 may all be moving images or all be fixed images.

[0124] It is preferable that the moving image acquiring unit 133 acquires a moving image that satisfies the storage condition. It is also preferable that the moving image acquired by the moving image acquiring unit 133 is a moving image that corresponds to an attribute value set and a right holder identifier.

[0125] The storage condition is a condition for storing a video. For example, the storage condition is to match the condition specified by the inquiry. For example, the storage condition is to match the conservation condition described below. The storage condition may be the same as the adoption condition.

[0126] The device in which the video to be acquired is stored may be the mobile terminal 2, the information processing device 1, or another device (not shown). The other device may be, for example, a device that constitutes a blockchain.

[0127] For example, when the inquiry receiving unit 122 receives an inquiry, the video acquiring unit 133 acquires one or more videos corresponding to the inquiry from the mobile terminal 2 or the fixed terminal 3. For example, the video acquiring unit 133 acquires from the mobile terminal 2 or the fixed terminal 3 one or more videos that are paired with an attribute value set that satisfies a condition related to environmental information included in the inquiry.

[0128] The video acquisition unit 133, for example, interprets the inquiry, detects a fixed terminal 3 that can provide a video that matches the inquiry, and receives a video shot by the fixed terminal 3 from the fixed terminal 3. For example, the video acquisition unit 133 acquires a terminal identifier that pairs with location information that matches a first location condition for location information included in the inquiry from the fixed terminal management unit 112, transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier, and receives a video corresponding to the video transmission instruction from the fixed terminal 3. For example, the video acquisition unit 133 acquires a terminal identifier that pairs with location information and direction information that match a first location condition for location information and direction information included in the inquiry from the fixed terminal management unit 112, transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier, and receives a video from the fixed terminal 3.

[0129] Note that a video transmission instruction is usually an instruction to transmit a video currently being shot, but may also be an instruction to transmit a video previously shot by fixed terminal 3 and stored in fixed terminal 3 or another device not shown. Also, a video transmission instruction may be an inquiry. In other words, the inquiry may be an instruction to transmit a video currently being shot.

[0130] The video acquisition unit 133, for example, transmits a query to one or more mobile terminals 2, and receives one or more videos corresponding to the query from one or more mobile terminals 2. Such processing is called unregistered video search processing. The unregistered video search processing is processing for acquiring a video that matches a condition from unregistered videos stored in the mobile terminal 2. The unregistered video search processing is processing for acquiring a video that matches a query from unregistered videos stored in the mobile terminal 2, for example.

[0131] For example, if the video acquisition unit 133 interprets a query and determines that there is no fixed terminal 3 that can provide a video that matches the query, it is preferable for the video acquisition unit 133 to transmit the query to one or more mobile terminals 2 and receive one or more videos corresponding to the query from the one or more mobile terminals 2.

[0132] For example, the video acquisition unit 133 acquires a video corresponding to the inquiry from among videos taken by one or more mobile terminals 2 corresponding to the movement information. The video acquisition unit 133 transmits an inquiry to each of the one or more mobile terminals 2 corresponding to the movement information, and receives a video that matches the inquiry from the mobile terminal 2. Note that the mobile terminal 2 corresponding to the movement information is a mobile terminal 2 that is moving and is capable of transmitting a video.

[0133] For example, when the inquiry receiving unit 122 receives an inquiry, the video acquisition unit 133 determines one or more attribute value sets that match the inquiry, and acquires videos corresponding to each of the one or more attribute value sets from the storage unit 11.

[0134] The video acquisition unit 133, for example, refers to the mobile terminal management unit 111, determines one or more videos corresponding to the inquiry, and acquires the one or more videos from the mobile terminal 2.

[0135] The process of determining a video corresponding to a query by referring to the mobile terminal management unit 111 is to determine an attribute value set that matches the query from among one or more attribute value sets held by the mobile terminal management unit 111. This process is called a registered video search process. The registered video search process is a process of searching registered videos for a video that matches a condition. The registered video search process is, for example, a process of searching registered videos for a video that matches the query. Note that a registered video is a video for which a first preservation process, described below, or a second preservation process, described below, has been performed.

[0136] The video acquisition unit 133, for example, refers to the mobile terminal management unit 111, determines the video corresponding to the inquiry, determines whether the video is in a state where it can be transmitted from the mobile terminal 2, and acquires the video from the mobile terminal 2 only if it is in a state where it can be transmitted.

[0137] For example, when the video acquisition unit 133 attempts to communicate with the mobile terminal 2 and can receive information from the mobile terminal 2, it determines that the mobile terminal 2 is in a state in which it can transmit video. For example, the video acquisition unit 133 determines whether or not mobile information corresponding to the mobile terminal 2 is stored in the storage unit 11, and if it is stored, it determines that the mobile terminal 2 is in a state in which it can transmit video. Note that a state in which it is possible to transmit video is, for example, a state in which the power supply of the mobile terminal 2 or a mobile object corresponding to the mobile terminal 2 is ON (for example, a state in which the engine of a car, which is a mobile object, is ON). In addition, the method by which the video acquisition unit 133 determines whether or not it is in a state in which it can transmit video is not important.

[0138] The video acquisition unit 133, for example, refers to the fixed terminal management unit 112, determines one or more fixed videos corresponding to the inquiry, and acquires the fixed videos from the fixed terminal management unit 112.

[0139] For example, the moving image acquiring unit 133 acquires a moving image associated with a mobile attribute value set that satisfies the preservation condition from the mobile terminal 2. For example, the moving image acquiring unit 133 acquires a moving image associated with a fixed attribute value set that satisfies the preservation condition from the fixed terminal 3. By this process, the moving image can be acquired automatically.

[0140] A conservation condition is a condition for storing video. A conservation condition is a condition related to an attribute value set. It is preferable that a conservation condition is a condition related to one or more tags. For example, a conservation condition is that the attribute value set includes a tag indicating "accident", that the attribute value set includes a tag indicating "traffic jam", or that the attribute value set includes a tag indicating a specific location. A tag indicating a specific location is, for example, the name of a specific parking lot, a specific place name, the name of a specific scenic spot, or the name of a specific landscape.

[0141] The preservation condition is that the moving image corresponds to an attribute value set that satisfies a predetermined condition for an attribute value set corresponding to one or more moving images stored in a predetermined device or area (e.g., storage unit 11). The predetermined condition is, for example, that an attribute value possessed by the attribute value set does not exist in the attribute value set corresponding to one or more moving images stored in the predetermined device or area.

[0142] The video acquisition unit 133 may acquire a processed image acquired by processing a video captured by the mobile terminal 2 or the fixed terminal 3, for example. Such a processed image is, for example, an around-view image. An around-view image is an image displayed on an around-view monitor. An around-view image is an image that appears as if an area including a moving object was captured from directly above.

[0143] The moving video acquisition means 1331 performs a process of acquiring a moving video among the processes performed by the video acquisition section 133 .

[0144] The fixed moving image acquisition means 1332 performs a process of acquiring a fixed moving image, out of the processes performed by the moving image acquisition section 133 .

[0145] The moving image constructing unit 134 combines the moving image and the fixed image in terms of time or space to construct one moving image.

[0146] The moving image composing unit 134 composes one moving image using the multiple moving images acquired by the moving image acquiring unit 133. Compiling one moving image may mean sequentially passing parts of the moving images that are seen as one moving image by the user to the moving image transmitting unit 141. The multiple moving images include moving images and fixed moving images. Also, one moving image may be a collection of multiple partial moving images with a transmission interval.

[0147] The following describes further specific examples of the processing by the moving image composer 134. The moving image composer 134 performs (1) temporal combination of moving images, or (2) spatial combination of moving images, or both (1) and (2). (1) Temporal video merging

[0148] The moving image constructing section 134 constructs one moving image by, for example, temporally combining one or more moving images and one or more fixed images.

[0149] The moving image composing unit 134, for example, temporally combines a plurality of moving images having different time information corresponding to each of the plurality of moving images acquired by the moving image acquiring unit 133 to compose one moving image. Combining a plurality of moving images temporally to compose one moving image may mean passing the plurality of moving images in order to the moving image transmitting unit 141. In other words, it is sufficient that the result of combining a plurality of moving images temporally to compose one moving image is that the user sees the moving images as one moving image.

[0150] For example, the moving image composing unit 134 combines the multiple moving images acquired by the moving image acquiring unit 133 in the time order corresponding to the moving images to compose one moving image. For example, the moving image composing unit 134 acquires a part of each of the multiple moving images acquired by the moving image acquiring unit 133, and combines the parts of the moving images in order to compose one moving image. Combining multiple moving images in time usually means combining parts of the moving images taken by the mobile terminal 2 or the fixed terminal 3 in order. Combining parts of the moving images in order may also mean passing parts of the moving images to the moving image transmitting unit 141 in order. Combining in the time order corresponding to the moving images means combining the moving images in the time order indicated by the time information corresponding to the moving images, or combining the moving images in the order in which the moving images were received. (2) Spatial video integration

[0151] The moving image composing unit 134 composes one moving image, for example, by spatially fusing a portion of each of the multiple moving images having different position information corresponding to each of the multiple moving images acquired by the moving image acquiring unit 133. The moving image composing unit 134 composes one frame using, for example, some or all of the frames constituting each of the multiple moving images acquired by the moving image acquiring unit 133, and connects the multiple frames in time to compose one moving image.

[0152] For example, the moving image constructing section 134 constructs one moving image having a plurality of frames by spatially fusing at least a portion of the frames of the moving moving image and at least a portion of the frames of the fixed moving image.

[0153] The moving image composing unit 134 composes one moving image, for example, by processing each of the multiple moving images acquired by the moving image acquiring unit 133. For example, the moving image composing unit 134 spatially connects an overhead frame obtained by combining frames of a moving image and a fixed image to compose one frame, and composes one moving image having one frame.

[0154] The moving image composition unit 134, for example, synthesizes a plurality of around-view images to compose a wide-area around-view image. Note that around-view is a registered trademark.

[0155] The multiple original moving images used by the moving image composer 134 to compose one moving image include moving moving images and fixed moving images.

[0156] The process of spatially fusing frames included in each of a plurality of moving images is, for example, the following processes (a) and (b). (a) Image processing-based methods

[0157] The moving image composing unit 134 performs, for example, a process of adjusting the direction and scale of each of the multiple frames that are the subject of the spatial connection process. Next, the moving image composing unit 134 detects, for example, the same area in each of the multiple frames. Next, the moving image composing unit 134 performs, for example, a process of overlapping multiple frames having the same area to compose one wide-area frame. Note that it is possible to detect the same area in each of the multiple frames using publicly known technology. (b) Machine learning-based methods

[0158] The video composition unit 134, for example, provides a plurality of frames and a learning model to a module that performs machine learning prediction processing, executes the module, and acquires one wide-area frame.

[0159] The learning model is obtained by providing multiple teacher data sets, which have multiple frames as explanatory variables and a wide-area frame composed of the multiple frames as the objective variable, to a module that performs machine learning learning processing, executing the module, and so on.

[0160] The learning model may be called a learner, a classifier, a classification model, etc. The machine learning algorithm is preferably deep learning, but may be random forest, etc. For machine learning, various machine learning functions such as the TensorFlow library and the random forest module of the R language, and various existing libraries may be used.

[0161] The moving image processing unit 135 acquires a stereoscopic three-dimensional moving image from at least a part of the one moving image composed by the moving image composition unit 134.

[0162] A stereoscopic 3D video is a 3D video composed of illustrated frames. Note that the technology for acquiring stereoscopic 3D video from camera images is publicly known, so a detailed explanation is omitted (see the Internet URL "https: / / xtech.nikkei.com / atcl / nxt / column / 18 / 01883 / 00004 / ").

[0163] The moving image processing section 135 determines, for example, a partial moving image that satisfies the processing conditions from one moving image constructed by the moving image constructing section 134, and obtains a stereoscopic three-dimensional moving image from the partial moving image.

[0164] The processing conditions are conditions for identifying partial videos from which a stereoscopic 3D video is obtained. The processing conditions are, for example, conditions based on one or more video attribute values, or conditions based on tags obtained by analyzing the video. For example, the processing conditions are "1 minute before and after the frame corresponding to 'tag=accident'" and "weather=snow". Note that a partial video here is a video that constitutes one video.

[0165] The video processing unit 135, for example, provides each frame of the video and a learning model to a machine learning prediction module, executes the prediction module, and acquires a stereoscopic 3D image that constitutes a stereoscopic 3D video. The learning model here is information acquired by providing a plurality of teacher data including a captured still image and an image obtained by converting the still image into a stereoscopic 3D image to a machine learning learning module, and executing the learning module. The machine learning algorithm is preferably deep learning, but may be other algorithms such as random forest.

[0166] The video processing unit 135 creates a three-dimensional illustration for each frame of the video by using known image processing, for example, to create a stereoscopic three-dimensional video.

[0167] The rights holder processing unit 136 performs rights holder processing.

[0168] The right holder process is a process related to the rights of one video. The right holder process is, for example, a process related to a right holder identified by a right holder identifier corresponding to one video composed by the video composition unit 134. The right holder process is, for example, a process related to a right holder identified by a right holder identifier corresponding to a stereoscopic 3D video acquired by the video processing unit 135. The right holder process is, for example, a first security process described later, a second security process described later, a third security process described later, a fourth security process described later, and a remuneration process described later.

[0169] The right holder processing unit 136 performs right holder processing, which is processing in response to the transmission of a moving image by the moving image transmission unit 141 and is processing related to a right holder identified by a right holder identifier associated with the moving image. Note that the moving image transmitted by the moving image transmission unit 141 is a moving image composed by the moving image composition unit 134 or a stereoscopic three-dimensional moving image acquired by the moving image processing unit 135.

[0170] The right holder identifier associated with one moving image is, for example, a right holder identifier associated with each of the multiple moving images that are the basis of the constructed moving image, or an identifier of a user who requested the one moving image.

[0171] The right holder processing unit 136, for example, stores the video acquired by the video acquisition unit 133 in association with a right holder identifier. It is preferable that the right holder processing unit 136 stores only the video acquired by the video acquisition unit 133 as satisfying the storage condition. It is also preferable that the right holder processing unit 136 does not store the video that does not satisfy the storage condition.

[0172] The rights holder identifier is the rights holder identifier of the rights holder of the original video, the rights holder identifier of the rights holder of one video, or the rights holder identifier of the rights holder of the stereoscopic 3D video.

[0173] For example, when the video acquisition unit 133 determines that the mobile terminal 2 holding the video corresponding to the inquiry is in a state where it cannot transmit the video, the rights holder processing unit 136 acquires status information relating to that state.

[0174] The status information is information about a state in which a video cannot be sent. The status information is, for example, information indicating the state of the mobile terminal 2 that holds the video. For example, the status information is "The video exists, but it is not currently in a state where it can be sent to you," or "The video exists on Mr. XX's mobile terminal, but it is not currently in a state where it can be sent to you." The status information is, for example, information indicating that the power of the mobile terminal 2 is OFF, or information indicating that the power of the mobile terminal 2 is ON.

[0175] For example, when the video acquisition unit 133 determines that the mobile terminal 2 holding the video corresponding to the inquiry is in a state in which the video cannot be transmitted, the right holder processing unit 136 acquires the needs information.

[0176] The need information is information indicating that there is a need for a video. For example, the need information is "Your XXX video is being requested by other users" or "Your XXX video is being requested by other users for XX yen."

[0177] The first preservation means 1361 performs a first preservation process of storing one video constructed by the video constructing unit 134 or a stereoscopic 3D video acquired by the video processing unit 135 in association with an attribute value set corresponding to each of the multiple videos that are the source of the video. The first preservation means 1361 may also perform a first preservation process of storing a video received from the mobile terminal 2 or the fixed terminal 3 in association with an attribute value set corresponding to the video. The attribute value set corresponding to the video is a moving attribute value set corresponding to a moving video or a fixed attribute value set corresponding to a fixed video.

[0178] The second preservation means 1362 performs a second preservation process in which a video composed by the video composition unit 134 or a stereoscopic 3D video acquired by the video processing unit 135 is stored in association with a rights holder identifier corresponding to each of the multiple videos that were the source of the video.

[0179] In addition, the first preservation means 1361 or the second preservation means 1362 may store a video constructed by the video construction unit 134 or a stereoscopic 3D video acquired by the video processing unit 135 in association with a set of attribute values ​​corresponding to each of the multiple videos that were the basis for the video, and with a rights holder identifier corresponding to each of the multiple videos that were the basis for the video.

[0180] The third security means 1363 stores the one video composed by the video composition unit 134 or the stereoscopic 3D video acquired by the video processing unit 135 in association with a right holder identifier that identifies the user of the user terminal 4. Note that the user of the user terminal 4 here is the person who views the one video or the stereoscopic 3D video. The user terminal 4 here is, for example, the terminal that sent the inquiry.

[0181] In addition, the destination for storing the one video or the stereoscopic 3D video is, for example, the storage unit 11, but it may be another device, another device constituting a blockchain, etc. In addition, the stored video is usually associated with a video identifier that identifies the video.

[0182] The fourth security means 1364 performs a fourth security process to store security information including access information for accessing one of the stored moving images or stereoscopic 3D moving images. The order of storing the moving images and the fourth security process for the security information corresponding to the moving images does not matter.

[0183] The fourth maintenance means 1364 performs a fourth maintenance process, for example, of storing maintenance information including access information for accessing one stored video composed by the video composition unit 134 in the blockchain.

[0184] The fourth security means 1364 performs a fourth security process of, for example, storing security information including access information for accessing the stored stereoscopic 3D video acquired by the video processing unit 135 in the blockchain.

[0185] In addition, it is preferable that the fourth maintenance means 1364 accumulates the maintenance information in a blockchain. That is, it is preferable that the fourth maintenance means 1364 accumulates the maintenance information in a distributed ledger of the blockchain. It is preferable that the fourth maintenance means 1364 registers the maintenance information as an NFT (non-fungible token). It is preferable that the fourth maintenance means 1364 registers the maintenance information in a distributed file system in an IPFS (Inter Planetary File System) network.

[0186] The preservation information is information for ensuring the originality of a video. It can be said that the preservation information is heading information for a video. The preservation information is, for example, access information and one or more video attribute values. It is preferable that the preservation information has, for example, one or more right holder identifiers. When the preservation information has multiple right holder identifiers, the right holder of the video may be shared, and the multiple right holder identifiers may be right holder history information. The right holder history information is a collection of right holder identifiers and is information indicating the history of the right holder. The fourth preservation process can ensure the originality of the preservation information of the registered video. By ensuring the originality of the preservation information, the originality of the video corresponding to the preservation information is also guaranteed. The access information is information for accessing a video. The access information is information that identifies the storage destination of the video. The access information is, for example, a URL or a URI.

[0187] It is preferable that the security information includes information (which may be called a flag) indicating whether the video can be provided to a third party. The flag may be, for example, information indicating that the video can be viewed by a third party, information indicating that the video can be sold, or information indicating that neither viewing nor selling is permitted.

[0188] The remuneration means 1365 performs, for example, remuneration processing for each right holder identified by a right holder identifier corresponding to each of a plurality of moving images that are the basis of one moving image composed by the moving image composition unit 134 .

[0189] The remuneration means 1365 performs, for example, remuneration processing for each right holder identified by a right holder identifier corresponding to each of the multiple moving images that are the basis of the stereoscopic 3D moving image acquired by the moving image processing unit 135.

[0190] The reward process is a process of giving a reward. The reward process is, for example, a process of increasing points managed in pairs with one or more right holder identifiers corresponding to a video. The reward process is, for example, a process of making a payment to a right holder identified by one or more right holder identifiers corresponding to a video. The reward process is, for example, a process of transmitting a video or other content to a user terminal 4 of a right holder identified by one or more right holder identifiers corresponding to a video. The reward process may be of any kind as long as a merit is given to a right holder identified by one or more right holder identifiers corresponding to a video. The reward may be, for example, money, points, goods, content, or the like, and the content may be of any kind.

[0191] It is preferable that the reward means 1365 performs reward processing, which is a process of acquiring one or more video attribute values ​​corresponding to each of multiple videos that were the basis for the single video transmitted by the video transmission unit 141, determining rewards for each of multiple rights holders using the one or more video attribute values, and awarding the rewards.

[0192] Here, the one or more moving image attribute values ​​are, for example, the amount of data of the moving image, the duration of the moving image, the number of frames of the moving image, and the resolution of the moving image.

[0193] It is preferable that the reward means 1365 performs a reward process, which is a process of acquiring a reward amount corresponding to a service identifier that identifies a service provided to a target video, and giving a reward corresponding to the reward amount. The service identifier is, for example, "view" or "purchase." In such a case, for example, the storage unit 11 stores the reward amount corresponding to the service identifier, or information for determining the reward amount corresponding to the service identifier.

[0194] The reward means 1365 performs reward processing, which is a process of acquiring a reward amount using, for example, one or more pieces of information among one or more video attribute values ​​and one or more service identifiers, and giving a reward corresponding to the reward amount. In such a case, for example, an arithmetic expression or a correspondence table corresponding to each of a plurality of service identifiers is stored in the storage unit 11. The arithmetic expression is an expression for calculating the reward amount using one or more video attribute values ​​as parameters. The correspondence table is a table having a plurality of correspondence information for managing the reward amount corresponding to one or more video attribute values.

[0195] The remuneration means 1365 normally performs a process of charging a remuneration to a user who has enjoyed a service related to the target video.

[0196] The process of charging the reward is, for example, a process of charging the amount of reward acquired. The process of charging the reward is, for example, a process of charging the amount of reward acquired and the amount of profit obtained by the management of the information processing device 1. The process of charging the reward is, for example, a process of deducting points corresponding to the user who enjoyed the service, a settlement process using the credit card number of the user, etc.

[0197] The transmission unit 14 transmits various types of information or instructions to the mobile terminal 2, the fixed terminal 3, the user terminal 4, or the navigation terminal 6. The various types of information or instructions are, for example, video, inquiries, status information, and needs information.

[0198] For example, the moving image transmitting unit 141 transmits one moving image composed by the moving image composing unit 134. For example, the moving image transmitting unit 141 transmits a stereoscopic 3D moving image instead of or in addition to the one moving image.

[0199] The video transmitting unit 141 transmits one video or a stereoscopic 3D video to the user terminal 4. It is preferable that the video transmitting unit 141 transmits a video in response to receiving an inquiry. The video transmitting unit 141 transmits a video to, for example, the user terminal 4 or the navigation terminal 6. Transmitting one video may mean that the video transmitting unit 141 sequentially transmits parts of the one video that have been handed over to it.

[0200] The video transmitting unit 141 transmits, for example, the video acquired by the video acquiring unit 133 to the user terminal 4.

[0201] When the video cannot be transmitted, the status transmission unit 142 transmits status information relating to the status to the user terminal 4. For example, the status transmission unit 142 transmits status information acquired by the right holder processing unit 136 to the user terminal 4. Whether the video cannot be transmitted is determined by, for example, the video acquisition unit 133.

[0202] When it is determined that a video cannot be transmitted, the needs transmission unit 143 transmits needs information indicating that there is a need for the video to the user corresponding to the video. For example, the needs transmission unit 143 transmits the needs information acquired by the right holder processing unit 136 to the user corresponding to the video.

[0203] Sending to a user corresponding to a video means, for example, sending the needs information by email to a notification destination indicated by the notification destination information paired with the rights holder identifier corresponding to the video. Sending to a user corresponding to a video means, for example, sending the needs information by short message to a telephone number destination paired with the rights holder identifier corresponding to the video. Sending to a user corresponding to a video means, for example, sending the needs information to a mobile terminal 2 paired with the rights holder identifier corresponding to the video. The user corresponding to a video is usually the rights holder of the video. The notification destination information is, for example, an email address, a telephone number, an IP address, or an ID. <Details of components of mobile terminal 2>

[0204] Various types of information are stored in the movement storage unit 21 constituting the mobile terminal 2. The various types of information are, for example, a moving video, a moving attribute value set, a right holder identifier, movement information indicating the start of movement, a pair of an attribute value tag condition and a tag, a pair of a video tag condition and a tag, one or more conservation conditions, one or more pieces of acquisition information, and a usage mode flag. The movement storage unit 21 typically stores one or more pairs of an attribute value tag condition and a tag. The movement storage unit 21 typically stores one or more pairs of a video tag condition and a tag.

[0205] For example, one or more moving image attribute values ​​constituting a moving attribute value set correspond to one or more still images (which may also be called fields or frames) constituting a moving moving image. The one or more moving image attribute values ​​may correspond to all still images, may correspond to some of the still images, or may correspond to multiple still images.

[0206] An attribute value tag condition is a condition for acquiring a tag based on one or more mobile object attribute values. An attribute value tag condition is a condition related to one or more mobile object attribute values. Examples of the attribute value tag condition are "the brake is applied suddenly," "the degree of deceleration of the speed within a unit time (acceleration) is below a threshold value (the brake is applied suddenly)," "the airbag is deployed," "the time at or below a first speed has elapsed for a second hour or more (there is a traffic jam)," and "the location information corresponds to a specific name." Examples of tags paired with an attribute value tag condition are "abnormal driving," "accident," "traffic jam," and "specific location name."

[0207] A video tag condition is a condition related to a video. A video tag condition is a condition based on the analysis result of a video. Examples of video tag conditions are "having a still image of multiple cars colliding (it is an accident)", "having a frame in which the distance between multiple cars is zero (cars are colliding)", "having a period in which a first threshold or more of cars are present within a specified distance that is equal to or greater than a second threshold (it is a traffic jam)", and "the cumulative amount of change in the distance between the center of gravity of the car in front and the lane per unit time is equal to or greater than a threshold (the car in front is swerving)". Examples of tags that pair with a video tag condition are "accident", "traffic jam", and "abnormal driving".

[0208] The preservation condition is a condition for preserving a video. The preservation condition is a condition related to a set of attribute values. For example, the preservation condition is "an airbag has been deployed (an accident has occurred)" or "a specific tag has been added to the video (the specific tag is, for example, "accident" or "dangerous driving")". The preservation condition is associated with, for example, the acquired information. The acquired information is information that identifies the video to be acquired. The acquired information is information that identifies the video to be acquired when the preservation condition is satisfied. For example, the preservation condition is "(1) the number of cars in the screen is equal to or greater than a first threshold value & (2) the traveling speed of the cars is equal to or less than a second threshold value & the duration of (1) and (2) is equal to or greater than a third threshold value (traffic jam continues)". For example, the acquired information is information that indicates "a video from one minute before the preservation condition is satisfied until the preservation condition is no longer satisfied after the preservation condition is satisfied".

[0209] The mobile receiving unit 22 receives various types of information, such as inquiries, needs information, videos captured by other mobile terminals 2, and videos captured by the fixed terminal 3.

[0210] The movement processing unit 23 performs various processes. The various processes are, for example, processes performed by the photographing unit 231, the tag acquiring unit 232, and the movement information acquiring unit 233. The movement processing unit 23 converts the information received by the movement receiving unit 22 into a data structure to be output.

[0211] The movement processing unit 23 detects, for example, the start of movement. The detection of the start of movement is, for example, detection of the power supply being turned on of the mobile terminal 2 or detection of the engine being turned on of the moving object.

[0212] The movement processing unit 23 detects, for example, the end of movement. The detection of the end of movement is, for example, detection of the power supply of the mobile terminal 2 being turned off, or detection of the engine of the moving object being turned off.

[0213] The movement processing unit 23 determines, for example, an attribute value set that matches the inquiry received by the movement receiving unit 22, and acquires from the movement storage unit 21 a video that is paired with the attribute value set.

[0214] The movement processing unit 23 acquires, for example, a video from the start of movement to the end of movement. The movement processing unit 23 acquires, for example, a video from the start of movement to the end of movement in association with a set of attribute values ​​acquired during the movement.

[0215] The movement processing unit 23, for example, acquires an attribute value set during shooting. The movement processing unit 23 accumulates the acquired movement attribute value set in the movement storage unit 21. The movement processing unit 23 also, for example, associates the acquired movement attribute value set with a moving image. Associating with a moving image usually means associating with a frame constituting the moving image. It is preferable that the attribute value set and the frame are synchronized in time.

[0216] The movement attribute value set is, for example, one or more pieces of environmental information. The environmental information is, for example, location information, time information, weather information, temperature information, and season information.

[0217] The movement processing unit 23 acquires location information, for example, during shooting. For example, the movement processing unit 23 having a function of a GPS receiver acquires location information. The movement processing unit 23 acquires location information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied. The acquisition condition is a condition for acquiring information. The acquisition condition is, for example, when an accident is detected, when a traffic jam is detected, or when there is a change in weather information.

[0218] The movement processing unit 23 acquires time information from a clock (not shown) during shooting, for example. The movement processing unit 23 acquires time information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied.

[0219] For example, during shooting, the movement processing unit 23 acquires time information from a clock (not shown) and acquires season information corresponding to the time information.

[0220] The movement processing unit 23 acquires weather information, for example, during shooting. The movement processing unit 23 acquires weather information corresponding to location information, for example, from a server (not shown). The movement processing unit 23 acquires weather information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied.

[0221] The movement processing unit 23 acquires temperature information, for example, during shooting. The movement processing unit 23 acquires temperature information corresponding to location information, for example, from a server (not shown). The movement processing unit 23 acquires temperature information, for example, from a temperature sensor installed on a moving object. The movement processing unit 23 acquires temperature information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied.

[0222] The movement processing unit 23, for example, judges whether the acquired movement attribute value set matches the preservation condition. The movement processing unit 23, for example, judges whether the acquired time-series movement attribute value set matches the preservation condition. For example, when the preservation condition is met, the movement processing unit 23 acquires a moving image corresponding to the movement attribute value set. For example, when the preservation condition is met, the movement processing unit 23 acquires a moving image corresponding to the preservation condition. For example, when the preservation condition is met, the movement processing unit 23 acquires acquisition information paired with the preservation condition, and acquires a moving image according to the acquisition information.

[0223] It is preferable that the movement processing unit 23 includes, for example, a microphone, acquires sound information, and stores the sound information in association with the video acquired by the imaging unit 231. Such a function is, for example, a function of a drive recorder.

[0224] The movement processing unit 23, for example, acquires the usage mode flag of the movement storage unit 21, and judges whether the usage mode flag is information indicating whether to inquire of the user whether to transmit the moving video before transmitting the moving video. For example, when the usage mode flag is a value indicating "non-provisional request for actual use", the movement processing unit 23 outputs inquiry information which is information for inquiring of the user whether to transmit the moving video. Note that the inquiry information is, for example, screen information (for example, a panel) for inquiring whether to transmit the moving video, or audio information for inquiring whether to transmit the moving video.

[0225] The photographing unit 231 photographs a moving image. The photographing unit 231 starts photographing after detecting the start of movement, for example. It is preferable that the photographing unit 231 continues photographing until detecting the end of movement, for example.

[0226] It is preferable that the imaging unit 231 stores the captured video in the moving storage unit 21. When storing video in the moving storage unit 21 with a limited storage capacity, it is preferable that the imaging unit 231 overwrites new video in an area in which old video is stored. In other words, it is preferable that the structure of the moving storage unit 21 is a ring buffer structure.

[0227] The tag acquiring unit 232 acquires one or more tags for the video captured by the imaging unit 231 and associates them with the video.

[0228] The tag acquisition unit 232, for example, analyzes a moving image captured by the image capture unit 231 and acquires one or more tags for the moving image.

[0229] The tag acquiring unit 232 acquires one or more tags, for example, by using one or more moving object attribute values ​​acquired during shooting by the shooting unit 231. The moving object attribute values ​​are, for example, CAN data.

[0230] For example, the tag acquiring unit 232 determines one or more still images that match the video tag condition, and acquires a tag that is paired with the video tag condition. The tag acquiring unit 232 may also associate the tag with the one or more still images. Note that the still images are frames that constitute the video.

[0231] For example, if the video tag condition is "the number of cars whose distance from the preceding car is within a threshold is equal to or greater than a threshold and the moving speed of the car is equal to or less than a threshold" and the tag paired with the video tag condition is "traffic jam", the tag acquisition unit 232 analyzes frames of the video, recognizes multiple cars, and acquires the distance between each of the multiple cars. The tag acquisition unit 232 also acquires the number of cars whose distance between the cars is within a threshold. The tag acquisition unit 232 also acquires the moving distance and frame rate of one car in multiple frames, and acquires the speed of the car. Then, the tag acquisition unit 232 uses the number of cars whose distance between the cars is within a threshold and the speed of the car to determine whether or not the video tag condition is met. Then, if the video tag condition is met, the tag acquisition unit 232 acquires the tag "traffic jam" that is paired with the video tag condition. The tag acquisition unit 232 may also associate the tag "traffic jam" with the analyzed frame.

[0232] For example, the tag acquiring unit 232 determines one or more moving object attribute values ​​that match the attribute value tag condition, and acquires a tag that is paired with the attribute value tag condition. The tag acquiring unit 232 may also associate the tag with a video that is paired with the one or more moving object attribute values.

[0233] For example, if the attribute value tag condition is "driving time at a speed less than 30 km for 10 minutes or more & the ratio of the speed less than 30% during the driving time is 80% or more" and the tag paired with the attribute value tag condition is "traffic jam", the tag acquisition unit 232 detects CAN data that matches the attribute value tag condition using the speed history of the CAN data associated with each field of the video, acquires the tag "traffic jam" that matches the attribute value tag condition, and associates the tag with the field associated with the CAN data. Note that the CAN data associated with each field of the video is the CAN data acquired at the same timing as the field was shot.

[0234] The movement information acquisition unit 233 detects the movement of the mobile terminal 2, and acquires movement information, for example, at the start of the movement. The movement information acquisition unit 233 acquires movement information, which is, for example, a right holder identifier of the movement storage unit 21. The movement information acquisition unit 233 acquires, for example, the movement information of the movement storage unit 21. Note that the movement information is, for example, a right holder identifier, and information indicating that movement has started. Detection of the movement of the mobile terminal 2 is, for example, detection of the engine being ON, or detection of the power supply being ON of the mobile terminal 2.

[0235] The movement transmitting unit 24 transmits various types of information to the information processing device 1. The various types of information are, for example, movement information, movement video, and a movement attribute value set.

[0236] The movement information transmitting unit 241 normally transmits the movement information acquired by the movement information acquiring unit 233 to the information processing device 1 at the start of movement.

[0237] The movement video sending unit 242 transmits the movement video captured by the imaging unit 231 to the information processing device 1. The timing at which the movement video sending unit 242 transmits the video does not matter. For example, after the movement processing unit 23 acquires a video paired with a movement attribute value set that matches the query received, the movement video sending unit 242 transmits the video to the information processing device 1. For example, when the movement processing unit 23 detects the end of the movement, the movement video sending unit 242 transmits the video accumulated after the detection of the start of the movement to the information processing device 1. For example, when it is determined that the conservation conditions are met, the movement video sending unit 242 transmits the video corresponding to the determination to the information processing device 1.

[0238] It is preferable that the mobile transmission unit 24 does not transmit the sound information acquired by the mobile processing unit 23 even if it transmits the mobile video captured by the imaging unit 231 to the information processing device 1. The sound information may be, for example, voice information of the driver or passengers in the moving object, and transmitting such sound information to the information processing device 1 and providing it to the user terminal 4 or the like may threaten the privacy of the driver or passengers, which is not appropriate.

[0239] When the movement processing unit 23 inquires of the user of the mobile terminal 2 whether or not to transmit the movement video based on the usage mode flag, the movement video transmitting unit 242 transmits the movement video only when information indicating that the movement video is to be transmitted is received in response to the inquiry. Note that the information indicating that the movement video is to be transmitted or that the movement video is not to be transmitted is received by a movement receiving unit (not shown) that the mobile terminal 2 has.

[0240] The set sending unit 243 sends the set of movement attribute values ​​in the movement storage unit 21 to the information processing device 1. For example, when the movement of the mobile terminal 2 ends, the set sending unit 243 sends the set of movement attribute values ​​in the movement storage unit 21 to the information processing device 1. Here, the set of movement attribute values ​​in the movement storage unit 21 is a set of movement attribute values ​​for moving images stored in the movement storage unit 21.

[0241] Note that, once the set transmitting unit 243 acquires the moved attribute value set, it may immediately transmit the attribute value set to the information processing device 1. In other words, the set transmitting unit 243 may transmit the moved attribute value set at any time. <Details of components of fixed terminal 3>

[0242] Various types of information are stored in the fixed storage unit 31 constituting the fixed terminal 3. The various types of information are, for example, a fixed video, a fixed attribute value set, a right holder identifier, camera information, a set of an attribute value tag condition and a tag, a set of a video tag condition and a tag, one or more conservation conditions, and one or more pieces of acquired information. The fixed storage unit 31 typically stores one or more sets of a video tag condition and a tag. It is preferable that the fixed attribute value set includes location information of the fixed terminal 3.

[0243] For example, one or more moving image attribute values ​​constituting a fixed attribute value set correspond to one or more still images (which may also be called fields or frames) constituting a fixed moving image. The one or more moving image attribute values ​​may correspond to all still images, may correspond to some of the still images, or may correspond to multiple still images.

[0244] A video tag condition is a condition related to a video. A video tag condition is a condition based on the analysis results of a video. Examples of video tag conditions are "having a still image of multiple cars colliding (it is an accident)", "having a frame in which the distance between multiple cars is zero (cars are colliding)", and "the cumulative amount of change per unit time between the center of gravity of the car in front and the lane is greater than or equal to a threshold (the car in front is swerving)". Tags that pair with a video tag condition are, for example, "accident" and "abnormal driving".

[0245] The fixed receiving unit 32 receives various types of information, such as inquiries and various instructions.

[0246] The fixed processing unit 33 performs various types of processing. The various types of processing are, for example, processing performed by the fixed camera 331. The fixed processing unit 33 configures information to be transmitted in response to an inquiry received by the fixed receiving unit 32. The fixed processing unit 33 configures information to be transmitted by the fixed transmitting unit 34. The transmitted information includes a fixed moving image. It is preferable that the transmitted information includes a fixed attribute value set and a right holder identifier.

[0247] The fixed processing unit 33 acquires a fixed attribute value set during shooting, for example. The fixed processing unit 33 accumulates the acquired fixed attribute value set in the fixed storage unit 31. Furthermore, the fixed processing unit 33, for example, associates the acquired fixed attribute value set with a moving image. Associating with a moving image usually means associating with a frame constituting the moving image. It is preferable that the fixed attribute value set and the frame are synchronized in time.

[0248] The fixed attribute value set is, for example, one or more pieces of environmental information. The environmental information is, for example, time information, weather information, temperature information, and season information.

[0249] The fixed processing unit 33 acquires time information from a clock (not shown) during shooting, for example. The fixed processing unit 33 acquires time information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied.

[0250] For example, during shooting, the fixed processing unit 33 obtains time information from a clock (not shown) and obtains season information corresponding to the time information.

[0251] The fixed processing unit 33 acquires weather information, for example, during shooting. The fixed processing unit 33 acquires weather information corresponding to the location information of the fixed terminal 3, for example, from a server (not shown). The fixed processing unit 33 acquires weather information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied.

[0252] The fixed processing unit 33 acquires temperature information, for example, during shooting. The fixed processing unit 33 acquires temperature information corresponding to the location information of the fixed terminal 3, for example, from a server not shown. The fixed processing unit 33 acquires temperature information, for example, from a temperature sensor installed in a moving object. The fixed processing unit 33 acquires temperature information, for example, constantly, at predetermined intervals, or when an acquisition condition is satisfied.

[0253] The fixed processing unit 33, for example, judges whether the acquired fixed attribute value set matches the preservation condition. The fixed processing unit 33, for example, judges whether the acquired time-series fixed attribute value set matches the preservation condition. For example, when the preservation condition is met, the fixed processing unit 33 acquires a moving image corresponding to the fixed attribute value set. For example, when the preservation condition is met, the fixed processing unit 33 acquires a moving image corresponding to the preservation condition. For example, when the preservation condition is met, the fixed processing unit 33 acquires acquisition information paired with the preservation condition, and acquires a moving image according to the acquisition information.

[0254] It is preferable that the fixed processing unit 33 includes, for example, a microphone, acquires sound information, and stores the sound information in association with the fixed moving image acquired by the fixed camera 331.

[0255] The fixed processing unit 33, for example, analyzes a fixed moving image and acquires one or more tags for the fixed moving image. Note that the process of acquiring such tags is the same as the process performed by the tag acquiring unit 232 described above.

[0256] The fixed camera 331 shoots and acquires a moving image. Such a moving image is a fixed moving image. The fixed camera 331 is a camera that does not move. The fixed camera 331 is a camera that is installed at a fixed position. The fixed camera 331 is a camera whose shooting position is fixed. A fixed shooting position does not necessarily mean that the shooting direction is also fixed, and includes the case where the shooting direction changes.

[0257] The fixed transmission unit 34 transmits various types of information to the information processing device 1. The various types of information are, for example, fixed moving images, fixed attribute value sets, and sound information. <Details of components of user terminal 4>

[0258] Various types of information are stored in the user storage unit 41 constituting the user terminal 4. The various types of information are, for example, a user identifier and a video.

[0259] The user reception unit 42 receives various instructions and information, such as inquiries and purchase instructions.

[0260] It should be noted that a purchase instruction is an instruction to purchase a video. The purchase instruction is associated with a user identifier. The purchase instruction typically includes information that identifies a video. The purchase instruction has, for example, a video identifier. The purchase instruction includes, for example, an inquiry. The purchase instruction includes, for example, purchase conditions. The purchase conditions are, for example, a purchase price. The purchase conditions include, for example, information that identifies a rights period.

[0261] The means for inputting various instructions and information may be any means, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0262] The user processing unit 43 performs various processes. For example, the various processes are processes for creating a data structure for transmitting various instructions and information received by the user receiving unit 42. For example, the various processes are processes for creating a structure for transmitting information received by the user receiving unit 45.

[0263] The user transmission unit 44 transmits various instructions and information to the information processing device 1. The various instructions and information are, for example, an inquiry, a purchase instruction, location information received from the target terminal 5, and route information received from the navigation terminal 6.

[0264] The user receiving unit 45 receives various types of information and instructions, such as inquiries, videos, and status information.

[0265] The user output unit 46 outputs various types of information, such as video and status information.

[0266] Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over the processing results to other processing devices or other programs, etc.

[0267] The target terminal 5 acquires and transmits location information that identifies the location of the target terminal 5. The target terminal 5 transmits the location information to, for example, the information processing device 1 or the user terminal 4. It is preferable that the target terminal 5 transmits the location information in pair with a target identifier. The target identifier is information that identifies the monitoring target. The target identifier is, for example, an ID, name, telephone number, email address, or MAC address of the target terminal 5. The target terminal 5 acquires the location information by, for example, a GPS receiver, but the method of acquiring the location information is not important.

[0268] The navigation terminal 6 has the functions of a known navigation terminal. The navigation terminal 6 accepts input of a destination, acquires a current position, and acquires route information from the current position to the destination. The navigation terminal 6 transmits, for example, the accepted destination. The navigation terminal 6 transmits, for example, the acquired route information. The destination or route information is transmitted to, for example, the information processing device 1 or the user terminal 4. The navigation terminal 6 may receive a destination from the user terminal 4, and acquire route information from the current position to the destination.

[0269] Storage unit 11, mobile terminal management unit 111, fixed terminal management unit 112, mobile storage unit 21, fixed storage unit 31, and user storage unit 41 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0270] There is no restriction on the process by which information is stored in the storage unit 11, etc. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.

[0271] The receiving unit 12, the mobile information receiving unit 121, the inquiry receiving unit 122, the collective receiving unit 123, the mobile receiving unit 22, and the user receiving unit 45 are typically realized by wireless or wired communication means, but may also be realized by means for receiving broadcasts.

[0272] The processing unit 13, the movement information storage unit 131, the collection storage unit 132, the video acquisition unit 133, the video composition unit 134, the video processing unit 135, the right holder processing unit 136, the movement video acquisition means 1331, the fixed video acquisition means 1332, the first security means 1361, the second security means 1362, the third security means 1363, the fourth security means 1364, the reward means 1365, the movement processing unit 23, the tag acquisition unit 232, the movement information acquisition unit 233, the fixed processing unit 33, and the user processing unit 43 can usually be realized by a processor, a memory, or the like. The processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor is a CPU, an MPU, a GPU, etc., and the type does not matter.

[0273] The system includes a transmission unit 14, a video transmission unit 141, a status transmission unit 142, and a needs transmission unit 143.

[0274] In the mobile terminal 2, the mobile transmitting unit 24, the mobile information transmitting unit 241, the mobile video transmitting unit 242, the collective transmitting unit 243, the fixed transmitting unit 34, and the user transmitting unit 44 are typically realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0275] The photographing unit 231 is realized by a camera. The fixed camera 331 is a fixed camera. The camera may be, for example, a CCD camera, a CMOS camera, a 3D camera, a LiDAR, or an omnidirectional camera, but the type is not limited thereto.

[0276] The user reception unit 42 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0277] The user output unit 46 may or may not include an output device such as a display, a speaker, etc. The user output unit 46 may be realized by driver software for an output device, or a combination of driver software for an output device and the output device, etc. <Operation>

[0278] Next, an example of the operation of the information system A will be described. First, an example of the operation of the information processing device 1 will be described with reference to the flowchart of Fig. 5. Note that "S" in each flowchart used in the following description indicates a step.

[0279] (S501) The movement information receiving unit 121 determines whether or not movement information associated with a right holder identifier has been received from the mobile terminal 2. If movement information has been received, the process proceeds to S502, and if not, the process proceeds to S503.

[0280] (S502) The movement information receiving unit 121 associates the movement information received in S501 with the right holder identifier and stores it in the storage unit 11. Return to S501. Note that if the movement information is a right holder identifier, the movement information receiving unit 121 stores the right holder identifier.

[0281] (S503) The set receiving unit 123 judges whether or not a moved attribute value set associated with the right holder identifier has been received from the mobile terminal 2. If a moved attribute value set has been received, the process proceeds to S504, and if not, the process proceeds to S506.

[0282] (S504) The set accumulation unit 132 accumulates the moved attribute value set received in S503 in the set storage unit 111 in association with the right holder identifier.

[0283] (S505) Processing unit 13 deletes the movement information corresponding to the right holder identifier from storage unit 11. Return to S501. Note that deleting the movement information is equivalent to "rewriting the movement information to information indicating that the video is not in a state where it can be transmitted."

[0284] (S506) The receiving unit 12 judges whether or not a video or the like associated with the right holder identifier has been received from the mobile terminal 2 or the fixed terminal 3. If a video or the like has been received, the process proceeds to S507, and if not, the process proceeds to S510.

[0285] (S507) The processing unit 13 performs a process of acquiring an attribute value set corresponding to the moving image received in S506. An example of such an attribute value set acquisition process will be described with reference to the flowchart in FIG. 6. The attribute value set acquisition process here may be a process of acquiring an attribute value set associated with the received moving image. The moving image here is a moving moving image or a fixed moving image. The attribute value set acquired here is a moving attribute value set or a fixed attribute value set.

[0286] (S508) The first maintenance means 1361 stores the video received in S506 in association with the attribute value set acquired in S507. Note that the first maintenance means 1361 stores the video in the storage unit 11, for example, but may store the video in another device such as a device constituting a blockchain. Also, the process in which the first maintenance means 1361 stores the video is the first maintenance process.

[0287] (S509) The fourth maintenance means 1364 performs a fourth maintenance process. Return to S501. An example of the fourth maintenance process will be described with reference to the flowchart in FIG.

[0288] (S510) The inquiry receiving unit 122 judges whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S511, and if not, the process proceeds to S519. The inquiry receiving unit 122 receives an inquiry from, for example, the user terminal 4 or the navigation terminal 6. The navigation terminal 6 that transmits the inquiry may be the user terminal 4.

[0289] (S511) The video acquisition unit 133 judges whether the received query is a real-time video search. If it is a real-time video search, the process proceeds to S512, and if it is not a real-time video search, the process proceeds to S515. This judgment can be made by inspecting the received query. A real-time video search is a process in which the mobile terminal 2 or the fixed terminal 3 searches for videos that it is currently holding or is currently being shot.

[0290] (S512) The video acquisition unit 133 judges whether the received query requires spatial fusion of multiple videos. If spatial fusion is required, the process proceeds to S513. If spatial fusion is not required, the process proceeds to S514. Note that such a judgment can be made by inspecting the received query.

[0291] (S513) The moving image acquisition unit 133, moving image composition unit 134, moving image processing unit 135, etc. perform moving image fusion processing. Go to S518. The moving image fusion processing is a process of spatially fusing multiple moving images to acquire and transmit a moving image having one or multiple frames. An example of the moving image fusion processing will be described with reference to the flowchart of FIG. 10. Note that the moving image to be transmitted may be a stereoscopic three-dimensional moving image.

[0292] (S514) The moving image acquisition unit 133, the moving image composition unit 134, the moving image processing unit 135, etc. perform moving image combination processing. Go to S518. The moving image combination processing is processing for transmitting one moving image in which multiple moving images are combined in terms of time. An example of the moving image combination processing will be described with reference to the flowchart in FIG. 12.

[0293] (S515) The moving image acquiring unit 133 and the like perform a registered moving image search process. An example of the registered moving image search process will be described with reference to the flowchart in Fig. 13. Note that there may be cases where the moving image acquiring unit 133 is unable to acquire a moving image.

[0294] (S516) The moving image acquiring unit 133 determines whether or not the moving image was acquired in S515. If the moving image was acquired, the process proceeds to S517, and if the moving image was not acquired, the process proceeds to S518.

[0295] (S517) The moving image acquiring unit 133 performs an unregistered moving image search process. An example of the unregistered moving image search process will be described with reference to the flowcharts in Fig. 14 and Fig. 15. Note that there may be a case where the moving image acquiring unit 133 is unable to acquire a moving image.

[0296] (S518) The right holder processing unit 136 performs a protection process. Return to S501. An example of the protection process will be described with reference to the flowchart in FIG.

[0297] (S519) The receiving unit 12 determines whether or not a purchase instruction has been received from the user terminal 4. If a purchase instruction has been received, the process proceeds to S520, and if not, the process returns to S501.

[0298] (S520) The video acquiring unit 133 acquires a video corresponding to the purchase instruction. For example, the video acquiring unit 133 receives the video corresponding to the purchase instruction from the mobile terminal 2 or the fixed terminal 3. For example, the video acquiring unit 133 acquires the video corresponding to the purchase instruction from registered videos. For example, the video acquiring unit 133 may acquire the video corresponding to the purchase instruction from unregistered videos.

[0299] (S521) The fourth security means 1364 acquires a user identifier corresponding to the user terminal 4 that transmitted the purchase instruction. This user identifier becomes the right holder identifier for the purchased video.

[0300] (S522) The fourth security means 1364 performs a fourth security process using the user identifier acquired in S516. An example of the fourth security process will be described with reference to the flowchart of FIG.

[0301] (S523) The compensation means 1365 performs compensation processing for the original rights holder of the purchased video. Return to S501. An example of the compensation processing will be described with reference to the flowchart in FIG.

[0302] In the flowchart of FIG. 5, the process ends when the power is turned off or an interrupt occurs to end the process.

[0303] Also, in the flowchart of Figure 5, it is preferable that the receiver 12 receives from each of one or more mobile terminals 2 location information that identifies the current location of the mobile terminal 2 in pair with an identifier of the mobile terminal 2 (e.g., a rights holder identifier) ​​and temporarily stores the information. The frequency or time interval at which the receiver 12 receives the location information from the mobile terminal 2 and temporarily stores it is not important. The receiver 12, for example, continuously receives the location information and temporarily stores it. The receiver 12, for example, periodically receives the location information and temporarily stores it.

[0304] 5, the receiving unit 12 may receive location information, for example, continuously, from the user terminal 4, the target terminal 5, or the navigation terminal 6. In this case, the information processing device 1 may receive a video acquired by the mobile terminal 2 or a video acquired by the fixed terminal 3 using the received location information, and transmit the video to the user terminal 4 or the navigation terminal 6. Note that the frequency or time interval at which the location information is continuously received does not matter.

[0305] 5, the receiving unit 12 may receive the fixed attribute value set and the fixed video from the fixed terminal 3, paired with an identifier (e.g., a right holder identifier) ​​of the fixed terminal 3. In this case, the processing unit 13 stores the fixed attribute value set and the fixed video in the fixed terminal management unit 112, paired with the identifier of the fixed terminal 3.

[0306] Next, an example of the attribute value set acquisition process in S507 will be described with reference to the flowchart in FIG.

[0307] (S601) The movement processing unit 23 or the collection storage unit 132 or the fixed processing unit 33 (hereinafter referred to as the movement processing unit 23, etc.) acquires one or more pieces of environmental information etc. associated with a target moving image. The target moving image is a moving image for which a moving image attribute value is to be acquired, for example, a received moving image, a shot moving image, or a moving image being shot. The target moving image is a moving moving image or a fixed moving image. The environmental information etc. may be only environmental information, or may be environmental information and one or more moving object attribute values.

[0308] (S602) The movement processing unit 23 or the like acquires a rights holder identifier associated with the received moving image.

[0309] (S603) The movement processing unit 23 etc. judge whether or not to perform annotation processing on the video. If annotation processing is to be performed, the process proceeds to S604, and if not, the process proceeds to S611. It is assumed that whether or not annotation processing is to be performed is determined in advance. It is preferable to annotate the video on either the information processing device 1 or the terminal. The terminal here is the mobile terminal 2 or the fixed terminal 3.

[0310] (S604) The movement processing unit 23 etc. assign 1 to a counter i.

[0311] (S605) The movement processing unit 23 etc. judge whether or not the i-th annotation unit exists in the video. If the i-th annotation unit exists, the process proceeds to S606, and if not, the process proceeds to S611. Note that the annotation unit is the subject from which the annotation is acquired, and is, for example, one frame, a certain number of frames, or a video of a certain time (for example, 5 seconds, 10 seconds).

[0312] (S606) The movement processing unit 23 etc. judge whether or not to use the moving object attribute value at the time of annotation. If the moving object attribute value is to be used, the process proceeds to S607, and if not, the process proceeds to S608. It is assumed that whether or not to use the moving object attribute value at the time of annotation is determined in advance. Also, the movement processing unit 23 etc. can use the moving object attribute value only when the target video is a moving video.

[0313] (S607) The movement processing unit 23 and the like perform a moving object attribute value tag acquisition process. An example of the moving object attribute value tag acquisition process will be described with reference to the flowchart of FIG.

[0314] (S608) The movement processing unit 23 etc. judge whether or not to use a moving image when making an annotation. If a moving image is to be used, the process proceeds to S609, and if not, the process proceeds to S610. It is assumed that it is previously determined whether or not a moving image is to be used when making an annotation.

[0315] (S609) The movement processing unit 23 and the like perform a moving image tag acquisition process. An example of the moving image tag acquisition process will be described with reference to the flowchart of FIG.

[0316] (S610) The movement processing unit 23 and the like increment the counter i by 1. The process returns to S605.

[0317] (S611) The movement processing unit 23 etc. configures an attribute value set having one or more pieces of environmental information, one or more tags, and a rights holder identifier, and then returns to the upper level processing.

[0318] In the flowchart of FIG. 5, the moving image may be divided into a plurality of inspection units, and the processes from S601 to S611 may be performed for each inspection unit.

[0319] Next, an example of the moving object attribute value tag acquisition process in S607 will be described with reference to the flowchart in FIG.

[0320] (S701) The movement processing unit 23 or the collection storage unit 132 assigns 1 to a counter i.

[0321] (S702) The movement processing unit 23 etc. judge whether or not the i-th attribute value tag condition exists. If the i-th attribute value tag condition exists, the process proceeds to S703, and if not, the process returns to the upper process.

[0322] (S703) The movement processing unit 23 etc. acquire the i-th attribute value tag condition.

[0323] (S704) The movement processing unit 23 etc. acquire one or more moving object attribute values ​​to be used for determining the i-th attribute value tag condition.

[0324] (S705) The movement processing unit 23 etc. judges whether or not one or more moving object attribute values ​​acquired in S704 satisfy the i-th attribute value tag condition. If the attribute value tag condition is satisfied, the process proceeds to S706, and if not, the process proceeds to S707.

[0325] (S706) The movement processing unit 23 etc. acquires a tag that is paired with the i-th attribute value tag condition, and associates the tag with the corresponding moving image. Associating a tag with a moving image usually means associating the tag with a field that constitutes the moving image.

[0326] (S707) The movement processing unit 23 and the like increment the counter i by 1. The process returns to S702.

[0327] Next, an example of the video tag acquisition process in S609 will be described with reference to the flowchart in FIG.

[0328] (S801) The movement processing unit 23, the collection storage unit 132, or the fixed processing unit 33 assigns 1 to a counter i.

[0329] (S802) The movement processing unit 23 etc. judge whether or not the i-th video tag condition exists. If the i-th video tag condition exists, the process proceeds to S803, and if not, the process returns to the upper process.

[0330] (S803) The movement processing unit 23 etc. acquire the i-th moving image tag condition.

[0331] (S804) The movement processing unit 23 etc. acquire one or more moving image attribute values ​​to be used for determining the i-th moving image tag condition.

[0332] (S805) The movement processing unit 23 etc. judges whether or not one or more video attribute values ​​acquired in S804 satisfy the i-th attribute value tag condition. If the attribute value tag condition is satisfied, the process proceeds to S806, and if not, the process proceeds to S807.

[0333] (S806) The movement processing unit 23 etc. acquires a tag paired with the i-th attribute value tag condition, and associates the tag with the corresponding moving image. Associating a tag with a moving image usually means associating the tag with one or more frames paired with one or more moving image attribute values ​​used to determine the attribute value tag condition.

[0334] (S807) The movement processing unit 23 and the like increment the counter i by 1. The process returns to S802.

[0335] Next, an example of the fourth security process of S509 will be described with reference to the flowchart of FIG.

[0336] (S901) The fourth security means 1364 acquires access information that identifies a storage destination of a moving image. Note that the moving image is, for example, a fusion moving image, a combination moving image, or a stereoscopic three-dimensional moving image.

[0337] (S902) The fourth maintenance means 1364 acquires a set of attribute values ​​corresponding to the stored moving image. If the stored moving image is a moving image composed of multiple original moving images, the set of attribute values ​​corresponding to the moving image is a set of attribute values ​​of each of the multiple original moving images.

[0338] (S903) The fourth security means 1364 configures security information having the access information acquired in S901, the attribute value set acquired in S902, and the rights holder identifier of the video. Note that if a new rights holder identifier has been acquired, the fourth security means 1364 configures security information including, for example, the new rights holder identifier and the original rights holder identifier. (S904) The fourth maintenance means 1364 accumulates the maintenance information constructed in S903, and then returns to the upper level processing.

[0339] In addition, in S904, if the preservation information of the video corresponding to the preservation information to be stored has already been stored, the preservation information configured in S903 is overwritten with the preservation information. In this way, for example, the transition of the right holder of the video can be managed. In addition, the fourth preservation means 1364 stores the preservation information in, for example, a block chain.

[0340] Next, an example of the moving image fusion process in S513 will be described with reference to the flowchart in FIG.

[0341] (S1001) The video acquisition unit 133 acquires reference location information. The reference location information is, for example, location information included in the received inquiry. The reference location information is, for example, location information of the user terminal 4 or the target terminal 5. The reference location information is, for example, one piece of location information in the route information included in the inquiry. The one piece of location information is the first piece of location information in the route information. The location information is, for example, (latitude, longitude), but may also be a location name (for example, the name of a parking lot), a place name, etc.

[0342] (S1002) The moving image obtaining unit 133 assigns 1 to a counter i.

[0343] (S1003) The video acquisition unit 133 judges whether or not the i-th terminal to which the video can be transmitted exists. If the i-th terminal exists, the process proceeds to S1004, and if not, the process proceeds to S1008. Note that a terminal to which the video can be transmitted is usually a fixed terminal 3 or a mobile terminal 2 that is moving.

[0344] Moreover, the mobile terminal 2 in motion is the mobile terminal 2 corresponding to the motion information stored in the storage unit 11. Moreover, the fixed terminal 3 is the fixed terminal 3 corresponding to the fixed terminal information managed by the fixed terminal management unit 112. That is, the video acquisition unit 133 determines, for example, whether or not the i-th information exists among the motion information in the storage unit 11 or the fixed terminal information in the fixed terminal management unit 112.

[0345] (S1004) The video acquiring unit 133 acquires the latest location information transmitted by the mobile terminal 2 which is the i-th terminal, or the location information included in the fixed terminal information which is the i-th terminal.

[0346] (S1005) The video acquisition unit 133 determines whether or not the location information acquired in S1004 satisfies a second location condition. If the second location condition is satisfied, the process proceeds to S1006. If the second location condition is not satisfied, the process proceeds to S1007.

[0347] (S1006) Video acquiring unit 133 acquires an image from the i-th terminal, pairs the image with the right holder identifier and location information of the i-th terminal, and temporarily stores the image in a buffer (not shown).

[0348] In addition, the video acquisition unit 133 sends, for example, an inquiry to the i-th terminal, including a type identifier that specifies the type of image requested (for example, “video”, “still image”, or “around view image”), and receives an image corresponding to the inquiry from the i-th mobile terminal 2.

[0349] (S1007) The moving image obtaining unit 133 increments the counter i by 1. The process returns to S1003.

[0350] (S1008) Moving image composition unit 134 calculates the difference (distance) between the position information for each image temporarily stored in a buffer (not shown) and the reference position information. Moving image composition unit 134 sorts the images in ascending order using the difference as a key.

[0351] (S1009) The moving image composing unit 134 acquires a reference image. The reference image is an image that corresponds to the reference position information. The reference image is, for example, an image transmitted from a user terminal 4 that is present at the position of the reference position information. Note that this image is an image of a type identified by a type identifier that specifies the type of image requested.

[0352] (S1010) The moving image composing unit 134 assigns 1 to a counter j.

[0353] (S1011) The moving image composing unit 134 judges whether or not the jth other image is present among the images sorted in S1008. If the jth other image is present, the process proceeds to S1012, and if not, the process proceeds to S1014. The other images are the images sorted in S1008.

[0354] (S1012) When the jth other image is an image of a video (for example, a moving video) that does not correspond to the priority type, the video composition unit 134 judges whether or not there is a priority image that can be substituted for the jth other image. If there is a priority image that can be substituted, the process proceeds to S1013, and if there is no priority image, the process proceeds to S1014.

[0355] The replaceable priority image is an image captured of an area equivalent to the area of ​​the jth other image, and is an image paired with the position information of a position whose distance from the position indicated by the position information of the jth other image is within a threshold. The priority image is an image of a moving image (for example, a fixed moving image) corresponding to a priority type. Note that the equivalent area is, for example, an area in which the ratio of the overlapping area is equal to or greater than a threshold, and an area in which the overlapping area is equal to or greater than a threshold.

[0356] For example, the moving image composing unit 134 checks whether an image that is paired with position information of a position whose distance from the position indicated by the position information of the jth other image is within a threshold is a priority image, in order of the shortest distance.

[0357] (S1013) The moving image composition unit 134 acquires a replaceable priority image.

[0358] (S1014) The moving image constructing unit 134 performs image fusion processing to fuse the reference image acquired in S1009 with the j-th other image or the priority image acquired in S1013. An example of the image fusion processing will be described with reference to the flowchart in FIG.

[0359] (S1015) The moving image composition unit 134 increments the counter j by α. Return to S1011. Note that when the jth other image is used in the image fusion process, α=1, and when the priority image is used, α is the number by which the counter j advances to the next other image after the priority image.

[0360] (S1016) The moving image constructing unit 134 constructs an image indicating the position indicated by the reference position information in the fusion image finally constructed in S1014, and temporarily stores the image in a buffer (not shown). Note that this image is a fusion image.

[0361] (S1017) The video transmission unit 141 transmits the fusion image constructed in S1016. Note that the video transmission unit 141 may transmit the fusion image finally constructed in S1014. The destination of the fusion image is usually the user terminal 4 that transmitted the inquiry.

[0362] (S1018) The moving image transmission unit 141 determines whether or not to end the transmission of the fusion image. If the transmission of the fusion image is to be ended, the process proceeds to S1019, and if not, the process returns to S1001.

[0363] When an end instruction is received from the user terminal 4 that has transmitted the inquiry, the video transmitting unit 141 determines to end the transmission of the fusion image.

[0364] (S1019) The moving image constructing unit 134 temporally combines the transmitted fusion images to construct a fusion moving image, and temporarily stores the fusion moving image in a buffer (not shown).

[0365] (S1020) The moving image processing unit 135 performs a 3D process on the fusion moving image constructed in S1019. The process returns to the upper level process. The 3D process is a process of performing image processing on moving images that satisfy conditions and constructing a stereoscopic 3D moving image. An example of the 3D process will be described with reference to the flowchart of FIG. 18.

[0366] 10, the moving image transmitting unit 141 transmits the fusion image. However, the moving image transmitting unit 141 may transmit the moving image that has been subjected to the three-dimensional processing in S1020 instead of the processing in S1017.

[0367] Next, an example of the image fusion process in S1014 will be described with reference to the flowchart in FIG.

[0368] (S1101) The moving image composition unit 134 acquires a base image. Note that the first base image is the reference image acquired in S1009. Also, the base image is the base image updated in S1105 in the loop immediately before S1101 to S1105.

[0369] (S1102) The moving image composing unit 134 acquires area information that specifies the area covered by the base image acquired in S1101. Note that the area information is, for example, a set of position information of vertices that configure a polygon, or a set of position information of two diagonal corners that configure a rectangle.

[0370] (S1103) The moving image composition unit 134 acquires an image to be used for image fusion. Note that the other image to be used for image fusion is the j-th other image in S1011 or the priority image in S1013.

[0371] (S1104) The moving image composing unit 134 obtains area information that specifies the area covered by the image obtained in S1103.

[0372] (S1105) The moving image composing unit 134 uses the area information of the base image and the area information acquired in S1104 to determine an area in the base image in which the image acquired in S1103 is to be placed, places another image in that area, and acquires an updated base image. Then, the process returns to the upper process.

[0373] The two images on which the image fusion process is performed (the base image and the image acquired in S1103) each have an overlapping area. The moving image composition unit 134 then fuses the two images based on the overlapping area to compose one image. This image is the updated base image. Furthermore, image fusion process for two images having an overlapping area can be performed using known techniques.

[0374] Next, an example of the moving image combining process in S514 will be described with reference to the flowchart in FIG.

[0375] (S1201) The video acquisition unit 133 acquires location information of a target. The location information of the target is usually location information included in a query. The location information of the target is, for example, information that identifies a location close to a location indicated by location information corresponding to the video to be acquired.

[0376] (S1202) The moving image obtaining unit 133 assigns 1 to a counter i.

[0377] (S1203) The video acquisition unit 133 judges whether or not there is an i-th terminal to which video can be transmitted, in order from a terminal corresponding to the priority type (e.g., the fixed terminal 3) to a terminal not corresponding to the priority type (e.g., the mobile terminal 2). If there is an i-th terminal, the process proceeds to S1204, and if there is no i-th terminal, the process proceeds to S1214. The terminals that can transmit video are usually the fixed terminal 3 and the mobile terminal 2 that is moving.

[0378] (S1204) Video acquisition unit 133 acquires location information of the i-th terminal. When the i-th terminal is fixed terminal 3, video acquisition unit 133 normally acquires location information contained in the i-th fixed terminal information of fixed terminal management unit 112. When the i-th terminal is mobile terminal 2, video acquisition unit 133 acquires the latest location information transmitted by the i-th terminal.

[0379] (S1205) The video acquisition unit 133 determines whether or not the location information of the i-th terminal satisfies a first location condition for the target location information. If the first location condition is satisfied, the process proceeds to S1206. If the first location condition is not satisfied, the process proceeds to S1210.

[0380] (S1206) Video acquiring unit 133 acquires the video from the i-th terminal, pairs the video with the right holder identifier and location information of the i-th terminal, and temporarily stores the video in a buffer (not shown).

[0381] In addition, the video acquisition unit 133 may, for example, send an inquiry including a type identifier (e.g., "video" or "still image") that identifies the type of image requested to the i-th terminal, and receive an image corresponding to the inquiry from the i-th terminal.

[0382] (S1207) The video transmitting unit 141 transmits the video acquired in S1206. The video transmitting unit 141 normally transmits the video to the user terminal 4 that has transmitted the inquiry.

[0383] (S1208) The video acquisition unit 133 acquires the location information of the target. For example, if the target terminal 5 that transmits the location information is moving, the location information of the target acquired here differs from the location information acquired in S1201. However, if the target terminal 5 is not moving, if an inquiry having one fixed location information is received, etc., the location information of the target acquired here is the same as the location information acquired in S1201.

[0384] (S1209) The video acquisition unit 133 acquires the location information of the i-th terminal. Return to S1205. Note that if the i-th terminal is a mobile terminal 2, the location information of the i-th terminal here is updated location information. If the i-th terminal is a fixed terminal 3, the location information of the i-th terminal here does not change.

[0385] (S1210) The moving image composition unit 134 determines whether or not a transmitted moving image exists, which is a moving image acquired by the i-th terminal. If a transmitted moving image exists, the process proceeds to S1211, and if a transmitted moving image does not exist, the process proceeds to S1213.

[0386] (S1211) The moving image composing unit 134 composes a combined moving image by combining the moving image acquired by the i-th terminal with the moving image already transmitted. The moving image composing unit 134 also adds the rights holder identifier of the i-th terminal to the combined moving image, and temporarily stores the combined moving image in a buffer (not shown).

[0387] In the initial process of S1211, the combined moving image is the moving image acquired by the i-th terminal.

[0388] (S1212) The moving image obtaining unit 133 assigns 1 to the counter i. Then, the process returns to S1203.

[0389] (S1213) The moving image obtaining unit 133 increments the counter i by 1. The process returns to S1203.

[0390] (S1214) The moving image acquisition unit 133 determines whether or not to end the moving image combination process. If it is to be ended, the process proceeds to S1215, and if it is not to be ended, the process proceeds to S1216.

[0391] For example, if there is no next location information included in the received inquiry, or if an end instruction is received, the video acquisition section 133 determines to end the video combination process.

[0392] (S1215) The video acquiring unit 133 acquires the location information of the next target. For example, the video acquiring unit 133 acquires the next location information included in the route information that the inquiry has. Return to S1203.

[0393] (S1216) The moving image composing unit 134 composes a combined moving image by linking the multiple combined moving images composed in S1211 in the order in which they were acquired, and stores the combined moving image. Next, the moving image composing unit 134 temporarily stores the combined moving image in a buffer (not shown) in pairs with the multiple right holder identifiers and location information of the original moving images from which the combined moving image is composed.

[0394] It should be noted that if there is only one combined moving image constructed in S1211, the moving image constructing unit 134 does not perform a process of combining the combined moving images.

[0395] (S1217) The moving image processing unit 135 performs a three-dimensional processing on the combined moving image formed in S1216. Then, the processing returns to the upper level processing. An example of the three-dimensional processing will be described with reference to the flowchart of FIG.

[0396] Next, an example of the registered moving image search process in S515 will be described with reference to the flowchart in FIG.

[0397] (S1301) The moving image obtaining unit 133 assigns 1 to a counter i.

[0398] (S1302) The video acquisition unit 133 judges whether or not the i-th target location information is present in the received inquiry. If the i-th location information is present, the process proceeds to S1303, and if not, the process proceeds to S1317.

[0399] The target location information is location information used for searching for videos. The target location information is, for example, one piece of location information in the received inquiry, or each piece of location information included in the route information in the received inquiry.

[0400] (S1303) The video acquisition unit 133 acquires the i-th target location information from the received inquiry.

[0401] (S1304) The video acquisition unit 133 acquires all the maintenance information. For example, the video acquisition unit 133 acquires the maintenance information present on the blockchain.

[0402] (S1305) The moving image obtaining unit 133 assigns 1 to a counter j.

[0403] (S1306) The video acquisition unit 133 judges whether or not the j-th preservation information is present in the preservation information acquired in S1304. If the j-th preservation information is present, the process proceeds to S1307, and if not, the process proceeds to S1316.

[0404] (S1307) The video acquisition unit 133 checks the j-th security information and judges whether the video corresponding to the j-th security information can be provided to a third party. If it can be provided, the process proceeds to S1308, and if it cannot be provided, the process proceeds to S1315. Note that being available means, for example, that it can be viewed, that it can be sold, or that it can be viewed and sold.

[0405] (S1308) The moving image obtaining unit 133 assigns 1 to a counter k.

[0406] (S1309) The video acquisition unit 133 determines whether or not the kth search unit exists in the jth preservation information or the video corresponding to the jth preservation information. If the kth search unit exists, the process proceeds to S1310. If not, the process proceeds to S1315.

[0407] A search unit is a quantity of video that determines whether or not a query is satisfied. Videos in search units are, for example, video from the start to the end of shooting, video for a certain period of time, video until a predetermined event occurs (for example, video from a speed that is a moving object attribute value of 0 to the next speed 0, video from leaving a residence position (for example, a parking lot at home) to returning to the residence position), and video of a predetermined period before and after a predetermined tag (for example, "accident" or "traffic jam") is attached. However, the search unit is not important.

[0408] (S1310) The moving image obtaining unit 133 obtains a set of moving image attribute values ​​of the kth search unit from the jth maintenance information.

[0409] (S1311) The video acquisition unit 133 judges whether or not the set of video attribute values ​​of the kth search unit for the i-th target location information satisfies the first location condition. If the first location condition is satisfied, the process proceeds to S1312, and if not, the process proceeds to S1314.

[0410] (S1312) The moving image acquiring unit 133 acquires a moving image etc. corresponding to the j-th security information, and temporarily stores the moving image etc. in a buffer (not shown). The moving image etc. is, for example, a set of moving images and moving image attribute values, or a moving image.

[0411] (S1313) The moving image obtaining unit 133 increments the counter i by 1. The process returns to S1302.

[0412] (S1314) The moving image obtaining unit 133 increments the counter k by 1. The process returns to S1309.

[0413] (S1315) The moving image obtaining unit 133 increments the counter j by 1. The process returns to S1306.

[0414] (S1316) The moving image obtaining unit 133 increments the counter i by 1. The process returns to S1302.

[0415] (S1317) The moving image composing unit 134 determines whether or not multiple moving images, etc. have been acquired in S1312. If multiple moving images, etc. have been acquired, the process proceeds to S1318, and if only one moving image, etc. has been acquired, the process proceeds to S1319.

[0416] (S1318) The moving image composing unit 134 combines the multiple moving images stored in a buffer (not shown) in the order in which they were stored, composes a combined moving image, and stores the combined moving image in a buffer (not shown). Note that it is preferable that the moving image composing unit 134 also associates the rights holder identifier of the moving images that are the basis of the combined moving image and one or more moving image attribute values ​​with the combined moving image and stores them in a buffer (not shown).

[0417] (S1319) Video transmission unit 141 transmits the combined video or the acquired video. Then, the process returns to the upper level process. Note that it is preferable that the video be associated with a rights holder identifier and one or more video attribute values.

[0418] 13, each time a moving image is acquired in S1312, the moving image transmission unit 141 may transmit the moving image. In this case, the process of S1319 is not necessary.

[0419] In the flowchart of FIG. 13, the moving image sending unit 141 normally sends a combined moving image or a moving image to the user terminal 4 that has sent the inquiry.

[0420] Next, a first example of the unregistered video search process of S517 will be described with reference to the flowchart of Fig. 14. The first example of the unregistered video search process is a case in which attribute value sets of mobile terminal management unit 111 and fixed terminal management unit 112 are used.

[0421] (S1401) The moving image obtaining unit 133 assigns 1 to a counter i.

[0422] (S1402) The video acquisition unit 133 determines whether or not the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1403, and if not, the process proceeds to S1420.

[0423] (S1403) The video obtaining unit 133 obtains the i-th target position information from the received inquiry.

[0424] (S1404) The moving image obtaining unit 133 assigns 1 to a counter j.

[0425] (S1405) Video acquisition unit 133 judges whether or not the jth attribute value set exists in the fixed attribute value set of fixed terminal management unit 112 or the mobile attribute value set of mobile terminal management unit 111. If the jth attribute value set exists, the process proceeds to S1406, and if it does not exist, the process proceeds to S1419. Note that here, it is preferable for video acquisition unit 133 to check from the fixed attribute value set of fixed terminal management unit 112.

[0426] (S1406) The moving image obtaining unit 133 assigns 1 to a counter k.

[0427] (S1407) Video obtainment unit 133 determines whether or not the kth retrieval unit corresponding to the jth attribute value set exists. If the kth retrieval unit exists, the process proceeds to S1408, and if not, the process proceeds to S1414.

[0428] (S1408) Moving image obtaining unit 133 obtains a set of moving image attribute values ​​of the k-th retrieval unit in the j-th attribute value set from set storage unit 111. Note that there may be only one retrieval unit in the attribute value set.

[0429] (S1409) Moving image acquisition unit 133 determines whether or not the set of moving image attribute values ​​acquired in S1408 satisfies the query. If it does, the process proceeds to S1410, and if it does not, the process proceeds to S1414.

[0430] (S1410) The moving image obtaining unit 133 obtains a rights holder identifier corresponding to the moving image.

[0431] (S1411) The moving image acquiring unit 133 determines whether or not it is possible to acquire the moving image corresponding to the right holder identifier acquired in S1410. If it is possible to acquire the moving image, the process proceeds to S1412, and if it is not possible to acquire the moving image, the process proceeds to S1415.

[0432] Note that video can be acquired when the terminal corresponding to the j-th attribute value set is a fixed terminal 3, or when the terminal corresponding to the j-th attribute value set is a mobile terminal 2 and movement information for that terminal exists (for example, when the power of mobile terminal 2 is ON).

[0433] (S1412) The video acquiring unit 133 acquires the video corresponding to the k-th search unit from the terminal corresponding to the right holder identifier acquired in S1410, and temporarily stores the video in a buffer (not shown).

[0434] For example, the video acquisition unit 133 transmits a request for a video corresponding to the k-th search unit to the terminal, and receives the video corresponding to the request from the terminal. The range of videos corresponding to the k-th search unit does not matter. The videos corresponding to the k-th search unit may be all videos stored in the terminal, or may be only a part of the videos.

[0435] (S1413) The moving image obtaining unit 133 increments the counter k by 1. The process returns to S1407.

[0436] (S1414) The moving image obtaining unit 133 increments the counter j by 1. The process returns to S1405.

[0437] (S1415) The video obtaining unit 133 obtains state information etc. Note that the state information etc. includes, for example, state information indicating that the terminal is the mobile terminal 2 and that the power supply is OFF.

[0438] (S1416) Status transmission unit 142 transmits the status information etc. acquired in S1415 to the user who transmitted the inquiry.

[0439] (S1417) The video acquiring unit 133 acquires needs information.

[0440] (S1418) The needs transmitting unit 143 transmits the needs information acquired in S1417 to the rights holder of the video. Return to S1414.

[0441] (S1419) The moving image obtaining unit 133 increments the counter i by 1. The process returns to S1402.

[0442] (S1420) Moving image composing unit 134 determines whether or not multiple moving images were acquired in S1412. If multiple moving images were acquired, the process proceeds to S1421, and if only one moving image was acquired, the process proceeds to S1422.

[0443] (S1421) The moving image composing unit 134 combines the multiple moving images stored in a buffer (not shown) in S1412 in the order in which they were stored, composes one combined moving image, and stores the combined moving image in a buffer (not shown).

[0444] (S1422) Video sending unit 141 sends the combined video or a video, and then returns to the upper level process. Note that video sending unit 141 normally sends the combined video or a video to user terminal 4 that has sent the inquiry.

[0445] Next, a second example of the unregistered video search process of S517 will be described with reference to the flowchart of Fig. 15. The second example of the unregistered video search process is a case where an inquiry is made to the fixed terminal 3 or the mobile terminal 2.

[0446] (S1501) The moving image obtaining unit 133 assigns 1 to a counter i.

[0447] (S1502) The video acquisition unit 133 determines whether or not the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1503, and if not, the process proceeds to S1512.

[0448] (S1503) The video obtaining unit 133 obtains the i-th target position information from the received inquiry.

[0449] (S1504) The moving image obtaining unit 133 assigns 1 to a counter j.

[0450] (S1505) Video acquisition unit 133 determines whether or not a j-th terminal to which a query can be sent exists. If the j-th terminal exists, the process proceeds to S1503, and if the j-th terminal does not exist, the process proceeds to S1511.

[0451] The terminals that can transmit inquiries are terminals that are in a state where they can transmit moving images, and are fixed terminals 3 or mobile terminals 2 whose movement information exists in the storage unit 11 .

[0452] It is preferable that the video acquisition unit 133 selects as the j-th terminal only a fixed terminal 3 installed at a position indicated by location information that matches the first location condition for the i-th target location information. Also, it is preferable that the video acquisition unit 133 searches for a terminal that transmits a query from a terminal corresponding to the priority type (usually a fixed terminal 3), and when there is no terminal corresponding to the priority type that can transmit a query, searches for a terminal that does not correspond to the priority type (usually a mobile terminal 2).

[0453] (S1506) The video acquiring unit 133 transmits a query to the j-th terminal. Note that it is preferable that the video acquiring unit 133 transmits the query from a terminal corresponding to the priority type (usually, the fixed terminal 3) and transmits the query to a terminal not of the priority type (usually, the mobile terminal 2) if there is no terminal that can respond to the query.

[0454] (S1507) The video acquiring unit 133 determines whether or not information has been received from the j-th terminal. If information has been received, the process proceeds to S1508, and if not, the process returns to S1507.

[0455] (S1508) Video acquisition unit 133 determines whether or not a video is included in the information received in S1507. If a video is included, the process proceeds to S1509, and if not, the process proceeds to S1510.

[0456] (S1509) The moving image acquiring unit 133 temporarily stores the moving image etc. received in S1507 in a buffer (not shown). Then the process proceeds to S1511. Note that the moving image etc. is, for example, a moving image, an attribute value set, and a rights holder identifier.

[0457] (S1510) The moving image obtaining unit 133 increments the counter j by 1. The process returns to S1505.

[0458] (S1511) The moving image obtaining unit 133 increments the counter i by 1. The process returns to S1502.

[0459] (S1512) Moving image composing unit 134 determines whether or not multiple moving images were stored in S1509. If multiple moving images were stored, the process proceeds to S1513, and if only one moving image was acquired, the process proceeds to S1514.

[0460] (S1513) The moving image composing unit 134 combines the multiple moving images stored in the buffer (not shown) in S1509 in the order in which they were stored, composes one combined moving image, and stores the combined moving image in a buffer (not shown).

[0461] (S1514) The moving image transmitting unit 141 transmits the combined moving image or a moving image. The process returns to the upper level process. Note that the moving image transmitting unit 141 normally transmits the combined moving image or a moving image to the user terminal 4 that has transmitted the inquiry.

[0462] Next, an example of the security process in S518 will be described with reference to the flowchart in FIG.

[0463] (S1601) Rights holder processing unit 136 stores the transmitted moving image in association with a set of attribute values ​​corresponding to one or more original moving images from which the transmitted moving image is based.

[0464] It is preferable that the right holder processing unit 136 stores a video paired with a right holder identifier that identifies one or more right holders of the video. The right holder identifier here is, for example, one or more right holder identifiers of the video that is the source of the stored video. The right holder identifier here is, for example, a single right holder identifier that identifies the user who made the inquiry.

[0465] Furthermore, the right holder processing unit 136 accumulates the video in, for example, the storage unit 11 or a device other than the information processing device 1. The device other than the information processing device 1 may be a device that constitutes a blockchain.

[0466] (S1602) The fourth maintenance means 1364 performs the fourth maintenance process. An example of the fourth maintenance process has been described with reference to the flowchart in FIG.

[0467] (S1603) The right holder processing unit 136 assigns 1 to a counter i.

[0468] (S1604) Rights holder processing unit 136 judges whether or not the i-th moving image that was the source of the stored moving image exists. If the i-th moving image exists, the process proceeds to S1605, and if not, the process returns to the upper process.

[0469] (S1605) Reward means 1365 performs reward processing. An example of the reward processing will be described with reference to the flowchart in Fig. 17. The reward processing here is a processing for paying reward to the right holder of the i-th moving image that is the basis of the accumulated moving image.

[0470] (S1606) Rights holder processing unit 136 determines whether or not to change the rights holder of the i-th moving image that is the source of the stored moving image. If the rights holder is to be changed, the process proceeds to S1607. If the rights holder is not to be changed, the process proceeds to S1608.

[0471] In addition, whether or not to change the rights holder may be based on a flag corresponding to the i-th video, or may be decided in advance, or the rights holder may be changed if the inquiry includes "information indicating a request to change the rights holder."

[0472] (S1607) The right holder processing unit 136 acquires the user identifier of the user terminal 4. This user identifier becomes a new right holder identifier.

[0473] (S1608) Rights holder processing unit 136 determines whether or not the i-th moving image that is the source of the stored moving image has already been registered. If it has already been registered, the process proceeds to S1611. If it has not been registered, the process proceeds to S1609.

[0474] (S1609) The first maintenance unit 1361 stores the i-th moving image that is the source of the stored moving images.

[0475] (S1610) The fourth maintenance means 1364 performs a fourth maintenance process on the i-th moving image that is the source of the stored moving image. An example of the fourth maintenance process has been described with reference to the flowchart in FIG.

[0476] (S1611) The right holder processing unit 136 increments the counter i by 1. The process returns to S1604.

[0477] In the flowchart of Figure 16, when a combined video is constructed, a process (S1609) of storing the multiple videos that are the basis of the combined video and a fourth preservation process (S1610) are performed, but such processes do not have to be performed.

[0478] Next, an example of the reward process such as S523 will be described with reference to the flowchart of FIG.

[0479] (S1701) The reward means 1365 acquires one or more right holder identifiers of a target video. The reward means 1365 may acquire right holder identifiers of previous right holders of the target video.

[0480] (S1702) The reward means 1365 acquires a set of attribute values ​​of the target video.

[0481] (S1703) The reward means 1365 acquires a service identifier that identifies a service provided to the target video. The service identifier is, for example, “view” or “purchase.”

[0482] (S1704) The reward means 1365 acquires a reward amount using one or more pieces of information among the attribute value set acquired in S1702 and the service identifier acquired in S1703.

[0483] If multiple right holder identifiers are obtained, the remuneration means 1365 obtains a remuneration amount for each right holder identifier. If the right holder history information including multiple right holder identifiers is obtained, the remuneration means 1365 may obtain a remuneration amount for each right holder identifier.

[0484] In addition, it is preferable that the remuneration means 1365, for example, acquires a moving image attribute value corresponding to each of the multiple moving images that are the source of the moving image transmitted by the moving image transmission unit 141, and determines the amount of remuneration for each of the multiple right holders using the moving image attribute value. It is preferable that the remuneration means 1365, for example, determines a larger amount of remuneration as the data amount, the time of the moving image, or the number of frames of the original moving image adopted in the moving image transmitted by the moving image transmission unit 141 increases. It is preferable that the remuneration means 1365, for example, determines a larger amount of remuneration as the resolution of the original moving image adopted in the moving image transmitted by the moving image transmission unit 141 increases.

[0485] (S1705) The reward means 1365 performs a process of giving a reward of the reward amount acquired in S1704 to the right holder identified by the right holder identifier acquired in S1701.

[0486] (S1706) Reward means 1365 performs a process of charging the user who has enjoyed the service related to the target video with a reward. The process returns to the upper level process. Note that the target video is usually a video transmitted to user terminal 4.

[0487] In the flowchart of FIG. 17, the profits obtained by the operator of the information processing device 1 may be acquired and accumulated.

[0488] Next, an example of the three-dimensional processing in S1019 and S1217 will be described with reference to the flowchart in FIG.

[0489] (S1801) The moving image processing unit 135 assigns 1 to a counter i.

[0490] (S1802) The moving image processing unit 135 judges whether or not the i-th inspection unit is present in the target moving image. If the i-th inspection unit is present, the process proceeds to S1803, and if the i-th inspection unit is not present, the process returns to the upper process.

[0491] The inspection unit here refers to the inspection target of the moving image for determining at once whether the captured moving image to be processed is to be converted into a stereoscopic 3D moving image. The inspection unit is, for example, a predetermined number of frames, a predetermined time, each moving image that is the source of the captured moving image to be processed, the entire object to be processed, etc. The captured moving image to be processed is, for example, a fused moving image or a combined moving image.

[0492] (S1803) The moving image processing unit 135 acquires one or more moving image attribute values ​​which are used for determining processing conditions and which are paired with the moving image of the i-th inspection unit.

[0493] The one or more video attribute values ​​are, for example, a tag, time information, weather information, temperature information, and season information.

[0494] (S1804) The moving image processing unit 135 determines whether or not the one or more moving image attribute values ​​acquired in S1803 satisfy the processing condition. If the processing condition is satisfied, the process proceeds to S1805, and if the processing condition is not satisfied, the process proceeds to S1811.

[0495] (S1805) The moving image processing unit 135 assigns 1 to a counter j.

[0496] (S1806) The moving image processing unit 135 determines whether or not a j-th frame corresponding to the i-th inspection unit exists. If the j-th frame exists, the process proceeds to S1807, and if the j-th frame does not exist, the process proceeds to S1811.

[0497] Note that the frame corresponding to the i-th inspection unit is usually a frame included in the i-th inspection unit, but it may be a frame temporally before or after the i-th inspection unit. For example, if the processing condition is "tag=accident", the frames to be made into the stereoscopic 3D video may include frames before and after the accident.

[0498] (S1807) The moving image processing unit 135 obtains the j-th frame corresponding to the i-th inspection unit.

[0499] (S1808) The moving image processing unit 135 obtains a stereoscopic 3D image from the frames obtained in S1807.

[0500] (S1809) The moving image processing unit 135 replaces the j-th frame with the stereoscopic 3D image acquired in S1808.

[0501] (S1810) The moving image processing unit 135 increments the counter j by 1. The process returns to S1806.

[0502] (S1811) The moving image processing unit 135 increments the counter i by 1. The process returns to S1802.

[0503] Next, an example of the operation of the mobile terminal 2 will be described with reference to the flowchart of FIG.

[0504] (S1901) The movement processing unit 23 judges whether or not the start of movement has been detected. If the start of movement has been detected, the process proceeds to S1902, and if not, the process proceeds to S1917. The start of movement is, for example, when the engine of the moving object on which the mobile terminal 2 is installed is turned on, or when the power of the mobile terminal 2 is turned on.

[0505] (S1902) The movement information acquisition unit 233 acquires movement information. The movement information transmission unit 241 transmits the movement information to the information processing device 1. Note that the movement information is stored in the movement storage unit 21, for example.

[0506] (S1903) The photographing unit 231 starts photographing.

[0507] (S1904) The imaging unit 231 acquires a video.

[0508] (S1905) The movement processing unit 23 acquires one or more moving image attribute values, and associates the one or more moving image attribute values ​​with the moving image acquired in S1904.

[0509] The movement processing unit 23 acquires, for example, position information, time information, weather information, temperature information, and season information. In addition, the movement processing unit 23 acquires, for example, one or more moving object attribute values ​​(for example, CAN data).

[0510] (S1906) The tag acquisition unit 232 judges whether or not to acquire a moving object attribute value tag. If a moving object attribute value tag is to be acquired, the process proceeds to S1907, and if not, the process proceeds to S1909. Note that whether or not to acquire a moving object attribute value tag is determined in advance.

[0511] (S1907) The tag acquiring unit 232 performs a moving object attribute value tag acquiring process. An example of the moving object attribute value tag acquiring process has been described with reference to the flowchart in Fig. 7. Note that there may be cases where a tag cannot be acquired here.

[0512] (S1908) If one or more tags are acquired in S1907, the tag acquiring unit 232 associates the one or more tags with the video acquired in S1904.

[0513] It is preferable that the tag acquiring unit 232 associates one or more tags with the video and stores them in the mobile storage unit 21. This allows the user of the mobile terminal 2 to search for videos, for example, using the tags as a key.

[0514] (S1909) Tag acquiring unit 232 judges whether or not to acquire a moving image tag. If a moving image tag is to be acquired, the process proceeds to S1910, and if not, the process proceeds to S1912. Note that whether or not to acquire a moving image tag is determined in advance.

[0515] (S1910) The tag obtaining unit 232 performs a video tag obtaining process. An example of the video tag obtaining process has been described with reference to the flowchart in Fig. 8. Note that there may be cases where a tag cannot be obtained at this point.

[0516] (S1911) If one or more tags are acquired in S1910, the tag acquiring unit 232 associates the one or more tags with the video acquired in S1904.

[0517] It is preferable that the tag acquiring unit 232 associates one or more tags with the video and stores them in the mobile storage unit 21. This allows the user of the mobile terminal 2 to search for videos, for example, using the tags as a key.

[0518] (S1912) The movement processing unit 23 judges whether or not the end of the movement has been detected. If the end of the movement has been detected, the process proceeds to S1913, and if not, the process returns to S1904.

[0519] For example, the movement processing unit 23 detects the end of movement by detecting that the engine is turned off, that the power supply of the mobile terminal 2 is turned off, or that the mobile terminal 2 has arrived at the destination.

[0520] (S1913) The movement processing unit 23 acquires the right holder identifier from the movement storage unit 21.

[0521] (S1914) The movement processing unit 23 configures an attribute value set having one or more moving image attribute values ​​acquired in S1905, one or more moving object attribute values ​​and tags acquired in S1907, and one or more tags acquired in S1910. Note that the attribute value set does not have any information that was not acquired above.

[0522] (S1915) The set sending unit 243 sends the set of attribute values ​​constructed in S1914 to the information processing device 1 in association with the right holder identifier.

[0523] (S1916) The moving video transmission unit 242 etc. performs a video transmission process. Return to S1901. An example of the video transmission process will be described with reference to the flowchart in Fig. 20. The video transmission process is a process for automatically transmitting a video that matches a conservation condition.

[0524] (S1917) The mobile receiving unit 22 determines whether or not an inquiry has been received from the information processing device 1. If an inquiry has been received, the process proceeds to S1918, and if not, the process returns to S1901.

[0525] (S1918) The movement processing unit 23 performs a terminal image acquisition process corresponding to the inquiry received in S1917. An example of the terminal image acquisition process will be described with reference to the flowchart in Fig. 21. The terminal image acquisition process is a process of acquiring a video that matches the inquiry from among the videos stored in the movement storage unit 21.

[0526] (S1919) The movement processing unit 23 determines whether or not an image was acquired in S1918. If an image was acquired, the process proceeds to S1920, and if not, the process proceeds to S1921.

[0527] (S1920) The moving video transmission unit 242 transmits images and the like to the information processing device 1. Return to S1901. Note that the images and the like are, for example, an image, an attribute value set, and a rights holder identifier. Also, the images are, for example, a video, a still image, or an around-view image.

[0528] (S1921) The moving video transmission unit 242 transmits error information. The error information is information indicating that the video could not be acquired.

[0529] In the flowchart of Figure 19, it is preferable to transmit a set of video attribute values, which are all or a part of one or more video attribute values ​​acquired in S1905, moving object attribute values ​​and tags acquired in S1907, and tags acquired in S1910, to the information processing device 1 each time a video is acquired, at predetermined intervals, or when a predetermined condition is met (for example, the tag ``accident'' or ``traffic jam'' is acquired).

[0530] 19, after the end of movement is detected, the video transmission process (processing of S1916) is performed, but the video transmission process may be performed at all times. Performing the video transmission process at all times means, for example, performing the video transmission process (S1916) immediately after S1911. In this way, even if the storage area for videos in the storage unit 11 is small, videos that meet the preservation conditions can be preserved.

[0531] Furthermore, in the flowchart of FIG. 19, the process ends when the power is turned off or an interrupt for ending the process occurs.

[0532] Next, an example of the video transmission process in S1916 will be described with reference to the flowchart in FIG.

[0533] (S2001) Movement processing unit 23 acquires a set of attribute values ​​from movement storage unit 21.

[0534] (S2002) The movement processing unit 23 assigns 1 to a counter i.

[0535] (S2003) Movement processing unit 23 judges whether or not the i-th retrieval unit exists. If the i-th retrieval unit exists, the process proceeds to S2004, and if not, the process returns to the upper process.

[0536] (S2004) Movement processing unit 23 acquires a set of moving image attribute values ​​of the i-th search unit.

[0537] (S2005) The movement processing unit 23 assigns 1 to a counter j.

[0538] (S2006) The movement processing unit 23 judges whether or not the j-th maintenance condition exists. If the j-th maintenance condition exists, the process proceeds to S2007, and if not, the process proceeds to S2010.

[0539] (S2007) The movement processing unit 23 judges whether or not the set of video attribute values ​​acquired in S2004 satisfies the j-th conservation condition. If the j-th conservation condition is satisfied, the process proceeds to S2008, and if not, the process proceeds to S2011.

[0540] (S2008) The movement processing unit 23 acquires the video etc. of the i-th search unit. Note that the video etc. of the i-th search unit is, for example, the video of the i-th search unit and a set of attribute values ​​corresponding to the video. Also, the video of the i-th search unit may include, for example, the video before the i-th search unit and the video after the i-th search unit.

[0541] (S2009) The moving moving image transmission unit 242 transmits the moving image etc. acquired in S2008 to the information processing device 1.

[0542] (S2010) The movement processing unit 23 increments the counter i by 1. The process returns to S2003.

[0543] (S2011) The movement processing unit 23 increments the counter j by 1. The process returns to S2006.

[0544] Next, an example of the terminal image acquisition process in S1918 will be described with reference to the flowchart in FIG.

[0545] (S2101) The movement processing unit 23 uses the received inquiry to determine whether or not to acquire a moving image. If a moving image is to be acquired, the process proceeds to S2102, and if a still image or the like is to be acquired, the process proceeds to S2108.

[0546] Note that the movement processing unit 23 checks, for example, a type identifier (for example, "video", "still image", or "around-view image") included in the received inquiry, and determines whether or not to acquire a video.

[0547] (S2102) The movement processing unit 23 assigns 1 to a counter i.

[0548] (S2103) Movement processing unit 23 judges whether or not the i-th retrieval unit exists. If the i-th retrieval unit exists, the process proceeds to S2104, and if not, the process returns to the upper process.

[0549] (S2104) Movement processing unit 23 acquires a set of video attribute values ​​of the i-th search unit.

[0550] (S2105) Movement processing unit 23 determines whether or not the set of video attribute values ​​of the i-th search unit matches the query. If it matches the query, the process proceeds to S2106, and if it does not match, the process proceeds to S2107.

[0551] (S2106) The movement processing unit 23 acquires a moving image etc. in the i-th search unit. Note that the moving image etc. is, for example, a set of moving images and moving image attribute values.

[0552] (S2107) The movement processing unit 23 increments the counter i by 1. The process returns to S2103.

[0553] (S2108) The movement processing unit 23 acquires the latest composite image or the latest still image, and then returns to the upper processing.

[0554] The composite image is an image synthesized using still images captured by a plurality of cameras installed on a moving object. The composite image is, for example, an around-view image. The still image is an image captured by one camera installed on the moving object. The technology for constructing an around-view image is a known technology, and therefore a detailed description thereof will be omitted.

[0555] Next, an example of the operation of the fixed terminal 3 will be described with reference to the flowchart of FIG.

[0556] (S2201) Fixed camera 331 starts capturing images. Note that a trigger for starting capturing images may be, for example, power-on, or receipt of an instruction from a user.

[0557] (S2202) The fixed processing unit 33 acquires video captured by the fixed camera 331 and temporarily stores it in a buffer (not shown).

[0558] (S2203) The fixed processing unit 33 acquires one or more moving image attribute values. The one or more moving image attribute values ​​are, for example, time information, weather information, temperature information, and season information.

[0559] (S2204) The fixed processing unit 33 associates the one or more moving image attribute values ​​acquired in S2203 with the moving image acquired in S2202.

[0560] (S2205) The fixed processing unit 33 judges whether or not to acquire a moving image tag. If a moving image tag is to be acquired, the process proceeds to S2206, and if a moving image tag is not to be acquired, the process proceeds to S2208. Note that whether or not to acquire a moving image tag is determined in advance.

[0561] (S2206) The fixed processing unit 33 acquires a moving image tag. Note that an example of such moving image tag acquisition processing is the same as the processing in the flowchart in FIG.

[0562] (S2207) The fixed processing unit 33 associates the one or more tags acquired in S2206 with the moving image acquired in S2202.

[0563] (S2208) The fixed processing unit 33 judges whether or not to end the process. If it is judged that the process should be ended, the process ends, and if it is judged that the process should not be ended, the process proceeds to S2209. Note that the fixed processing unit 33 judges that the process should be ended, for example, when the power is turned off or when an end instruction is received.

[0564] (S2209) Fixed receiving unit 32 determines whether or not an inquiry has been received from information processing device 1. If an inquiry has been received, the process proceeds to S2210, and if not, the process proceeds to S2202.

[0565] (S2210) The fixed processing unit 33 acquires a moving image corresponding to the inquiry. An example of such fixed moving image acquisition processing will be described with reference to the flowchart in FIG.

[0566] (S2211) The fixed transmission unit 34 transmits the fixed moving image etc. acquired in S2210 to the information processing device 1. Note that the fixed moving image etc. refers to, for example, a fixed moving image, one or more moving image attribute values, and a rights holder identifier.

[0567] (S2212) The fixed processing unit 33 judges whether or not to end the transmission of the fixed moving image. If the transmission is to be ended, the process returns to S2202, and if the transmission is not to be ended, the process returns to S2210.

[0568] Next, an example of the fixed video acquisition process in S2210 will be described with reference to the flowchart in Fig. 23. In the flowchart in Fig. 23, descriptions of the same steps as in the flowchart in Fig. 22 will be omitted.

[0569] (S2301) The fixed processing unit 33 judges whether or not to acquire a past fixed video, which is a fixed video that has already been stored. If a past fixed video is to be acquired, the process proceeds to S2302. If a past fixed video is not to be acquired, the process proceeds to S2202. The past video is stored in a buffer (not shown).

[0570] It should be noted that, for example, when the received inquiry includes time information indicating a time in the past, the fixed processing unit 33 determines to acquire the past fixed video.

[0571] (S2302) The fixed processing unit 33 assigns 1 to the counter i.

[0572] (S2303) The fixed processing unit 33 judges whether or not the i-th retrieval unit exists in the fixed moving image existing in the fixed storage unit 31. If the i-th retrieval unit exists, the process proceeds to S2304, and if the i-th retrieval unit does not exist, the process returns to the upper process.

[0573] (S2304) Fixed processing unit 33 obtains from fixed storage unit 31 a set of attribute values ​​corresponding to the i-th retrieval unit.

[0574] (S2305) The fixed processing unit 33 judges whether or not the attribute value set acquired in S2304 matches the received query. If it matches the query, the process proceeds to S2306, and if it does not match the query, the process proceeds to S2307.

[0575] (S2306) The fixed processing unit 33 acquires the fixed video etc. corresponding to the i-th search unit. The fixed video etc. here are, for example, a fixed video, an attribute value set, and a rights holder identifier. The fixed video here may be the video of the i-th search unit, or may include a video temporally before and after the video of the i-th search unit based on the video of the i-th search unit.

[0576] (S2307) The fixed processing unit 33 increments the counter i by 1. The process returns to S2303.

[0577] Next, an example of the operation of the user terminal 4 will be described with reference to the flowchart of FIG.

[0578] (S2401) The user reception unit 42 judges whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S2402, and if not, the process proceeds to S2407.

[0579] Note that the reception of an inquiry is, for example, the reception of an input from a user, the reception of position information from the target terminal 5, or the reception of destination or route information from the navigation terminal 6.

[0580] (S2402) The user processing unit 43 composes a query to be transmitted. Next, the user transmission unit 44 transmits the query to the information processing device 1 in association with the user identifier.

[0581] (S2403) The user receiving unit 45 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S2404, and if information has not been received, the process returns to S2403.

[0582] (S2404) User processing unit 43 judges whether or not the information received in S2403 includes a moving image. If a moving image is included, the process proceeds to S2405, and if a moving image is not included, the process proceeds to S2406.

[0583] (S2405) User output unit 46 outputs the received video. Return to S2401.

[0584] (S2406) User output unit 46 outputs the received other information. Return to S2401. Note that the other information is, for example, information indicating that the video could not be acquired.

[0585] (S2407) The user processing unit 43 judges whether it is time to make an inquiry. If it is time to make an inquiry, the process proceeds to S2408. If it is not time to make an inquiry, the process proceeds to S2409.

[0586] The timing of the inquiry is, for example, when a destination is set in the user terminal 4 which is a navigation terminal, or when the user terminal 4 installed in the vehicle detects congestion.

[0587] (S2408) The user processing unit 43 performs an inquiry generation process. The process proceeds to S2402. An example of the inquiry generation process will be described with reference to the flowchart in FIG.

[0588] (S2409) The user reception unit 42 judges whether or not a purchase instruction has been received from the user. If a purchase instruction has been received, the process proceeds to S2410, and if not, the process returns to S2401. The purchase instruction includes, for example, an inquiry.

[0589] (S2410) The user processing unit 43 composes a purchase instruction to be transmitted. Next, the user transmitting unit 44 transmits the purchase instruction to the information processing device 1 in association with the user identifier.

[0590] (S2411) The user receiving unit 45 judges whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S2412, and if information has not been received, the process returns to S2411. The information is, for example, a moving image, information indicating that the transfer of the rights holder has been completed, and a set of attribute values ​​of the moving image.

[0591] (S2412) User processing unit 43 uses the received information to configure information to be output. User output unit 46 outputs the information. Return to S2401.

[0592] In the flowchart of FIG. 24, the process ends when the power is turned off or an interrupt occurs to end the process.

[0593] Next, an example of the inquiry generation process in S2408 will be described with reference to the flowchart in FIG.

[0594] (S2501) User processing unit 43 acquires a destination.

[0595] (S2502) User processing unit 43 acquires the current location.

[0596] (S2503) User processing unit 43 searches for a route using the destination and the current location, and acquires route information.

[0597] (S2504) The user processing unit 43 acquires one or more pieces of point information that are points on the route specified by the route information acquired in S2503 and that satisfy the extraction condition. Note that the point information is usually position information included in the route information.

[0598] (S2505) The user processing unit 43 constructs a query using one or more pieces of location information acquired in S2504. The query is returned to the upper level process. For example, the user processing unit 43 constructs a query such as "Search for videos that are within the next hour, have a tag of either "traffic jam" or "accident" added, and are paired with an attribute value set that includes one or more pieces of location information."

[0599] 25, the user processing unit 43 may obtain the current location and generate an inquiry including the current location. The inquiry is a request to transmit a video corresponding to a predetermined tag (e.g., "accident" or "traffic jam") in a surrounding area including the current location (e.g., a distance within a threshold).

[0600] Next, an example of the operation of the target terminal 5 will be described. The target terminal 5 acquires location information. Next, the target terminal 5 transmits the location information to the information processing device 1 or the user terminal 4 registered in the target terminal 5. Note that the target terminal 5 acquires and transmits the location information, for example, by receiving an instruction from the holder of the target terminal 5 (for example, a child to be watched over). Note that the target terminal 5 has, for example, a GPS receiver and acquires the location information by the GPS receiver, but the method of acquiring the location information is not important. Also, the user terminal 4 registered in the target terminal 5 is, for example, the guardian of the holder of the target terminal 5.

[0601] Next, an example of the operation of the navigation terminal 6 will be described. The navigation terminal 6 accepts a destination from a user. Next, the navigation terminal 6 performs the inquiry generation process described in FIG. 25. Next, the navigation terminal 6 performs the processes from S2402 to S2405 and S2406 in FIG. 24. That is, the navigation terminal 6 accepts the destination, acquires route information, queries the information processing device 1 using the route information, and receives from the information processing device 1 videos of one or more points on the route indicated by the route information, and outputs them. <Specific operation example>

[0602] A specific example of the operation of the information system A in this embodiment will be described below.

[0603] Currently, a mobile terminal management table having the structure shown in Fig. 26 is stored in mobile terminal management unit 111 of information processing device 1. The mobile terminal management table is a table for managing information on one or more mobile terminals. The mobile terminal management table is a table for managing mobile terminals 2 that transmit moving videos. Note that a mobile terminal 2 that is not managed in the mobile terminal management table may transmit moving videos to information processing device 1.

[0604] The mobile terminal management table manages one or more records having an "ID," "terminal identifier," "video information," "movement information," "registration flag," and "provision flag." "Video information" is information about a captured video, and has a "frame identifier" and a "video attribute value." The "video attribute value" has "environmental information" and "tag." "Environmental information" has "location information," "direction information," "camera information," "time information," "weather information," and "temperature information." "Camera information" has "angle of view" and "resolution." "Environmental information" is information about the environment around the mobile terminal 2 when the video was captured, etc. "Tags" include "accident," "traffic jam," and "dangerous driving." In other words, here, a video may be assigned one or more tags, for example, "accident," "traffic jam," and "dangerous driving."

[0605] "ID" is information for identifying a record. "Terminal identifier" is an identifier of the mobile terminal 2, and is the same as the right holder identifier for identifying the right holder of the video at the time of transmission of the video. "Frame identifier" is the ID of a frame constituting the video. A frame may be called a field or a still image. "Location information" is (latitude, longitude) here. "Direction information" is the shooting direction of the camera, and here, is the angle in a clockwise direction from true north. In other words, if the shooting direction of the camera is due east, the direction information is "90 degrees". "Angle of view" is the angle of view of the camera. "Resolution" is the resolution of the camera. "Time information" is here, year, month, day, hour, minute, and second. "Weather information" is, for example, "sunny", "rain", "cloudy", or "snow". "Temperature information" is the temperature (℃) outside the moving object. A value of "1" for "accident" indicates that a tag indicating that an accident has occurred is attached to the corresponding frame. A value of "1" for "traffic jam" indicates that a tag indicating that a traffic jam has occurred is attached to the corresponding frame. A "Dangerous Driving" value of "1" indicates that a tag indicating that the car or the like in front is driving dangerously has been attached to the corresponding frame. "Movement Information = 1" indicates that the mobile terminal 2 is currently in a state where it can transmit video. "Movement Information = 0" indicates that the mobile terminal 2 is currently in a state where it cannot transmit video due to the power being turned off, etc. "Registration Flag = 1" indicates that the video has already been registered and can be obtained from the registration destination (e.g., storage unit 11 or another device). "Provision Flag" of "1" indicates that viewing of the video is permitted. "Provision Flag" of "2" indicates that sale of the video (transfer of rights holder) is permitted.

[0606] Furthermore, the fixed terminal management unit 112 of the information processing device 1 stores a fixed terminal management table having a structure shown in Fig. 27. The fixed terminal management table is a table for managing information on one or more fixed terminals. The fixed terminal management table is a table for managing fixed terminals 3 that transmit fixed videos.

[0607] The fixed terminal management table manages one or more records having an "ID," "terminal identifier," "location information," "direction information," and "camera information." The "camera information" here includes an "angle of view" and a "resolution."

[0608] Furthermore, a registered video management table having the structure shown in FIG. 28 is stored on the blockchain. The registered video management table is a table for managing registered videos. The registered video management table is a table for managing preservation information. Registered videos are, for example, videos provided to a user in response to an inquiry from the user, and videos that satisfy preservation conditions. In other words, here, videos that match an inquiry received by the information processing device 1 and videos that satisfy preservation conditions are automatically accumulated.

[0609] The registered video management table manages one or more records having an "ID," "video identifier," "access information," "rights holder identifier," "rights registration date," "video information," and "provision flag." It is assumed that registered videos are stored in the information processing device 1, another device, or the blockchain. It is also assumed that each record in the registered video management table is preservation information here.

[0610] In this context, five specific examples will be described. Specific examples 1 to 4 are examples of using real-time video. Specific example 5 is an example of using video captured in the past. <Example>

[0611] Specific example 1 is a case in which, in order to monitor a target person (e.g., a child or a wandering elderly person), a combined video that combines fixed images taken by one or more fixed terminals 3 and moving videos taken by one or more mobile terminals 2 installed on each of the moving objects is transmitted to a user terminal 4 of a user related to the target person (e.g., a guardian).

[0612] Specific example 2 is a case in which a combined video combining one or more fixed images and one or more moving videos is output to the user terminal 4 in response to an inquiry using route information corresponding to a destination set in the user terminal 4 or the navigation terminal 6.

[0613] Specific example 3 is a case where the user terminal 4 or the navigation terminal 6 detects a traffic jam, obtains a combined video that allows the user to understand the reason for the traffic jam, and outputs the combined video to the user terminal 4 or the navigation terminal 6. Note that the combined video here is a video in which one or more fixed images and one or more moving videos are combined.

[0614] Concrete example 4 is a case where a fused video is output that allows a user to grasp the situation of a parking lot so that the user can find an empty space in the parking lot. Note that the fused video here is a video that is composed of one or more fixed images and one or more moving videos fused together. In other words, one or more fixed terminals 3 are installed in the parking lot.

[0615] Concrete example 5 is a case where a combined video obtained by combining registered videos using route information that identifies a route traveled by a user in the past (for example, a route traveled) is output to a user terminal 4. Note that the combined video here is a video obtained by combining one or more fixed images and one or more moving videos.

[0616] (Example 1) The storage unit 11 of the information processing device 1 stores management information having a user identifier (e.g., the IP address of the user terminal 4) for sending a video to a user terminal 4 of a guardian P, and a target identifier "T001" that identifies a target terminal 5 of a child A of the guardian P.

[0617] Next, assume that child A turns on the target terminal 5 to return home from school. Next, the target terminal 5 periodically acquires location information and sends a query including the location information and the target identifier "T001" (for example, "Video transmission instruction Target identifier=T001, location information (x t1 ,y t1 )") to the information processing device 1. It is assumed that the target terminal 5 stores a target user identifier "T001" and communication destination information (e.g., IP address) of the information processing device 1 for transmitting information to the information processing device 1.

[0618] In addition, the receiving unit 12 of the information processing device 1 periodically receives location information that identifies the location of the mobile terminal 2, paired with the terminal identifier, from each of one or more mobile terminals 2, and stores the location information in a mobile terminal management table (Figure 26), paired with the terminal identifier.

[0619] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry “video transmission instruction target identifier=T001, location information (x t1 ,y t1 )" from the target terminal 5 and stores the location information (x t1 ,y t1 ) and the target user identifier “T001.” It is assumed that the location information of the target terminal 5 is received periodically, and the latest location information is stored in a buffer (not shown).

[0620] Next, the video acquisition unit 133 determines the fixed terminal 3 or the first mobile terminal 2 corresponding to the location information that satisfies the first location condition with respect to the location indicated by the latest location information of the target terminal 5, by the operation described in the flowchart of FIG. 12. Note that here, it is assumed that the priority video is a "fixed video". Then, the video acquisition unit 133 checks whether or not there is location information, direction information, and angle of view that satisfies the first location condition (that the location indicated by the latest location information of the target terminal 5 is photographed) with respect to the latest location information of the target terminal 5, from the fixed terminal management table (FIG. 27). Then, if there is no location information, etc. that satisfies the first location condition in the fixed terminal management table (FIG. 27), it is determined whether or not the latest location information, direction information, and angle of view of each mobile terminal 2 in the mobile terminal management table satisfies the first location condition with respect to the latest location information of the target terminal 5.

[0621] Then, it is assumed that the video acquiring unit 133 has determined that the fixed terminal management table (FIG. 27) does not contain location information etc. that satisfies the first location condition, and therefore no video of any of the fixed terminals 3 can be used.

[0622] Next, it is assumed that the video acquisition unit 133 has determined a first terminal (here, a mobile terminal 2 mounted on an automobile). The video acquisition unit 133 then receives a video from the first mobile terminal 2. Child A appears in the video. It has been determined that the first location condition is satisfied based on location information and the like of the target terminal 5 of child A. A video is usually associated with location information and a first rights holder identifier of the mobile terminal 2, which is the first terminal. It is also preferable that the received video is a video that was transmitted immediately after being shot by the mobile terminal 2.

[0623] Next, the video sending unit 141 sends the video received by the video acquiring unit 133 to the user terminal 4 of the guardian P who is identified by the user identifier paired with the target person identifier "T001".

[0624] Then, the video acquisition unit 133 temporarily stores the video received from the first terminal in a buffer (not shown) in a pair with the first rights holder identifier and the location information of the first terminal.

[0625] In addition, the video acquisition unit 133 receives at least the video received from the first terminal while the location information of the target terminal 5 of child A and the location information of the first terminal, etc., satisfy the first location condition, and the video transmission unit 141 transmits the video to the user terminal 4 of the guardian P.

[0626] Next, it is assumed that the video acquisition unit 133 determines that the location information of the target terminal 5 of child A and the location information of the first terminal, etc., no longer satisfy the first location condition (child A is no longer captured). This is because the mobile terminal 2 has moved and is no longer in a position to capture child A, who is walking home.

[0627] Next, the video acquisition unit 133 again tries to determine, by referring to the fixed terminal management table (FIG. 27), the location information of a second terminal that satisfies the first location condition for the latest location information of the target terminal 5. Here, the video acquisition unit 133 determines that the area captured based on the location information (FX1, FY1), direction information "Fd1", and angle of view "FD1" in the fixed terminal management table (FIG. 27) satisfies the first location condition for the latest location information of the target terminal 5.

[0628] Next, the video acquisition unit 133 transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier "U101", and receives the fixed video from the fixed terminal 3. Next, the video acquisition unit 133 temporarily stores the video in a buffer (not shown) in a pair with the second rights holder identifier "U101" of the second terminal (the fixed terminal 3) and location information.

[0629] Next, the video sending unit 141 sends to the user terminal 4 of the guardian P the fixed video that has been received by the video acquiring unit 133 and sent from the second terminal.

[0630] Then, after the passage of time (for example, 10 seconds), the video acquisition unit 133 determines that the position information of Child A's target terminal 5 and the position information, direction information, and angle of view of the second terminal no longer satisfy the first position condition (Child A is no longer in the image).

[0631] Next, the video acquisition unit 133 again attempts to determine the location information of a third terminal whose latest location information satisfies the first location condition by referring to the fixed terminal management table in Figure 27 for the latest location information of the target terminal 5, but it is assumed that there is no location information of a third terminal (fixed terminal 3) that satisfies the first location condition.

[0632] Next, the video acquisition unit 133 refers to Fig. 26 and attempts to determine the location information, etc. of the mobile terminal 2 whose latest location information satisfies the first location condition with respect to the latest location information of the target terminal 5, but assumes that there is no location information, etc. of the mobile terminal 2 whose latest location information satisfies the first location condition. In such a case, the video transmission unit 141 may transmit information to the user terminal 4 of the guardian P that the video containing the child A cannot be transmitted, or may stop transmission of the video or information to the user terminal 4.

[0633] Through the above video transmission, the user terminal 4 sequentially receives and outputs the moving video acquired by the mobile terminal 2 and the fixed video acquired by the fixed terminal 3. This allows the guardian P to see how the child A returns home.

[0634] Then, as time passes, a third terminal, a mobile terminal 2, approaches child A, and the video acquisition unit 133 is able to determine the location information, etc. of the third terminal that satisfies the first location condition with respect to the latest location information of the target terminal 5 by referring to Fig. 26. However, it is assumed that the video acquisition unit 133 was unable to detect the location information, etc. of the fixed terminal 3 that satisfies the first location condition with respect to the latest location information of the target terminal 5.

[0635] Next, the video acquisition unit 133 acquires the video from the third terminal. Next, the video acquisition unit 133 temporarily stores the video in a buffer (not shown) in a pair with the third rights holder identifier and location information of the third terminal.

[0636] Next, the video sending unit 141 sends the video received by the video acquiring unit 133 from the third terminal to the user terminal 4 of the guardian P identified by the user identifier paired with the target person identifier "T001".

[0637] Next, the user terminal 4 receives and outputs the video acquired by the third terminal. As a result, the guardian P is able to see the child A's return home again, although he was unable to see the child A's return home for a short time.

[0638] The above process continues until the target terminal 5 of child A is powered off (until child A returns home), and parent B can monitor child A's return home from school.

[0639] In addition, the reward means 1365 performs the above-mentioned reward processing for the right holders identified by the right holder identifiers of the first mobile terminal 2, the fixed terminal 3 which is the second terminal, and the third mobile terminal 2 which provided the video to guardian B.

[0640] Furthermore, moving image composer 134 combines the moving images transmitted from each mobile terminal 2 and fixed terminal 3 in the order in which they were transmitted, to compose a combined moving image.

[0641] Moreover, the right holder processing unit 136 stores the combined moving image in a pair with a right holder identifier, which is the identifier of the guardian P.

[0642] Moreover, the rights holder processing unit 136 stores the combined moving image in association with a set of attribute values ​​corresponding to one or more original moving images from which the combined moving image is created.

[0643] Next, the fourth maintenance means 1364 acquires access information that specifies the storage destination of the combined video. The fourth maintenance means 1364 also acquires a set of attribute values ​​corresponding to the accumulated combined video. Next, the fourth maintenance means 1364 constructs maintenance information having the acquired access information, the acquired set of attribute values, and the rights holder identifier of the video. Next, the fourth maintenance means 1364 accumulates the constructed maintenance information in the blockchain. An example of such maintenance information is the record of "ID=2" in FIG. 28.

[0644] As described above, according to this specific example, the combined video can be used to monitor the subject who has the target terminal 5. In addition, a reward can be given to the right holder who provided the video that is the source of the combined video for monitoring. In addition, the combined video can be appropriately managed.

[0645] (Example 2) It is assumed that a user inputs a destination to a user terminal 4 having a navigation function. Next, the user accepting unit 42 of the user terminal 4 accepts the destination. In addition, the user processing unit 43 acquires a current position. Next, the user processing unit 43 acquires route information from the current position to the destination. It is assumed here that the route information includes multiple pieces of position information.

[0646] Next, in response to the acquisition of the route information by the user processing unit 43, the user transmitting unit 44 of the user terminal 4 automatically transmits an inquiry including the route information to the information processing device 1.

[0647] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry. Next, the moving image acquiring unit 133 and the moving image composing unit 134 perform moving image combining processing as follows.

[0648] That is, first, the video acquisition unit 133 acquires, from among a plurality of pieces of position information contained in the route information, position information (first position information) of the first intersection in the route specified by the route information. That is, it is preferable that the video acquisition unit 133 acquires a video using only a portion of the position information contained in the received route information. It is preferable that the video acquisition unit 133 acquires a video using only a portion of the position information contained in the received route information that satisfies a specific condition. Note that the specific condition is, for example, that the position information is information indicating the position of an intersection, and that the distance from the previously used position information is equal to or greater than a threshold.

[0649] Next, the video acquisition unit 133 determines the position information, direction information, and angle of view of the first terminal that satisfies the first position condition for the acquired position information. The first terminal is the fixed terminal 3 or the mobile terminal 2. In addition, the video acquisition unit 133 preferably searches the fixed terminal management table (FIG. 27) according to the priority type "fixed terminal", and if the position information, etc. of the first terminal cannot be determined, searches the mobile terminal management table (FIG. 26).

[0650] Then, the video acquisition unit 133 acquires the video from the determined first terminal, pairs the video with the first right holder identifier of the first terminal, the first location information, etc., and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.

[0651] Next, the video acquisition unit 133 acquires second position information of the next intersection that is closer to the destination than the position indicated by the previously acquired first position information of the intersection.

[0652] Next, the video acquisition unit 133 determines the location information of the second terminal that satisfies the first location condition for the second location information. The second terminal is the fixed terminal 3 or the mobile terminal 2.

[0653] Then, the video acquisition unit 133 acquires the video from the determined second terminal, pairs the video with the second rights holder identifier and second location information of the second terminal, and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.

[0654] Next, the video acquisition unit 133 acquires third position information of the next intersection that is closer to the destination than the position indicated by the second position information.

[0655] Next, the video acquisition unit 133 determines the location information, etc. of a third terminal that satisfies the first location condition for the third location information. Then, the video acquisition unit 133 acquires the video from the determined third terminal, pairs the video with the third rights holder identifier of the third terminal and the third location information, etc., and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.

[0656] The information processing device 1 repeats the above process until transmitting a video corresponding to the n-th location information that identifies the destination.

[0657] Then, the user receiving unit 45 of the user terminal 4 sequentially receives the first moving image, the second moving image, the third moving image, ..., the nth moving image. The user output unit 46 sequentially outputs the first moving image, the second moving image, the third moving image, ..., the nth moving image.

[0658] Furthermore, the reward means 1365 of the information processing device 1 performs reward processing to reward the first right holder, the second right holder, . . . , and the n-th right holder identified by the first right holder identifier for providing the video.

[0659] Furthermore, the moving image constructing unit 134 constructs a combined moving image by combining the moving images transmitted from the multiple terminals in the order in which they were transmitted. As a result, the combined moving image is a moving image in which, for example, the moving moving image transmitted from the mobile terminal 2 and the fixed moving image transmitted from the fixed terminal 3 are combined in time in the order in which they were received.

[0660] Moreover, the rights holder processing unit 136 stores the combined moving image in a pair with a rights holder identifier that is an identifier of the user of the user terminal 4. That is, in this case, the rights holder of the combined moving image is the user.

[0661] Moreover, the rights holder processing unit 136 stores the combined moving image in association with a set of attribute values ​​corresponding to one or more original moving images from which the combined moving image is created.

[0662] Next, the fourth security means 1364 acquires access information that specifies the storage destination of the combined moving image. The fourth security means 1364 also acquires a set of attribute values ​​corresponding to the stored combined moving image. Next, the fourth security means 1364 constructs security information having the acquired access information, the acquired set of attribute values, and the rights holder identifier of the moving image. Next, the fourth security means 1364 stores the constructed security information.

[0663] As described above, according to this specific example, videos transmitted from multiple mobile terminals 2 are, at least in appearance, combined in time to create a combined video, which can be used to check the route to the destination in order of proximity from the current position, thereby providing support for traveling using a mobile object such as a car.

[0664] In addition, a reward can be given to the rights holder who provided the original video of the combined video, and the combined video can be appropriately stored and managed.

[0665] (Example 3) For example, the navigation terminal 6 detects a traffic jam on the route to the destination. The function of detecting a traffic jam can be realized by known technology. Next, the navigation terminal 6 acquires route information having one or more pieces of location information that specify a road with a traffic jam, and transmits an inquiry including the route information and a user identifier stored in the navigation terminal 6 to the information processing device 1. The user identifier is an identifier of the user terminal 4 that receives a video transmitted from the fixed terminal 3 or the mobile terminal 2 that is located at the position of the road with a traffic jam, and is, for example, an IP address. Also, the route information is information that specifies one or more roads with a traffic jam. The user terminal 4 is, for example, a terminal of a passenger in the front passenger seat.

[0666] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry including the route information and the user identifier. Next, the moving image acquiring unit 133 and the moving image composing unit 134 perform moving image combining processing as follows.

[0667] First, the video acquisition unit 133 acquires the location information closest to the current location among the location information included in the route information. Note that the location information closest to the current location is, for example, the first location information among the location information included in the route information.

[0668] Next, the video acquisition unit 133 acquires the last piece of position information that is the next piece of position information and is located within the threshold value relative to the adjacent pieces of position information. The video acquisition unit 133 sets the acquired position information as the target position information to be used for video acquisition.

[0669] Next, the video acquisition unit 133 determines a first terminal, such as location information, that satisfies a first location condition for the target location information. The first terminal is a fixed terminal 3 or a mobile terminal 2. Here, it is preferable that the video acquisition unit 133 searches from the fixed terminal 3.

[0670] Next, the video acquisition unit 133 acquires the video from the terminal, pairs the video with the first rights holder identifier of the first terminal, location information, etc., and temporarily stores the video in a buffer (not shown).

[0671] Then, the video acquisition unit 133 receives video from the first terminal and temporarily stores the video in a buffer (not shown) until the location information, etc. of the first terminal no longer satisfies the first location condition.

[0672] Moreover, the video transmitting unit 141 transmits the acquired videos to the user terminal 4 in sequence.

[0673] Next, it is assumed that the mobile terminal 2 leaves the congestion and the video acquisition unit 133 determines that the location information etc. of the first terminal no longer satisfies the first location condition.

[0674] Next, the video acquisition unit 133 determines a second terminal having location information, etc., that satisfies the first location condition for the target location information. The first terminal is the fixed terminal 3 or the mobile terminal 2.

[0675] Next, the video acquisition unit 133 acquires the video from the second terminal, pairs the video with the second rights holder identifier and location information, etc. of the second terminal, and temporarily stores the video in a buffer (not shown).

[0676] Then, the video acquisition unit 133 receives video from the second mobile terminal 2 and temporarily stores the video in a buffer (not shown) until the location information, etc. of the second terminal no longer satisfies the first location condition.

[0677] Moreover, the video transmitting unit 141 transmits the acquired videos to the user terminal 4 in sequence.

[0678] Then, the user terminal 4 receives the video from the information processing device 1 and outputs it.

[0679] By repeating the above process, the user of the user terminal 4 can continuously know the state of the traffic jam. In addition, the user can know the point where the traffic jam will be resolved.

[0680] Furthermore, when the route information includes information identifying two or more congested roads, the information processing device 1 performs the same process using the route information identifying the second and subsequent locations. As a result, the user terminal 4 can also receive and output video for grasping the state of congestion at the second and subsequent locations. Note that, when there are two or more congested locations, it is preferable that the user terminal 4 automatically or in response to a user's instruction switches between the locations and receives and outputs the video.

[0681] In this specific example, although explanation is omitted, reward processing, various security processing, etc. may also be performed.

[0682] As described above, according to this specific example, by using the video transmitted from a plurality of terminals including the fixed terminal 3 and the mobile terminal 2, the state of traffic congestion on the route to the destination can be grasped.

[0683] (Example 4) It is assumed that a user terminal 4 installed in a car in a large parking lot transmits an inquiry including position information of the user terminal 4 to ascertain the situation of the parking lot to the information processing device 1. It is assumed that the inquiry includes "type identifier=around view image", position information of the user terminal 4 (reference position information), and the around view image acquired by the user terminal 4.

[0684] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry. Next, the moving image acquiring unit 133 determines from the received inquiry that spatial fusion of multiple moving images is necessary. Next, the moving image acquiring unit 133 and the moving image composing unit 134 perform moving image fusion processing as follows.

[0685] That is, the video acquisition unit 133 acquires the reference location information contained in the received inquiry. Next, the video acquisition unit 133 determines one or more mobile terminals 2 corresponding to location information that satisfies the second location condition with respect to the reference location information. Next, the video acquisition unit 133 transmits an instruction to transmit the current around-view image to one or more terminals. Note that each of the multiple terminals is a fixed terminal 3 or a mobile terminal 2. Also, the second location condition here is that the location indicated by the location information is present in the area of ​​the parking lot that includes the reference location information. Note that it is assumed that each of the one or more mobile terminals 2 continuously acquires around-view images. It is also assumed that one or more fixed terminals 3 are installed on the ceiling of the parking lot, capture an image directly below, and acquire around-view images.

[0686] Then, the video acquisition unit 133 receives, from each of the one or more terminals, an around-view image associated with the position information of the terminal.

[0687] Next, the moving image composing unit 134 calculates the difference (distance) between the position information paired with each received around-view image and the reference position information. Next, the moving image composing unit 134 sorts the around-view images in ascending order using the difference as a key.

[0688] Next, the moving image composition unit 134 acquires a reference image. Note that the reference image is an around-view image included in the query.

[0689] Next, the moving image composition unit 134 performs the process described using the flowchart in FIG. 11, and arranges each of the around-view images received from other terminals in an appropriate position relative to the reference image, in the sorted order, using the position information, to compose a fusion image.

[0690] Furthermore, the moving image constructing unit 134 constructs an image that clearly indicates the position indicated by the reference position information (the position of the user terminal 4 that sent the inquiry) in the finally constructed fusion image. Note that such an image is also a fusion image.

[0691] Next, the moving image transmission unit 141 transmits the constructed fusion image to the user terminal 4.

[0692] The user terminal 4 then receives and outputs the fusion image.

[0693] Furthermore, the information processing device 1 repeats the above process, and as a result, transmits a fusion moving image including a plurality of fusion images to the user terminal 4. Then, the user terminal 4 receives and outputs the fusion moving image.

[0694] As described above, according to this embodiment, images transmitted from one or more other mobile terminals 2 or fixed terminals 3 can be fused to assist in finding an empty space in a parking lot.

[0695] In this specific example, the information processing device 1 fuses around-view images. However, the information processing device 1 may fuse multiple normal camera images or multiple images captured by an omnidirectional camera using position information corresponding to each image to construct a fused image and transmit the fused image to the user terminal 4. Also, a video of a fixed terminal 3 installed at an event venue such as a live performance may be fused with a video of a user terminal 4 held by an audience member who has gathered at the event venue, and the video of an area that could not be captured by the user terminal 4 held by the audience member may be supplemented with the video of the fixed terminal 3.

[0696] (Example 5) It is assumed that user B drives a driving course on a rainy day, and that route information specifying the route that user B drove by car is stored in the user terminal 4.

[0697] Next, assume that after returning home, user B inputs an inquiry including the route information stored in the user terminal 4 and the environmental information "weather information = sunny" to the user terminal 4. Then, the user terminal 4 accepts the inquiry and transmits the inquiry to the information processing device 1. Note that this inquiry is an inquiry for acquiring a video of the driving course traveled by the user on a sunny day.

[0698] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry. Next, the video acquiring unit 133 determines that the received inquiry is not a real-time video search. Next, the video acquiring unit 133 and the like perform a registered video search process as follows.

[0699] That is, the video acquisition unit 133 acquires the first location information contained in the route information in the received inquiry.

[0700] Next, the video acquisition unit 133 acquires all the preservation information from the registered video management table (FIG. 28). That is, the video acquisition unit 133 acquires, for example, the preservation information present on the block chain.

[0701] Next, the video acquisition unit 133 determines, from among the acquired conservation information, conservation information that includes "1" in the provision flag, has location information that satisfies the first location condition for the acquired location information, and includes "weather information = sunny." Here, it is assumed that the video acquisition unit 133 has determined the conservation information of "ID = 1" in FIG. 28, for example.

[0702] Next, the video acquisition unit 133 acquires the access information "Address 01" that pairs with the determined security information. Next, the video acquisition unit 133 acquires a video using the access information "Address 01". Next, the video acquisition unit 133 cuts out a video from the video corresponding to the access information "Address 01" up to the point where the second or subsequent location information in the route information of the received inquiry satisfies (is the same as or is close to) the first location condition as the location information of the security information, etc. In other words, the video acquisition unit 133 cuts out and acquires a video from the video corresponding to the access information "Address 01" up to the point where the user deviates from the route of the driving course traveled by the user.

[0703] Next, the video acquisition unit 133 acquires second location information, which is location information when the video corresponding to the access information "address 01" deviates from the driving course and is included in the route information included in the received inquiry.

[0704] Then, the video acquisition unit 133 determines, from the acquired conservation information, conservation information that includes a provision flag of "1", has location information that satisfies the first location condition for the acquired second location information, and includes "weather information = sunny".

[0705] Next, the video acquisition unit 133 acquires access information (for example, "Address X") that pairs with the determined security information. Next, it is assumed that the video acquisition unit 133 acquires a second video using the access information "Address X". Next, the video acquisition unit 133 cuts out from the second video corresponding to the access information "Address X" a video up to the point where the location information beyond the second location information contained in the route information contained in the received inquiry satisfies the location information of the security information. In other words, the video acquisition unit 133 cuts out and acquires from the video corresponding to the access information "Address X" a video up to the point where the user deviates from the route of the driving course traveled by the user.

[0706] The video acquisition unit 133 performs the above process until it uses the last location information contained in the route information in the received inquiry.

[0707] Next, the moving image constructing unit 134 constructs a combined moving image by combining the multiple moving images acquired by the moving image acquiring unit 133 in the order in which they were acquired (in the order of the route). It is preferable that the multiple moving images that are the source of the combined moving image include a fixed moving image and a moving moving image.

[0708] Next, the moving image transmission unit 141 transmits the combined moving image to the user terminal 4 of the user B.

[0709] Next, the user terminal 4 receives and outputs the combined moving image.

[0710] In addition, the reward means 1365 of the information processing device 1 acquires the rights holder identifiers (e.g., "U001") of each of the original multiple videos used in the combined video, and performs reward processing to give rewards to the rights holders identified by each of the multiple rights holder identifiers.

[0711] Moreover, the rights holder processing unit 136 performs the various types of protection processing described above on the constructed combined moving image.

[0712] As described above, according to this specific example, a combined moving image can be constructed by combining a plurality of moving images previously taken along a route traveled by a user using route information of the route, and the combined moving image can be output.

[0713] As described above, according to this embodiment, a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 are used to construct and provide a useful video.

[0714] Furthermore, according to this embodiment, a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be combined in terms of time to configure and provide a single useful video.

[0715] Furthermore, according to this embodiment, a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be spatially merged to form a single useful video, which can be provided.

[0716] Furthermore, according to this embodiment, it is possible to select an appropriate moving image from among the moving images captured by the mobile terminal 2 and the moving images captured by the fixed terminal 3, and to configure and provide one useful moving image.

[0717] Furthermore, according to this embodiment, a useful stereoscopic three-dimensional video can be constructed and provided by using the video captured by the mobile terminal 2 and the video captured by the fixed terminal 3.

[0718] Furthermore, according to this embodiment, a moving video and a fixed video based on the intention of the user of the mobile terminal 2 can be used to configure and provide a useful single video.

[0719] According to this embodiment, the camera used to capture the video may be an omnidirectional camera or the like, and the type of camera is not important.

[0720] The processing in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to the other embodiments in this specification. The software that realizes information system A in this embodiment is a program such as the following. In other words, this program causes a computer to execute a video acquisition step of acquiring a mobile video, which is a video taken by a mobile terminal and corresponds to an attribute value set including one or more environmental information including location information identifying the shooting location or time information identifying the time of shooting, and transmitted from the mobile terminal, and acquiring a fixed video, which is a video taken by a fixed camera whose shooting position is fixed and corresponds to an attribute value set including one or more environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera, a video composition step of temporally combining the mobile video and the fixed video to compose a single video, or compose a single video having two or more frames by spatially fusing at least a portion of the frames of the mobile video and at least a portion of the frames of the fixed video, and a video transmission step of transmitting the single video composed in the video composition step.

[0721] Also, Fig. 29 shows the appearance of a computer that executes the programs described in this specification to realize the information processing device 1, user terminal 4, target terminal 5, navigation terminal 6, etc. of the various embodiments described above. The above-mentioned embodiments can be realized by computer hardware and computer programs executed thereon. Fig. 29 is an overview of this computer system 300, and Fig. 30 is a block diagram of the system 300. The mobile terminal 2 and fixed terminal 3 are equipped with cameras.

[0722] In FIG. 29, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0723] 30, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0724] A program for causing computer system 300 to execute the functions of information processing device 1 and the like of the above-mentioned embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and further transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 at the time of execution. The program may be loaded directly from CD-ROM 3101 or the network.

[0725] The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the information processing device 1 of the above-mentioned embodiment. The program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to obtain a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0726] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0727] The program may be executed by a single computer or a plurality of computers. That is, the program may be executed by a centralized processing or a distributed processing.

[0728] Furthermore, in each of the above embodiments, it goes without saying that a plurality of communication means present in one device may be realized physically by one medium.

[0729] In each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0730] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0731] As described above, the information processing device 1 of the present invention has the effect of being able to use videos taken by a mobile terminal and videos taken by a fixed camera to construct and provide a useful video, and is useful as a server that provides videos, etc.

Claims

1. an acquisition unit that acquires mobile images, which are images captured by a mobile terminal, corresponding to an attribute value set including one or more pieces of environmental information including location information specifying a location of the image capture or time information specifying a time of the image capture, and transmitted from the mobile terminal, and acquires fixed images, which are images captured by a fixed camera having a fixed location of the image capture, corresponding to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera; a configuration unit that configures a single image by combining the moving image and the fixed image in time, or that configures a single image having a plurality of frames by spatially fusing at least a portion of a frame of the moving image and at least a portion of a frame of the fixed image; a transmission unit that transmits the one image constructed by the construction unit, The acquisition unit is An information processing device acquires an image corresponding to a priority type indicating a priority type of image among the fixed images and the moving images, when the fixed images and the moving images correspond to position information indicating positions close enough to each other to satisfy the first position condition, and the image corresponds to a priority type of image among the fixed images and the moving images, and the image corresponds to a priority type of image among the fixed images and the moving images, and the image corresponds to a priority type of image among the fixed images and the moving images.

2. The present invention further includes a receiving unit for receiving the moving image from the mobile terminal and for receiving the fixed image from a fixed terminal equipped with the fixed camera, The acquisition unit is acquiring the moving image and the fixed image received by the receiving unit; The receiving unit is 2. The information processing apparatus according to claim 1, further comprising: a function for receiving the moved image from the mobile terminal that has accepted a user's permission in accordance with a usage mode flag for the image stored in the mobile terminal.

3. The usage state flag is information indicating whether or not the mobile image transmitted from the mobile terminal has a "non-provisional use request," The receiving unit is An information processing device as described in claim 2, wherein when the usage mode flag stored in the mobile terminal is information indicating "there is a desire for non-temporary use", the mobile image is received from the mobile terminal only if the mobile terminal accepts the user's permission.

4. The moving image or the fixed image is associated with a rights holder identifier that identifies a rights holder of the moving image or the fixed image, 2. The information processing apparatus according to claim 1, further comprising a right holder processing unit for performing right holder processing relating to a right holder identified by a right holder identifier corresponding to the one image constructed by the construction unit.

5. An information processing method realized by an acquisition unit, a configuration unit, and a transmission unit, comprising: an acquisition step in which the acquisition unit acquires mobile images, which are images taken by a mobile terminal, corresponding to an attribute value set including one or more pieces of environmental information including location information specifying a location of the image or time information specifying a time of the image, and transmitted from the mobile terminal, and acquires fixed images, which are images taken by a fixed camera having a fixed shooting location, corresponding to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera; a constructing step in which the constructing unit constructs one image by combining the moving image and the fixed image in time, or constructs one image having a plurality of frames by spatially fusing at least a portion of a frame of the moving image and at least a portion of a frame of the fixed image; a transmission step in which the transmission unit transmits the one image constructed in the image construction step, The obtaining step includes: An information processing method in which, when there are multiple images that satisfy adoption conditions, which are conditions for the original images that constitute the one image, and when there are fixed images and moving images whose position information corresponds to position information indicating positions close enough to satisfy a first position condition, an image corresponding to a priority type indicating a priority type of image among the fixed images and the moving images is obtained.

Images