Information processing device and information processing method

Abstract

[Problem] Conventionally, it has been impossible to use a moving image captured by a mobile terminal and a moving image captured by a fixed camera to form one useful moving image. [Solution] It is possible to use a moving image captured by a mobile terminal and a moving image captured by a fixed camera to form and provide one useful moving image by using an information processing device 1 comprising: a moving image acquisition unit 133 that acquires a mobile moving image which is a moving image captured by a mobile terminal 2, and that acquires a fixed moving image which is a moving image captured by a fixed camera; a moving image formation unit 134 that forms one moving image by temporally connecting the mobile moving image and the fixed moving image, or that forms one moving image which includes a plurality of frames obtained by spatially fusing at least some of the frames of the mobile moving image and at least some of the frames of the fixed moving image; and a moving image transmission unit 141 that transmits the one moving image formed by the moving image formation unit 134.

Description

Information processing device and information processing method

[0001] The present invention relates to an information processing device or the like that processes video captured by a mobile terminal and video captured by a fixed camera.

[0002] Conventionally, there have been image display systems that allow a user to view scenes other than the user's own field of view (scenes seen from moving objects other than the user's own field of view). In such image display systems, multiple videos captured and transmitted by multiple moving objects can be output on a single screen (see Patent Document 1).

[0003] JP 2008-154192 A (paragraph

[0140] , Figure 24, etc.)

[0004] However, conventional technology only allows users to view videos taken by multiple moving objects on a single screen, and it is not possible to create and provide a single useful video by using videos taken by mobile terminals and videos taken by fixed cameras.

[0005] The information processing device of the first invention is an information processing device comprising: a video acquisition unit that acquires mobile videos, which are videos taken by a mobile terminal and correspond to an attribute value set including one or more pieces of environmental information including location information that identifies the shooting location or time information that identifies the time of shooting, and transmitted from a mobile terminal; and fixed videos, which are videos taken by a fixed camera and correspond to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with a fixed camera whose shooting location is fixed; a video composition unit that temporally combines the mobile videos and the fixed videos to compose a single video, or composes a single video having multiple frames that spatially blend at least some of the frames of the mobile video and at least some of the frames of the fixed video; and a video transmission unit that transmits the single video composed by the video composition unit.

[0006] With this configuration, a useful video can be created by using videos captured by a mobile terminal and videos captured by a fixed camera.

[0007] In addition, the information processing device of the second invention is an information processing device in which, compared to the first invention, the video acquisition unit is an information processing device that acquires a video corresponding to a priority type indicating a priority type of video among the fixed video and the moving video when there are fixed video and moving video corresponding to location information indicating positions close enough to satisfy the first location condition, and the video acquisition unit is a plurality of videos that satisfy the adoption conditions, which are the conditions for the original videos that constitute one video.

[0008] With this configuration, it is possible to select appropriate videos from videos taken by a mobile terminal and videos taken by a fixed camera, and compose a useful video.

[0009] Furthermore, the information processing device of the third invention is an information processing device that, compared to the first or second invention, further includes a receiving unit that receives moving videos from a mobile terminal and fixed videos from a fixed terminal equipped with a fixed camera, and the video acquisition unit acquires the moving videos and fixed videos received by the receiving unit, and the receiving unit receives the moving videos from a mobile terminal that has accepted the user's permission in accordance with a usage mode flag for the videos stored in the mobile terminal.

[0010] With this configuration, a useful single video can be constructed by using moving video and fixed video based on the intention of the user of the mobile terminal.

[0011] Furthermore, the information processing device of the fourth invention differs from the third invention in that the usage mode flag is information indicating whether or not there is a "non-provisional desire for permanent use" for the moving video sent from the mobile terminal, and the receiving unit is an information processing device that receives the moving video from the mobile terminal only if the mobile terminal accepts the user's permission when the usage mode flag stored in the mobile terminal is information indicating "there is a non-provisional desire for permanent use."

[0012] With this configuration, a useful single video can be constructed by using moving video and fixed video based on the intention of the user of the mobile terminal.

[0013] In addition, the information processing device of the fifth invention is an information processing device in which, compared to any one of the first to fourth inventions, the moving video or fixed video is associated with a rights holder identifier that identifies the rights holder of the moving video or fixed video, and further includes a rights holder processing unit that performs rights holder processing related to the rights holder identified by the rights holder identifier that corresponds to a video composed by the video composition unit.

[0014] This configuration allows appropriate processing regarding the rights holder of the video.

[0015] In addition, the information processing device of the sixth invention is an information processing device in which, compared to the fifth invention, the rights holder identifier corresponding to one video is a rights holder identifier corresponding to each of multiple videos that were the basis for the one video, and the rights holder processing unit is an information processing device that is equipped with a reward means that performs reward processing to award rewards to each rights holder identified by the rights holder identifier corresponding to each of the multiple videos that were the basis for the one video.

[0016] With this configuration, it is possible to give a reward to the rights holder of the video captured by the mobile terminal.

[0017] In addition, the information processing device of the seventh invention is an information processing device in which, compared to the sixth invention, the reward means acquires video attribute values ​​corresponding to each of multiple videos that were the basis for a single video transmitted by the video transmission unit, and performs reward processing by using the video attribute values ​​to determine rewards for each of multiple rights holders and award the rewards.

[0018] With this configuration, appropriate compensation can be given to the rights holder of the video captured by the mobile terminal.

[0019] In addition, the information processing device of the eighth invention is an information processing device in which, compared to the fifth invention, the rights holder processing unit is equipped with a first security means that performs a first security processing in which a single video composed by a video composition unit is stored in association with a set of attribute values ​​corresponding to each of a plurality of videos that formed the single video.

[0020] With this configuration, it is possible to preserve a single video made up of multiple images.

[0021] In addition, the information processing device of the ninth invention is an information processing device in which, compared to the fifth invention, the rights holder processing unit is equipped with a second security means that performs a second security process to store a single video composed by the video composition unit in association with a rights holder identifier corresponding to each of the multiple videos that formed the single video.

[0022] With this configuration, the rights holder of a video made up of multiple images can be determined to be the appropriate rights holder.

[0023] Furthermore, the information processing device of the tenth invention is an information processing device that, compared to the fifth invention, further includes an inquiry receiving unit that receives from a user terminal an inquiry regarding environmental information relating to the environment in which the mobile terminal shot the video, the video acquisition unit acquires multiple videos corresponding to the inquiry received by the inquiry receiving unit, and the rights holder processing unit includes a third security means that stores a single video composed by the video composition unit in association with a rights holder identifier that identifies the user of the user terminal.

[0024] With this configuration, the rights holder of one video can be identified from multiple images as the rights holder who needs that video.

[0025] In addition, the information processing device of the eleventh invention is an information processing device in which, compared to the eighth invention, the rights holder processing unit is equipped with a fourth security means that performs a fourth security process that accumulates security information including access information for accessing a video in a blockchain.

[0026] With this configuration, it is possible to preserve management information for videos that need to be preserved.

[0027] Another information processing device of the present invention is an information processing device in which, compared to the first or second invention, the video acquisition unit sequentially acquires multiple videos that meet one or more of the following conditions: a location condition related to location information associated with the video; and a time condition related to time information associated with the video.

[0028] With this configuration, videos captured by mobile terminals and videos captured by fixed cameras can be appropriately used to create a single useful video.

[0029] According to the present invention, a video captured by a mobile terminal and a video captured by a fixed camera can be used to create and provide a useful video.

[0030] FIG. 1 is a conceptual diagram of an information system A in the first embodiment; FIG. 2 is a block diagram of the information system A; FIG. 3 is a block diagram of the information processing device 1; FIG. 4 is a block diagram of the fixed terminal 3; FIG. 5 is a flowchart illustrating an example of the operation of the information processing device 1; FIG. 6 is a flowchart illustrating an example of the attribute value set acquisition process; FIG. 7 is a flowchart illustrating an example of the mobile object attribute value tag acquisition process; FIG. 8 is a flowchart illustrating an example of the video tag acquisition process; FIG. 9 is a flowchart illustrating an example of the fourth maintenance process; FIG. 10 is a flowchart illustrating an example of the video fusion process;

[0031] Hereinafter, embodiments of an information processing device and the like will be described with reference to the drawings. Note that components with the same reference numerals in the embodiments perform similar operations, and therefore repeated description may be omitted.

[0032] (First embodiment) <Outline of first embodiment>

[0033] In this embodiment, an information processing device is described that uses videos captured by one or more fixed cameras and videos captured by cameras possessed by one or more mobile terminals to create a single video that is temporally combined or spatially integrated. The query for selecting multiple videos may be, for example, a query using location information specified by a user terminal, a query using one or more location information received from a target terminal to be monitored, a query using a destination set in a navigation terminal, or a query using route information. The conditions for selecting the original videos that compose a single video include, for example, conditions related to one or more of the location information and time information.

[0034] Note that a single video joined in time is a single video joined together from multiple original videos. When multiple original videos that form a single video joined in time are shot, it is preferable that they are consecutive in time, but they may also be separated in time. For example, the date on which one original video was shot may be August 20th, and the date on which another original video was shot may be October 21st.

[0035] A spatially fused video is a video that is constructed by constructing a frame using some or all of the frames that make up each of a plurality of original videos, and then temporally connecting the frames that are constructed. Note that at least one of the frames that make up the video contains some or all of the frames of each of the plurality of original videos.

[0036] In addition, in this embodiment, a priority type of video is determined between fixed videos taken by a fixed camera and mobile videos taken by a camera of a mobile terminal, and an information processing device is described that preferentially adopts the priority type of video.

[0037] In this embodiment, an information processing device is described that performs post-processing to convert a moving image into a stereoscopic 3D moving image and provides the stereoscopic 3D moving image. Note that the moving image to be post-processed may be only a portion of the moving image that satisfies the processing conditions.

[0038] In this embodiment, a case will be described in which the usage method of a video differs according to a usage flag set by the rights holder of the video that is the source of the video. The usage flag is, for example, a flag indicating whether or not there is a "non-provisional request for permanent use." For example, if there is a "non-provisional request for permanent use," a process of inquiring with the rights holder is performed before transmitting the video.

[0039] In this embodiment, an information processing device that performs right holder processing, which is processing related to the right holder of a video to be output, will be described. The right holder processing includes, for example, a compensation processing (to be described later) and various security processing (to be described later).

[0040] In this embodiment, an information processing device that automatically receives and stores videos that meet the preservation conditions from a mobile terminal will be described.

[0041] In this embodiment, an information processing device that handles videos to which one or more tags obtained by analyzing the videos, etc. are added will be described. The tagging process is performed, for example, by the information processing device, a mobile terminal, or a fixed terminal, but may also be performed by other devices.

[0042] In this embodiment, an information processing device will be described that performs processing for a user of a mobile terminal or a user who needs a video when a video desired by the user exists but cannot be transmitted. Note that a case where a video cannot be transmitted is, for example, when the power of the mobile terminal is turned off.

[0043] In this embodiment, an information processing device that receives and utilizes movement information related to the start of movement from a mobile terminal will be described.

[0044] In this embodiment, a mobile terminal that transmits movement information to an information processing device will be described.

[0045] In this embodiment, a mobile terminal that transmits the latest set of attribute values ​​for the captured video to the information processing device when the mobile terminal finishes moving will be described.

[0046] Furthermore, in this embodiment, a mobile terminal and a fixed terminal will be described that acquire one or more tags by analyzing a video and associates the tags with the video.

[0047] In this embodiment, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc. <Outline of Information System A>

[0048] 1 is a conceptual diagram of an information system A according to this embodiment. The information system A includes an information processing device 1, one or more mobile terminals 2, one or more fixed terminals 3, one or more user terminals 4, one or more target terminals 5, and one or more navigation terminals 6.

[0049] The information processing device 1 is a server that provides a video to a user terminal 4 using videos transmitted from one or more mobile terminals 2 and one or more fixed terminals 3. The information processing device 1 is, for example, a cloud server or an ASP server, but the type is not important. The information processing device 1 may also be a device that constitutes a blockchain.

[0050] The video here refers to images captured by the mobile terminal 2 or the fixed terminal 3. The time interval between captures of the multiple still images contained in the video does not matter. The video may be, for example, a video of 60 frames per second or a video of 30 frames per second, but it may also be a collection of multiple still images captured at intervals of a predetermined time (for example, one minute) or more, or a collection of multiple still images captured when predetermined conditions are met.

[0051] The mobile terminal 2 is a terminal installed on a moving object and is a terminal that captures video. The mobile terminal 2 is, for example, a drive recorder, a smartphone, a tablet terminal, or a camera with a communication function. Furthermore, "installed" usually means "fixed," but it may also be considered to include "contacting" or "holding." The mobile terminal 2 may also have a driving means such as an engine or a moving means such as wheels.

[0052] A mobile body is an object that moves, such as a terrestrial mobile device, a marine mobile device, an undersea mobile device, an aerial mobile device, a space mobile device, or a living thing.

[0053] Examples of ground vehicles include automobiles, bicycles, rickshaws, and other human-powered vehicles such as playgrounds, trains, steam locomotives, and other railway vehicles, amusement park rides, and commercial vehicles used in factories and other facilities. Ground vehicles may also be vehicles that are not ridden by people, such as various robots for commercial or recreational use, radio-controlled toys, and other play equipment. Examples of motor vehicles include passenger cars, trucks, buses, taxis, and motorcycles.

[0054] Examples of marine vehicles include various types of ships, jet skis, surfboards, rowboats, swim rings, rafts, and the like.

[0055] Examples of underwater mobile devices include submarines, diving robots, and diving equipment such as aqualungs.

[0056] Examples of aerial vehicles include various aircraft, helicopters, gliders, parachutes, balloons, kites, and the like.

[0057] Space vehicles include, for example, rockets, space probes, and artificial satellites.

[0058] Examples of living things include humans and non-human moving bodies. Examples of non-human moving bodies include birds, mammals, reptiles, amphibians, fish, insects, and other living things.

[0059] The fixed terminal 3 is a terminal that is installed at a fixed location and has a photographing function. The fixed terminal 3 has a photographing means and a communication means. The photographing means is, for example, a so-called surveillance camera. The fixed terminal 3 has, for example, a surveillance camera installed in a city. The fixed terminal 3 has, for example, a surveillance camera installed in a building such as a private home or a building.

[0060] The user terminal 4 is a terminal used by a user. A user is someone who watches videos or needs videos. The user terminal 4 may have the functions of the mobile terminal 2. In other words, the user terminal 4 may be a terminal of a user who provides videos. The user terminal 4 may also be a navigation terminal 6, which will be described later.

[0061] The target terminal 5 is a terminal that transmits location information. The target terminal 5 is, for example, a terminal provided at the monitoring target. The monitoring target is, for example, a living thing or an object. The living thing is, for example, a person, a pet, or other animal. The person is, for example, a child or an elderly person. The object to be monitored is, for example, an object that you do not want stolen, such as a car, a motorcycle, or a bicycle. However, the monitoring target is not limited.

[0062] The navigation terminal 6 is a so-called navigation terminal that provides the user with information about the user's current location and guidance to the destination to which the user is heading. The navigation terminal 6 is a terminal that is installed in a ground vehicle or held by a person.

[0063] The information processing device 1 and one or more mobile terminals 2 are typically capable of communicating via a network such as the Internet. The information processing device 1 and one or more fixed terminals 3 are typically capable of communicating via a network such as the Internet. The information processing device 1 and one or more user terminals 4 are typically capable of communicating via a network such as the Internet. The information processing device 1 and one or more target terminals 5 are typically capable of communicating via a network such as the Internet. Furthermore, the information processing device 1 and one or more navigation terminals 6 are typically capable of communicating via a network such as the Internet. The user terminal 4 and the target terminal 5, and the user terminal 4 and the navigation terminal 6 are typically capable of communicating via a network such as the Internet.

[0064] Fig. 2 is a block diagram of an information system A according to this embodiment. Fig. 3 is a block diagram of an information processing device 1. Fig. 4 is a block diagram of a fixed terminal 3. <Configuration examples of each device>

[0065] The information processing device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit 14. The storage unit 11 includes a mobile terminal management unit 111 and a fixed terminal management unit 112. The receiving unit 12 includes a movement information receiving unit 121, an inquiry receiving unit 122, and a collection receiving unit 123. The processing unit 13 includes a movement information accumulation unit 131, a collection accumulation unit 132, a video acquisition unit 133, a video composition unit 134, a video editing unit 135, and a rights holder processing unit 136. The video acquisition unit 133 includes a movement video acquisition means 1331 and a fixed video acquisition means 1332. The rights holder processing unit 136 includes a first security means 1361, a second security means 1362, a third security means 1363, a fourth security means 1364, and a reward means 1365. The transmitting unit 14 includes a video transmission unit 141, a status transmission unit 142, and a needs transmission unit 143.

[0066] The mobile terminal 2 includes a moving storage unit 21, a moving receiving unit 22, a moving processing unit 23, and a moving transmitting unit 24. The moving processing unit 23 includes a photographing unit 231, a tag acquiring unit 232, and a moving information acquiring unit 233. The moving transmitting unit 24 includes a moving information transmitting unit 241, a moving video transmitting unit 242, and a collective transmitting unit 243.

[0067] The fixed terminal 3 includes a fixed storage unit 31, a fixed processing unit 33, a fixed receiving unit 32, and a fixed transmitting unit 34. The fixed processing unit 33 includes a fixed camera 331.

[0068] The user terminal 4 includes a user storage unit 41, a user reception unit 42, a user processing unit 43, a user transmission unit 44, a user reception unit 45, and a user output unit 46. <Details of Components of Information Processing Device 1>

[0069] Various types of information are stored in the storage unit 11 constituting the information processing device 1. The various types of information include, for example, mobile terminal information (to be described later), fixed terminal information (to be described later), attribute value sets (to be described later), movement information (to be described later), video, and management information. The attribute value sets are movement attribute value sets or fixed attribute value sets.

[0070] The management information is information used to monitor the person being watched over. The management information includes a user identifier for transmitting video to the user terminal 4 of the person watching over the person being watched over, and a target person identifier for identifying the target terminal 5 of the person being watched over.

[0071] The mobile terminal management unit 111 stores one or more pieces of mobile terminal information. The mobile terminal information is information relating to the mobile terminal 2. The mobile terminal information includes information specifying the video currently held by the mobile terminal 2. The mobile terminal information includes a mobile terminal identifier and a set of mobility attribute values. The mobile terminal information may be associated with the video. Furthermore, the mobile terminal identifier may be included in the set of mobility attribute values.

[0072] The mobile terminal identifier is information that identifies the mobile terminal 2. The mobile terminal identifier may be a rights holder identifier that identifies the rights holder who is the user of the mobile terminal 2. The mobile terminal identifier is, for example, the ID of the mobile terminal 2, the user identifier of the user of the mobile terminal 2, the name of the mobile terminal 2, the IP address of the mobile terminal 2, or the MAC address of the mobile terminal 2.

[0073] A rights holder is someone who has some kind of right related to a video. A rights holder is, for example, the owner of the video, the copyright holder of the video, the owner of the mobile terminal 2 that shot the video, or the owner of the fixed terminal 3 that shot the video. A rights holder is the initial rights holder of the video. A rights holder is usually the owner of the mobile terminal 2 or the owner of the fixed terminal 3, but it can be anyone who has the rights to the video shot by the mobile terminal 2 or the video shot by the owner of the fixed terminal 3.

[0074] The rights holder identifier is an identifier of the rights holder of the video. The rights holder identifier may be a terminal identifier. The rights holder identifier is, for example, the rights holder's ID, name, email address, or telephone number. The rights holder ID is, for example, a user identifier.

[0075] A moving attribute value set is a set of one or more moving video attribute values. A moving video attribute value is an attribute value of a moving video. A moving video attribute value is, for example, environmental information. A moving video attribute value is, for example, a tag. A moving video attribute value is usually a dynamic attribute value that can change dynamically, but it may also be a static attribute value that does not change dynamically.

[0076] Environmental information is information about the environment in which a video was shot. Examples of environmental information include location information, direction information, camera information, time information, weather information, temperature information, and seasonal information. Location information is information that identifies the shooting location. The shooting location is the location of the camera that shot the video. Examples of location information include (latitude, longitude) and (latitude, longitude, altitude). However, location information may also be an area identifier that identifies an area on a map, an address, a road identifier that identifies a road, or a lane identifier that identifies a lane on a road. Direction information is information that identifies the direction of shooting. Examples of direction information include the angle from true north. Camera information is information about the camera. Examples of camera information include the angle of view and resolution. Time information is information that identifies the time when the video was shot. The time when the video was shot may be a time around the time the video was shot, and accuracy does not need to be high. Examples of time information include the time, year / month / date and time, year / month / date and time minute, year / month / date / time minute / second, year / month / day, and month / day. In other words, the granularity of the time indicated by the time information does not matter. Weather information is information that specifies the weather at the location where the video was shot at the time it was shot. Examples of weather information include "sunny," "rain," "snow," and "cloudy." Temperature information is information that specifies the outside temperature at the location where the video was shot at the time it was shot. Examples of temperature information include "25 degrees" and "above 30 degrees." Season information is information that specifies the season at the location where the video was shot at the time it was shot. Examples of season information include "spring," "summer," "early summer," and "winter."

[0077] A tag is information that identifies the characteristics of a video. A tag is, for example, information resulting from analyzing a video. A tag is, for example, information resulting from analyzing one or more moving object attribute values. A tag is, for example, information resulting from analyzing multiple moving object attribute values ​​in time series.

[0078] A mobile object attribute value is an attribute value related to a mobile object. The mobile object attribute value is, for example, information about movement that can be acquired by a mobile object while it is moving. The mobile object attribute value is, for example, CAN (Controller Area Network) data, information indicating the use of airbags. The CAN data is, for example, speed, engine RPM, and brake status. The tags are, for example, "accident," "traffic jam," "dangerous driving," "speeding," and the names of objects shown in the video (for example, "person," "bear," or the name of a celebrity).

[0079] The moving video attribute values ​​of position information, direction information, time information, weather information, temperature information, and season information are dynamic attribute values, while camera information is a static attribute value.

[0080] The fixed terminal management unit 112 stores one or more pieces of fixed terminal information. The fixed terminal information is information relating to the fixed terminal 3. The fixed terminal information includes a fixed terminal identifier and a set of fixed attribute values. The fixed terminal information may be associated with a video. Furthermore, the fixed terminal identifier may be included in the set of fixed attribute values.

[0081] The fixed terminal identifier is information for identifying the fixed terminal 3. The fixed terminal identifier may be a rights holder identifier for identifying the rights holder who is the user of the fixed terminal 3. The fixed terminal identifier is, for example, the ID of the fixed terminal 3, the name of the fixed terminal 3, the IP address of the fixed terminal 3, or the MAC address of the fixed terminal 3.

[0082] A fixed attribute value set is a set of one or more fixed video attribute values. A fixed video attribute value is an attribute value of a fixed video. Fixed video attribute values ​​are, for example, environmental information and tags. Fixed video attribute values ​​are dynamic attribute values ​​that can change dynamically, or static attribute values ​​that do not change dynamically. Note that static attribute values ​​of the fixed terminal 3 are, for example, location information and camera information. Dynamic attribute values ​​of the fixed terminal 3 are, for example, time information, weather information, temperature information, and season information.

[0083] The receiving unit 12 receives various information and instructions from the mobile terminal 2, the fixed terminal 3, the user terminal 4, the target terminal 5, or the navigation terminal 6. The various information and instructions include, for example, movement information, location information, an inquiry, an attribute value set, and a video.

[0084] The receiving unit 12 receives moving images from the mobile terminal 2. Such moving images are called moving images. The receiving unit 12 receives moving images from the mobile terminal 2 for which permission from the user has been accepted, in accordance with a usage mode flag stored in the mobile terminal 2.

[0085] The usage status flag is information that specifies the stance of the rights holder of the moving video when a third party uses the moving video. The usage status flag is, for example, information indicating that "there is a desire for permanent use" or information indicating that "temporary use is acceptable." Note that permanent use is not temporary use. The usage status flag is, for example, information indicating that "permission from the rights holder is obtained for third parties to use the moving video" or information indicating that "third parties may freely use the moving video."

[0086] For example, when the usage mode flag stored in the mobile terminal 2 indicates that there is a desire for non-provisional use, the receiving unit 12 receives the moving video from the mobile terminal 2 only when the mobile terminal 2 accepts permission from the user.

[0087] The receiving unit 12 receives the video captured by the fixed camera 331 from the fixed terminal 3. Such video is called a fixed video.

[0088] It is preferable that a rights holder identifier that identifies the rights holder of the video is associated with the video received by the receiving unit 12. For example, one or more video attribute values ​​are associated with the video received by the receiving unit 12.

[0089] The video received by the receiving unit 12 may be, for example, a video of a parking lot. The video received by the receiving unit 12 may be, for example, a video showing a child who is being watched over. However, the location from which the video received by the receiving unit 12 is obtained does not matter.

[0090] The movement information receiving unit 121 receives movement information from the mobile terminal 2 when the mobile terminal 2 starts moving. The start time is preferably immediately after the start, but may also be after a predetermined time (for example, one minute) has elapsed since the start of the movement.

[0091] The movement information is information that specifies that the mobile terminal 2 is moving. The information that specifies that the mobile terminal 2 is moving may also be information that specifies that the mobile terminal has started moving. The movement here may mean that the mobile terminal is about to start moving, or that the mobile terminal is currently moving. The movement information is, for example, a movement start flag and a terminal identifier. The movement start flag is information that indicates that the mobile terminal has started moving. The start of movement means, for example, that the engine has been turned on or that filming has started. The terminal identifier is the identifier of the mobile terminal 2 that is starting to move. The terminal identifier may be the same as the rights holder identifier.

[0092] The inquiry receiving unit 122 receives an inquiry regarding environmental information. The inquiry receiving unit 122 typically receives the inquiry from the user terminal 4. The inquiry receiving unit 122 may also receive the inquiry from the target terminal 5 or the navigation terminal 6.

[0093] An inquiry regarding environmental information is an inquiry that includes environmental information as a condition. Here, the environmental information is information about the environment in which the mobile terminal 2 or the fixed terminal 3 captured a video. The inquiry is a request for a video captured by the mobile terminal 2 or the fixed terminal 3. The query is, for example, SQL, but the format, data structure, etc., of the query are not important.

[0094] The inquiry receiving unit 122 receives an inquiry including, for example, location information. The inquiry receiving unit 122 receives an inquiry including, for example, location information from the user terminal 4.

[0095] The inquiry receiving unit 122 sequentially receives, for example, each of the multiple pieces of location information transmitted from the target terminal 5. The inquiry receiving unit 122 may receive each of the multiple pieces of location information from the target terminal 5, or may receive the information from the user terminal 4 or the like that has received the information from the target terminal 5.

[0096] The inquiry receiving unit 122 receives an inquiry including, for example, location information that identifies a destination set in the navigation terminal 6. For example, the inquiry receiving unit 122 may receive the location information from the navigation terminal 6, may receive the location information from a user terminal 4 that has received the location information from the navigation terminal 6, or may receive the location information from a user terminal 4 that has sent the location information to the navigation terminal 6 for setting the destination.

[0097] The inquiry receiving unit 122 receives an inquiry including route information including a plurality of pieces of location information, for example. The inquiry receiving unit 122 receives an inquiry including route information from the user terminal 4 or the navigation terminal 6, for example.

[0098] The route information included in the query is, for example, information specifying the route traveled by the user watching the video, or route information set in the navigation terminal 6 or the user terminal 4 with a navigation function. It is preferable that the route information includes time information corresponding to each of the multiple pieces of position information. The time information is, for example, information specifying the time when the user was at the location specified by the position information. The distance between the multiple pieces of position information included in the route information does not matter. The time when the user was at the location specified by the position information is the time when the video was filmed at that location.

[0099] The set receiving unit 123 receives the movement attribute value set from the mobile terminal 2. It is preferable that the set receiving unit 123 receives the movement attribute value set from the mobile terminal 2 when the movement of the mobile terminal 2 ends. The movement attribute value set here is information that identifies a video stored in the mobile terminal 2. Note that the movement attribute value set received by the set receiving unit 123 is associated with an identifier of the mobile terminal 2 (e.g., a terminal identifier, a rights holder identifier).

[0100] The set receiving unit 123 receives, for example, a set of fixed attribute values ​​associated with the identifier of the fixed terminal 3 from the fixed terminal 3. The set receiving unit 123 receives, for example, dynamic attribute values ​​that can change dynamically from among the fixed video attribute values ​​in the set of fixed attribute values. The dynamic attribute values ​​here are, for example, time information, weather information, temperature information, seasonal information, and tags. It is preferable that the set receiving unit 123 does not receive location information or camera information from the fixed terminal 3. The location information and camera information of the fixed terminal 3 are static attribute values, and are preferably stored in advance in the fixed terminal management unit 112.

[0101] The processing unit 13 performs various types of processing. The various types of processing include, for example, processing performed by a movement information storage unit 131, a collection storage unit 132, a video acquisition unit 133, a video composition unit 134, a video processing unit 135, and a rights holder processing unit 136.

[0102] The movement information storage unit 131 stores the movement information received by the movement information receiving unit 121 in association with the mobile terminal 2. Association with the mobile terminal 2 means association with, for example, a right holder identifier or a terminal identifier.

[0103] The set storage unit 132 stores the mobile attribute value set received by the set receiving unit 123 in the mobile terminal management unit 111. The set storage unit 132 normally stores the attribute value set in association with the mobile terminal 2 that has transmitted the mobile attribute value set. Associating with the mobile terminal 2 means, for example, associating with the right holder identifier or terminal identifier of the mobile terminal 2.

[0104] The set storage unit 132 stores the fixed attribute value set received by the set receiving unit 123 in the fixed terminal management unit 112. The set storage unit 132 normally stores the attribute value set in association with the fixed terminal 3 that transmitted the fixed attribute value set. Associating with the fixed terminal 3 means, for example, associating with the right holder identifier or terminal identifier of the fixed terminal 3.

[0105] The video acquisition unit 133 acquires moving videos captured by one or more mobile terminals 2. The video acquisition unit 133 also acquires fixed videos captured by one or more fixed terminals 3. The video acquisition unit 133 normally acquires moving videos and fixed videos received by the receiving unit 12.

[0106] The moving video acquired by the video acquisition unit 133 is usually associated with a moving attribute value set. The moving attribute value set includes one or more pieces of environmental information. The one or more pieces of environmental information preferably include location information that identifies the shooting location or time information that identifies the shooting time. In addition, the moving video is preferably associated with a rights holder identifier or a terminal identifier of the mobile terminal 2.

[0107] The fixed moving images acquired by the moving image acquisition unit 133 are usually associated with a set of fixed attribute values. The set of fixed attribute values ​​includes one or more pieces of environmental information. The one or more pieces of environmental information preferably include location information that identifies the shooting location or time information that identifies the shooting time. In addition, the fixed moving images are preferably associated with a rights holder identifier or a terminal identifier of the fixed terminal 3.

[0108] For example, when there are multiple videos that satisfy the adoption conditions and include multiple videos that include fixed videos and moving videos, the video acquisition unit 133 acquires a video that corresponds to the priority type. For example, when there are multiple videos that satisfy the adoption conditions and the fixed videos and moving videos that correspond to location information that indicates positions that are close enough to satisfy the first location condition, the video acquisition unit 133 acquires a video that corresponds to the priority type from the fixed videos and moving videos.

[0109] For example, when there are multiple videos that satisfy the adoption conditions, the video acquisition unit 133 may determine the video to be finally adopted by using one or multiple video attribute values ​​of the multiple videos. For example, when there are multiple videos that satisfy the adoption conditions, the video acquisition unit 133 determines the video with the highest resolution.

[0110] The adoption conditions are conditions for the original videos that make up one video. It is preferable that the adoption conditions include one or more of a location condition or a time condition. The location condition is a condition related to the location information associated with the video. For example, the location condition is a video associated with location information whose distance from the location indicated by the location information included in the inquiry received by the inquiry receiving unit 122 is within a threshold or less than a threshold. The time condition is a condition related to the time information associated with the video. For example, the time condition is a video associated with time information indicating the time closest to the current time. The location condition is, for example, a first location condition or a second location condition. Note that one video is a video that is ultimately provided to the user, such as a combined video or a fused video.

[0111] The first location condition is a condition related to location information. The first location condition is that the location information is in an approximate relationship with the location specified by the target location information. The approximate relationship means proximity, for example, a distance within a threshold, a distance less than a threshold, a travel time within a threshold, or a travel time less than a threshold. The target location information is, for example, location information included in an inquiry, received location information, or location information included in route information included in an inquiry.

[0112] The first position condition is typically a condition that assumes that an image of a position identified by the target position information is captured. In other words, the first position condition may include conditions other than position information. The first position condition may be a condition related to position information and time information. For example, the first position condition is that the position information and time information satisfy an approximate condition for a pair of target position information and time information. The position information and time information that satisfy the approximate condition for a pair of target position information and time information are, for example, position information indicating a position whose distance is within a threshold, whose distance is less than a threshold, whose travel time is within a threshold, or whose travel time is less than a threshold, relative to the position identified by the target position information, and time information whose time difference is within a threshold or less than a threshold, relative to the time identified by the target time information. The first position condition may be a condition related to the position information and the traveling direction of a moving object. The first position condition may be a condition related to the position information and camera direction information.

[0113] The second position condition is position information that is in an approximate relationship with the reference position or reference area of ​​the comparison target. The approximate relationship here means that the distance from the reference position is within a threshold, the distance from the reference position is less than a threshold, or the position is within the reference area. The reference area is, for example, a specific area such as a parking lot. The comparison target is a location where videos are to be synthesized, for example, a specific location such as a parking lot.

[0114] The priority type is information that specifies which type of moving image has priority, a fixed moving image or a moving image. The priority type is, for example, either a "fixed moving image" or a "moving moving image." It is preferable that the priority type is a "fixed moving image."

[0115] The video acquisition unit 133 sequentially acquires a plurality of videos that match one or more of the position condition and the time condition, for example. The plurality of videos includes one or more moving videos and one or more fixed videos.

[0116] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to the inquiries received by the inquiry receiving unit 122 .

[0117] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to location information that satisfies a first location condition using the location information received by the inquiry receiving unit 122 .

[0118] For example, the video acquisition unit 133 sequentially uses each of the plurality of pieces of location information received by the inquiry receiving unit 122 to sequentially acquire a plurality of videos corresponding to the location information that satisfies the first location condition.

[0119] The video acquisition unit 133 sequentially acquires, for example, a plurality of videos corresponding to location information that satisfies the first location condition using the location information received by the inquiry receiving unit 122 .

[0120] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to position information that satisfies a plurality of first position conditions using each of a plurality of pieces of position information included in the route information.

[0121] The video acquisition unit 133 acquires, for example, a plurality of videos corresponding to position information that satisfies a first position condition using each of a plurality of pieces of position information and time information included in the route information.

[0122] The video acquisition unit 133 acquires a plurality of videos, for example, where the location information associated with each of the plurality of videos satisfies a second location condition and the time information associated with each of the plurality of videos satisfies a time condition. Note that the second location condition, for example, is that the location information is information indicating a position within a predetermined area (for example, a specific parking lot). The time condition, for example, is that the time information is within a threshold value (for example, within 30 seconds) of the current time or a time before the threshold value.

[0123] The multiple videos acquired by the video acquisition unit 133 typically include one or more moving videos and one or more fixed videos. However, the multiple videos acquired by the video acquisition unit 133 may all be moving videos or all be fixed videos.

[0124] It is preferable that the video acquisition unit 133 acquires a video that satisfies the storage condition. It is also preferable that the video acquired by the video acquisition unit 133 is a video that corresponds to an attribute value set and a rights holder identifier.

[0125] The storage conditions are conditions for storing videos. For example, the storage conditions are those that match the conditions specified by the inquiry. For example, the storage conditions are those that match the conservation conditions described below. The storage conditions may be the same as the adoption conditions.

[0126] The device storing the video to be acquired may be the mobile terminal 2, the information processing device 1, or another device not shown. The other device may be, for example, a device that constitutes a blockchain.

[0127] For example, when the inquiry receiving unit 122 receives an inquiry, the video acquiring unit 133 acquires one or more videos corresponding to the inquiry from the mobile terminal 2 or the fixed terminal 3. For example, the video acquiring unit 133 acquires, from the mobile terminal 2 or the fixed terminal 3, one or more videos that are paired with an attribute value set that satisfies a condition related to environmental information included in the inquiry.

[0128] The video acquisition unit 133, for example, interprets the query, detects a fixed terminal 3 that can provide a video that matches the query, and receives a video that is being shot by the fixed terminal 3 from the fixed terminal 3. For example, the video acquisition unit 133 acquires a terminal identifier that pairs with location information that matches a first location condition for location information included in the query from the fixed terminal management unit 112, transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier, and receives a video corresponding to the video transmission instruction from the fixed terminal 3. For example, the video acquisition unit 133 acquires a terminal identifier that pairs with location information and direction information that match a first location condition for location information and direction information included in the query from the fixed terminal management unit 112, transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier, and receives a video from the fixed terminal 3.

[0129] Note that a video transmission instruction is usually an instruction to transmit a video currently being shot, but it may also be an instruction to transmit a video previously shot by the fixed terminal 3 and stored in the fixed terminal 3 or another device not shown. The video transmission instruction may also be an inquiry. In other words, the inquiry may be an instruction to transmit a video currently being shot.

[0130] The video acquisition unit 133, for example, transmits a query to one or more mobile terminals 2 and receives one or more videos corresponding to the query from one or more mobile terminals 2. This process is called unregistered video search processing. The unregistered video search processing is processing for acquiring videos that match a condition from unregistered videos that are stored in the mobile terminal 2. The unregistered video search processing is processing for acquiring videos that match a query from unregistered videos that are stored in the mobile terminal 2, for example.

[0131] For example, if the video acquisition unit 133 interprets a query and determines that there is no fixed terminal 3 that can provide a video that matches the query, it is preferable that the video acquisition unit 133 transmits the query to one or more mobile terminals 2 and receives one or more videos corresponding to the query from one or more mobile terminals 2.

[0132] The video acquisition unit 133 acquires a video corresponding to the inquiry from among videos taken by one or more mobile terminals 2 corresponding to the movement information, for example. The video acquisition unit 133 transmits an inquiry to each of the one or more mobile terminals 2 corresponding to the movement information, and receives a video that matches the inquiry from the mobile terminal 2. Note that the mobile terminal 2 corresponding to the movement information is a mobile terminal 2 that is moving and is capable of transmitting a video.

[0133] For example, when the inquiry receiving unit 122 receives an inquiry, the video acquisition unit 133 determines one or more attribute value sets that match the inquiry, and acquires videos corresponding to each of the one or more attribute value sets from the storage unit 11.

[0134] The video acquisition unit 133 , for example, refers to the mobile terminal management unit 111 , determines one or more videos corresponding to the inquiry, and acquires the one or more videos from the mobile terminal 2 .

[0135] The process of determining a video corresponding to a query by referring to the mobile terminal management unit 111 is to determine an attribute value set that matches the query from one or more attribute value sets held by the mobile terminal management unit 111. This process is called a registered video search process. The registered video search process is a process of searching registered videos for videos that match the conditions. The registered video search process is, for example, a process of searching registered videos for videos that match the query. Note that a registered video is a video that has undergone the first maintenance process described below or the second maintenance process described below.

[0136] The video acquisition unit 133, for example, refers to the mobile terminal management unit 111, determines the video corresponding to the inquiry, determines whether the video is in a state where it can be transmitted from the mobile terminal 2, and acquires the video from the mobile terminal 2 only if it is in a state where it can be transmitted.

[0137] For example, the video acquisition unit 133 attempts to communicate with the mobile terminal 2, and if it can receive information from the mobile terminal 2, it determines that the mobile terminal 2 is in a state where it can transmit video. For example, the video acquisition unit 133 determines whether or not movement information corresponding to the mobile terminal 2 is stored in the storage unit 11, and if it is stored, it determines that the mobile terminal 2 is in a state where it can transmit video. Note that a state where it is possible to transmit video refers to, for example, a state where the power of the mobile terminal 2 or a mobile object corresponding to the mobile terminal 2 is ON (for example, a state where the engine of a car that is a mobile object is ON). Furthermore, the method by which the video acquisition unit 133 determines whether or not it is in a state where it can transmit video is not important.

[0138] The video acquisition unit 133 , for example, refers to the fixed terminal management unit 112 , determines one or more fixed videos corresponding to the inquiry, and acquires the fixed videos from the fixed terminal management unit 112 .

[0139] The video acquisition unit 133 acquires, for example, a video associated with a mobile attribute value set that satisfies the conservation condition from the mobile terminal 2. The video acquisition unit 133 acquires, for example, a video associated with a fixed attribute value set that satisfies the conservation condition from the fixed terminal 3. By this processing, the video can be acquired automatically.

[0140] The preservation condition is a condition for storing video. The preservation condition is a condition related to a set of attribute values. Preferably, the preservation condition is a condition related to one or more tags. Examples of the preservation condition include that the set of attribute values ​​includes a tag indicating "accident," that the set of attribute values ​​includes a tag indicating "traffic jam," and that the set of attribute values ​​includes a tag indicating a specific location. Examples of tags indicating a specific location include the name of a specific parking lot, a specific place name, the name of a specific scenic spot, and the name of a specific landscape.

[0141] The preservation condition is that the moving image corresponds to an attribute value set that satisfies a predetermined condition for an attribute value set corresponding to one or more moving images stored in a predetermined device or area (e.g., storage unit 11). The predetermined condition is, for example, that an attribute value contained in the attribute value set does not exist in the attribute value set corresponding to one or more moving images stored in the predetermined device or area.

[0142] The video acquisition unit 133 may acquire a processed image by, for example, processing a video captured by the mobile terminal 2 or the fixed terminal 3. Such a processed image is, for example, an around-view image. An around-view image is an image displayed on an around-view monitor. An around-view image is an image that appears to be captured from directly above an area including a moving object.

[0143] The moving video acquisition unit 1331 performs the process of acquiring a moving video among the processes performed by the video acquisition unit 133 .

[0144] The fixed video acquisition unit 1332 performs the process of acquiring a fixed video out of the processes performed by the video acquisition unit 133 .

[0145] The moving image composition unit 134 combines the moving image and the fixed image in terms of time or space to compose one moving image.

[0146] The video composition unit 134 composes one video using the multiple videos acquired by the video acquisition unit 133. Composing one video may involve sequentially passing parts of the videos that appear as one video to the user to the video transmission unit 141. The multiple videos include moving videos and fixed videos. Alternatively, one video may be a collection of multiple partial videos that are transmitted at intervals.

[0147] Further specific examples of the processing of the moving image composition unit 134 are described below. The moving image composition unit 134 performs (1) temporal combination of moving images, or (2) spatial combination of moving images, or both (1) and (2). (1) Temporal combination of moving images

[0148] The moving image composition unit 134, for example, temporally combines one or more moving images and one or more fixed images to compose one moving image.

[0149] The video composition unit 134, for example, temporally combines multiple videos that have different time information associated with each of the multiple videos acquired by the video acquisition unit 133 to compose a single video. Temporally combining multiple videos to compose a single video may involve passing the multiple videos in order to the video transmission unit 141. In other words, temporally combining multiple videos to compose a single video is sufficient as long as the resulting video appears to the user as a single video.

[0150] The video composition unit 134, for example, combines multiple videos acquired by the video acquisition unit 133 in the time order corresponding to the videos to compose one video. The video composition unit 134, for example, acquires parts of multiple videos acquired by the video acquisition unit 133 and combines the parts of the videos in order to compose one video. Combining multiple videos in time usually means combining parts of videos taken by the mobile terminal 2 or the fixed terminal 3 in order. Combining parts of videos in order may also mean passing parts of the videos in order to the video transmission unit 141. Furthermore, combining in the time order corresponding to the videos means combining the videos in the time order indicated by the time information corresponding to the videos, or combining the videos in the order in which the videos were received. (2) Spatial Video Combination

[0151] The moving image composition unit 134 composes one moving image, for example, by spatially fusing parts of each of the multiple moving images that have different position information associated with each of the multiple moving images acquired by the moving image acquisition unit 133. The moving image composition unit 134 composes one frame using some or all of the frames that compose each of the multiple moving images acquired by the moving image acquisition unit 133, and connects the multiple frames in time to compose one moving image.

[0152] The moving image composition unit 134 composes a single moving image having a plurality of frames by spatially fusing, for example, at least some of the frames of the moving moving image and at least some of the frames of the fixed moving image.

[0153] The moving image composition unit 134 composes one moving image, for example, by processing each of the multiple moving images acquired by the moving image acquisition unit 133. For example, the moving image composition unit 134 spatially connects a directly above frame obtained by combining frames of a moving image and a fixed image to compose one frame, and composes one moving image having one frame.

[0154] The moving image composition unit 134, for example, combines a plurality of around-view images to compose a wide-area around-view image. Note that "around-view" is a registered trademark.

[0155] The original moving images used by the moving image composer 134 to compose one moving image include moving moving images and fixed moving images.

[0156] The process of spatially fusing frames included in each of a plurality of moving images is, for example, the following processes (a) and (b): (a) A method based on image processing

[0157] The video composition unit 134, for example, performs a process of matching the direction and scale of each of the multiple frames that are the target of the spatial connection process. Next, the video composition unit 134, for example, detects identical areas in each of the multiple frames. Next, the video composition unit 134, for example, performs a process of overlapping multiple frames that have identical areas to compose one wide-area frame. Note that it is possible to detect identical areas in each of the multiple frames using publicly known technology. (b) Method based on machine learning

[0158] The video composition unit 134, for example, provides a plurality of frames and a learning model to a module that performs machine learning prediction processing, executes the module, and acquires one wide-area frame.

[0159] The learning model is obtained by providing multiple training data sets, each with multiple frames as explanatory variables and a single wide-area frame composed of the multiple frames as the objective variable, to a module that performs machine learning learning processing, and then executing the module.

[0160] The learning model may also be referred to as a learner, a classifier, a classification model, etc. Deep learning is preferable as a machine learning algorithm, but random forests, etc., are also acceptable. For machine learning, various machine learning functions, such as the TensorFlow library and the random forest module of the R language, and various existing libraries can be used.

[0161] The moving image processing unit 135 acquires a stereoscopic three-dimensional moving image from at least a part of the moving image composed by the moving image composition unit 134 .

[0162] A stereoscopic 3D video is a 3D video composed of illustrated frames. Note that the technology for acquiring a stereoscopic 3D video from camera images is well known, so a detailed explanation will be omitted (see the internet URL "https: / / xtech.nikkei.com / atcl / nxt / column / 18 / 01883 / 00004 / ").

[0163] The moving image processing unit 135 determines, for example, a partial moving image that satisfies the processing conditions from one moving image constructed by the moving image construction unit 134, and obtains a stereoscopic three-dimensional moving image from the partial moving image.

[0164] The processing conditions are conditions for identifying a partial video from which a stereoscopic 3D video is obtained. The processing conditions are, for example, conditions based on one or more video attribute values, or conditions based on tags obtained by analyzing the video. For example, the processing conditions are "1 minute before and after the frame corresponding to 'tag = accident'" and "weather = snow." Note that a partial video here is a video that makes up one video.

[0165] The video processing unit 135, for example, provides each frame of the video and a learning model to a machine learning prediction module, executes the prediction module, and acquires three-dimensional images that constitute the three-dimensional video. The learning model here is information acquired by providing a machine learning learning module with multiple pieces of training data, including captured still images and three-dimensional images of the still images, and executing the learning module. Deep learning is a suitable machine learning algorithm, but other algorithms such as random forests may also be used.

[0166] The video processing unit 135 creates a three-dimensional illustration for each frame of the video by performing known image processing, and creates a stereoscopic three-dimensional video.

[0167] The rights holder processing unit 136 performs rights holder processing.

[0168] The rights holder processing is processing related to the rights of one video. The rights holder processing is, for example, processing related to a rights holder identified by a rights holder identifier associated with one video composed by the video composition unit 134. The rights holder processing is, for example, processing related to a rights holder identified by a rights holder identifier associated with a stereoscopic 3D video acquired by the video processing unit 135. The rights holder processing is, for example, a first security processing described below, a second security processing described below, a third security processing described below, a fourth security processing described below, and a compensation processing described below.

[0169] The right holder processing unit 136 performs right holder processing, which is processing related to a right holder identified by a right holder identifier associated with a video, in response to the video being transmitted by the video transmission unit 141. The video transmitted by the video transmission unit 141 is a video constructed by the video construction unit 134 or a stereoscopic three-dimensional video acquired by the video processing unit 135.

[0170] The rights holder identifier associated with one video is, for example, a rights holder identifier associated with each of the multiple videos that formed the one video, or an identifier of the user who requested the one video.

[0171] The rights holder processing unit 136 stores, for example, videos acquired by the video acquisition unit 133 in association with a rights holder identifier. It is preferable that the rights holder processing unit 136 stores only videos acquired by the video acquisition unit 133 that satisfy the storage conditions. It is also preferable that the rights holder processing unit 136 does not store videos that do not satisfy the storage conditions.

[0172] The rights holder identifier is the rights holder identifier of the rights holder of the original video, the rights holder identifier of the rights holder of one video, or the rights holder identifier of the rights holder of the stereoscopic 3D video.

[0173] For example, when the video acquisition unit 133 determines that the mobile terminal 2 holding the video corresponding to the inquiry is in a state where it cannot transmit the video, the rights holder processing unit 136 acquires status information regarding that state.

[0174] The status information is information indicating that a video cannot be sent. The status information is, for example, information indicating the status of the mobile terminal 2 that holds the video. Examples of the status information are, "The video exists, but it is not currently in a state where it can be sent to you." and "The video exists on Mr. / Ms. XX's mobile terminal, but it is not currently in a state where it can be sent to you." The status information is, for example, information indicating that the power of the mobile terminal 2 is OFF, or information indicating that the power of the mobile terminal 2 is ON.

[0175] The rights holder processing unit 136 acquires the needs information, for example, when the video acquisition unit 133 determines that the mobile terminal 2 holding the video corresponding to the inquiry is in a state where it cannot transmit the video.

[0176] Needs information is information indicating that there is a need for a video. Examples of needs information include "Your XXX video has been requested by another user" and "Your XXX video has been requested by another user for XX yen."

[0177] The first security means 1361 performs a first security process of storing one video constructed by the video construction unit 134 or a stereoscopic 3D video acquired by the video processing unit 135 in association with an attribute value set corresponding to each of the multiple videos that formed the video. The first security means 1361 may also perform a first security process of storing a video received from the mobile terminal 2 or the fixed terminal 3 in association with an attribute value set corresponding to the video. The attribute value set corresponding to the video is a moving attribute value set corresponding to a moving video or a fixed attribute value set corresponding to a fixed video.

[0178] The second security means 1362 performs a second security process in which a video constructed by the video construction unit 134 or a stereoscopic three-dimensional video acquired by the video processing unit 135 is stored in association with a rights holder identifier corresponding to each of the multiple videos that formed the video.

[0179] In addition, the first security means 1361 or the second security means 1362 may store a video constructed by the video construction unit 134 or a three-dimensional video acquired by the video processing unit 135 in association with a set of attribute values ​​corresponding to each of the multiple videos that formed the video, and a rights holder identifier corresponding to each of the multiple videos that formed the video.

[0180] The third security means 1363 stores a video created by the video composition unit 134 or a stereoscopic 3D video acquired by the video processing unit 135 in association with a rights holder identifier that identifies the user of the user terminal 4. Note that the user of the user terminal 4 here is the person who views the video or the stereoscopic 3D video. The user terminal 4 here is, for example, the terminal that sent the inquiry.

[0181] Furthermore, the destination for storing one video or a stereoscopic 3D video is, for example, the storage unit 11, but it may also be another device, another device constituting a blockchain, etc. Furthermore, the stored video is usually associated with a video identifier that identifies the video.

[0182] The fourth security means 1364 performs a fourth security process to store security information including access information for accessing one of the stored moving images or stereoscopic 3D moving images. The order of storing the moving images and the fourth security process for the security information corresponding to the moving images does not matter.

[0183] The fourth security means 1364 performs a fourth security process, for example, of storing security information including access information for accessing one video that has been composed and stored by the video composition unit 134 in the blockchain.

[0184] The fourth security means 1364 performs a fourth security process, for example, by storing security information including access information for accessing the stored stereoscopic 3D video acquired by the video processing unit 135 in the blockchain.

[0185] It is preferable that the fourth security means 1364 stores the security information in a blockchain. That is, it is preferable that the fourth security means 1364 stores the security information in a distributed ledger of the blockchain. It is preferable that the fourth security means 1364 registers the security information as an NFT (non-fungible token). It is preferable that the fourth security means 1364 registers the security information in a distributed file system in an IPFS (Inter Planetary File System) network.

[0186] The preservation information is information for ensuring the originality of a video. The preservation information can be said to be header information for the video. The preservation information is, for example, access information and one or more video attribute values. It is preferable that the preservation information has, for example, one or more rights holder identifiers. When the preservation information has multiple rights holder identifiers, the rights holder of the video may be shared, and the multiple rights holder identifiers may be rights holder history information. The rights holder history information is a collection of rights holder identifiers and is information that indicates the history of the rights holder. The fourth preservation process can ensure the originality of the preservation information of the registered video. By ensuring the originality of the preservation information, the originality of the video corresponding to the preservation information is also guaranteed. The access information is information for accessing the video. The access information is information that identifies the storage location of the video. The access information is, for example, a URL or a URI.

[0187] The security information preferably includes information (which may be called a flag) indicating whether the video can be provided to a third party. The flag may be, for example, information indicating that the video can be viewed by a third party, information indicating that the video can be sold, or information indicating that neither viewing nor selling is permitted.

[0188] The remuneration means 1365 performs remuneration processing for each right holder identified by a right holder identifier associated with each of the multiple videos that are the basis for one video composed by the video composition unit 134, for example.

[0189] The remuneration means 1365 performs remuneration processing for each right holder identified by a right holder identifier associated with each of the multiple videos that are the basis for the stereoscopic 3D video acquired by the video processing unit 135, for example.

[0190] The reward process is a process of providing a reward. The reward process is, for example, a process of increasing points managed in pairs with one or more rights holder identifiers associated with a video. The reward process is, for example, a process of depositing money to a rights holder identified by one or more rights holder identifiers associated with a video. The reward process is, for example, a process of transmitting a video or other content to a user terminal 4 of a rights holder identified by one or more rights holder identifiers associated with a video. The reward process is not limited to the type of merit provided to the rights holder identified by one or more rights holder identifiers associated with the video. The reward may be, for example, money, points, goods, content, or the like, and the type of merit is not limited.

[0191] It is preferable that the reward means 1365 acquires one or more video attribute values ​​corresponding to each of the multiple videos that formed the basis of the single video sent by the video sending unit 141, and performs reward processing, which is a process of determining rewards for each of the multiple rights holders using the one or more video attribute values ​​and awarding the rewards.

[0192] Here, the one or more moving image attribute values ​​are, for example, the amount of moving image data, the length of moving image, the number of frames of moving image, and the resolution of moving image.

[0193] The reward means 1365 preferably acquires a reward amount corresponding to a service identifier that identifies a service provided for a target video, and performs reward processing, which is processing for providing a reward corresponding to the reward amount. The service identifier may be, for example, "view" or "purchase." In such a case, for example, the storage unit 11 stores the reward amount corresponding to the service identifier, or information for determining the reward amount corresponding to the service identifier.

[0194] The reward means 1365 performs reward processing, which is a process of obtaining a reward amount using, for example, one or more pieces of information from one or more video attribute values ​​and one or more service identifiers, and providing a reward corresponding to the reward amount. In such a case, for example, an arithmetic expression or a correspondence table corresponding to each of a plurality of service identifiers is stored in the storage unit 11. The arithmetic expression is a formula for calculating the reward amount using one or more video attribute values ​​as parameters. The correspondence table is a table having multiple pieces of correspondence information for managing reward amounts corresponding to one or more video attribute values.

[0195] The remuneration means 1365 normally performs a process of charging a remuneration to a user who has enjoyed a service related to the target video.

[0196] The process of charging a reward is, for example, a process of charging the amount of reward obtained. The process of charging a reward is, for example, a process of charging the amount of reward obtained and the amount of profit obtained by the operator of the information processing device 1. The process of charging a reward is, for example, a process of deducting points corresponding to the user who has enjoyed the service, a settlement process using the user's credit card number, etc.

[0197] The transmitter 14 transmits various types of information or instructions to the mobile terminal 2, the fixed terminal 3, the user terminal 4, or the navigation terminal 6. The various types of information or instructions are, for example, videos, inquiries, status information, and needs information.

[0198] The video transmitting unit 141 transmits, for example, one video constructed by the video constructing unit 134. The video transmitting unit 141 transmits, for example, a stereoscopic three-dimensional video instead of or in addition to the one video.

[0199] The video sending unit 141 sends one video or a stereoscopic 3D video to the user terminal 4. It is preferable that the video sending unit 141 sends a video in response to receiving an inquiry. The video sending unit 141 sends a video to, for example, the user terminal 4 or the navigation terminal 6. Sending one video may mean that the video sending unit 141 sequentially sends parts of the one video that have been passed to it.

[0200] The video sending unit 141 sends, for example, the video acquired by the video acquiring unit 133 to the user terminal 4 .

[0201] When a video cannot be transmitted, the status transmission unit 142 transmits status information relating to the status to the user terminal 4. For example, the status transmission unit 142 transmits status information acquired by the rights holder processing unit 136 to the user terminal 4. Whether a video cannot be transmitted is determined by, for example, the video acquisition unit 133.

[0202] When it is determined that a video cannot be transmitted, the needs transmission unit 143 transmits needs information indicating that there is a need for the video to the user corresponding to the video. The needs transmission unit 143 transmits, for example, the needs information acquired by the rights holder processing unit 136 to the user corresponding to the video.

[0203] Sending to a user corresponding to a video means, for example, sending the needs information by email to the notification destination indicated in the notification destination information paired with the rights holder identifier corresponding to the video. Sending to a user corresponding to a video means, for example, sending the needs information by short message to the telephone number destination paired with the rights holder identifier corresponding to the video. Sending to a user corresponding to a video means, for example, sending the needs information to the mobile terminal 2 paired with the rights holder identifier corresponding to the video. The user corresponding to a video is usually the rights holder of the video. The notification destination information is, for example, an email address, telephone number, IP address, or ID. <Details of Components of Mobile Terminal 2>

[0204] Various types of information are stored in the movement storage unit 21 that constitutes the mobile terminal 2. The various types of information include, for example, a moving video, a moving attribute value set, a rights holder identifier, movement information indicating the start of movement, a pair of attribute value tag conditions and tags, a pair of video tag conditions and tags, one or more conservation conditions, one or more pieces of acquisition information, and a usage mode flag. The movement storage unit 21 typically stores one or more pairs of attribute value tag conditions and tags. The movement storage unit 21 typically stores one or more pairs of video tag conditions and tags.

[0205] For example, one or more moving image attribute values ​​constituting a moving attribute value set correspond to one or more still images (which may also be called fields or frames) constituting a moving moving image. The one or more moving image attribute values ​​may correspond to all still images, some still images, or multiple still images.

[0206] An attribute value tag condition is a condition for acquiring a tag based on one or more mobile object attribute values. The attribute value tag condition is a condition related to one or more mobile object attribute values. Examples of the attribute value tag condition include "the brake is applied suddenly," "the degree of speed deceleration (acceleration) within a unit time is equal to or less than a threshold (the brake is applied suddenly)," "the airbag is deployed," "the time at or below a first speed has elapsed for a second hour or more (there is a traffic jam)," and "the location information corresponds to a specific name." Examples of tags paired with an attribute value tag condition are "abnormal driving," "accident," "traffic jam," and "specific location name."

[0207] A video tag condition is a condition related to a video. The video tag condition is a condition based on the analysis results of a video. Examples of video tag conditions include "having a still image of multiple cars colliding (it is an accident)," "having a frame in which the distance between multiple cars is 0 (cars are colliding)," "having a period in which a first threshold or more of cars are present within a specified distance is greater than or equal to a second threshold (it is a traffic jam)," and "the cumulative change in the distance between the center of gravity of the preceding car and the lane per unit time is greater than or equal to a threshold (the preceding car is driving erratically)." Examples of tags that pair with a video tag condition are "accident," "traffic jam," and "abnormal driving."

[0208] A preservation condition is a condition for preserving a video. The preservation condition is a condition related to a set of attribute values. For example, such a preservation condition is "an airbag has been deployed (an accident has occurred)" or "a specific tag has been attached to the video (the specific tag is, for example, "accident" or "dangerous driving")." The preservation condition is associated with, for example, acquired information. The acquired information is information that identifies a video to be acquired. The acquired information is information that identifies a video to be acquired when the preservation condition is satisfied. For example, such a preservation condition is "(1) the number of cars on the screen is equal to or greater than a first threshold & (2) the vehicle's traveling speed is equal to or less than a second threshold & the duration of (1) and (2) is equal to or greater than a third threshold (traffic congestion continues)." For example, the acquired information is information indicating "a video from one minute before the preservation condition is met to the time after the preservation condition is met until the preservation condition is no longer met."

[0209] The mobile receiving unit 22 receives various types of information, such as inquiries, needs information, videos captured by other mobile terminals 2, and videos captured by the fixed terminal 3.

[0210] The movement processing unit 23 performs various processes. The various processes are, for example, processes performed by the photographing unit 231, the tag acquisition unit 232, and the movement information acquisition unit 233. The movement processing unit 23 converts the information received by the movement receiving unit 22 into a data structure to be output.

[0211] The movement processing unit 23 detects, for example, the start of movement. The detection of the start of movement is, for example, the detection of the power supply of the mobile terminal 2 being turned on or the detection of the engine of the mobile object being turned on.

[0212] The movement processing unit 23 detects, for example, the end of movement by detecting that the power supply of the mobile terminal 2 is turned off or that the engine of the mobile object is turned off.

[0213] The movement processing unit 23 determines, for example, an attribute value set that matches the query received by the movement receiving unit 22 , and acquires a video paired with the attribute value set from the movement storage unit 21 .

[0214] The movement processing unit 23 acquires, for example, a video from the start of movement to the end of movement. The movement processing unit 23 acquires, for example, the video from the start of movement to the end of movement, in association with the attribute value set acquired during movement.

[0215] The movement processing unit 23 acquires an attribute value set during shooting, for example. The movement processing unit 23 stores the acquired movement attribute value set in the movement storage unit 21. Furthermore, the movement processing unit 23 associates the acquired movement attribute value set with a moving image, for example. Associating with a moving image usually means associating with frames that make up the moving image. It is preferable that the attribute value set and the frames are synchronized in time.

[0216] The movement attribute value set is, for example, one or more pieces of environmental information, such as location information, time information, weather information, temperature information, and season information.

[0217] The movement processing unit 23 acquires location information, for example, during shooting. For example, the movement processing unit 23 having the function of a GPS receiver acquires location information. The movement processing unit 23 acquires location information, for example, constantly, at predetermined intervals, or when acquisition conditions are met. The acquisition conditions are conditions for acquiring information. Examples of acquisition conditions include when an accident is detected, when traffic congestion is detected, or when there is a change in weather information.

[0218] The movement processing unit 23 acquires time information from a clock (not shown), for example, during shooting. The movement processing unit 23 acquires time information, for example, constantly, at predetermined intervals, or when an acquisition condition is met.

[0219] For example, during shooting, the movement processing unit 23 acquires time information from a clock (not shown) and acquires season information corresponding to the time information.

[0220] The movement processing unit 23 acquires weather information, for example, during shooting. The movement processing unit 23 acquires weather information corresponding to location information, for example, from a server (not shown). The movement processing unit 23 acquires weather information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.

[0221] The movement processing unit 23 acquires temperature information, for example, during shooting. The movement processing unit 23 acquires temperature information corresponding to location information, for example, from a server (not shown). The movement processing unit 23 acquires temperature information, for example, from a temperature sensor installed on a moving object. The movement processing unit 23 acquires temperature information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.

[0222] The movement processing unit 23, for example, determines whether the acquired movement attribute value set matches the preservation condition. The movement processing unit 23, for example, determines whether the acquired time-series movement attribute value set matches the preservation condition. For example, if the preservation condition is met, the movement processing unit 23 acquires a video corresponding to the movement attribute value set. For example, if the preservation condition is met, the movement processing unit 23 acquires a video corresponding to the preservation condition. For example, if the preservation condition is met, the movement processing unit 23 acquires acquisition information paired with the preservation condition and acquires a video in accordance with the acquisition information.

[0223] It is preferable that the movement processing unit 23 is provided with, for example, a microphone, acquires sound information, and stores the sound information in association with the video acquired by the imaging unit 231. Such a function is, for example, a function of a drive recorder.

[0224] The movement processing unit 23, for example, acquires the usage mode flag of the movement storage unit 21 and determines whether the usage mode flag is information indicating that the user is to be inquired about whether or not to transmit the moving video before transmitting the moving video. For example, when the usage mode flag is a value indicating "non-provisional, permanent desire for use," the movement processing unit 23 outputs inquiry information that is information for inquiring about whether or not to transmit the moving video. Note that the inquiry information is, for example, screen information (e.g., a panel) for inquiring about whether or not to transmit the moving video, or audio information for inquiring about whether or not to transmit the moving video.

[0225] The image capturing unit 231 captures a video. The image capturing unit 231 starts capturing video, for example, after detecting the start of movement. It is also preferable that the image capturing unit 231 continues capturing video, for example, until detecting the end of movement.

[0226] It is preferable that the imaging unit 231 stores the captured video in the mobile storage unit 21. When the imaging unit 231 stores video in the mobile storage unit 21, which has a limited storage capacity, it is preferable that the imaging unit 231 overwrites new video in the area where old video is stored. In other words, it is preferable that the structure of the mobile storage unit 21 is a ring buffer structure.

[0227] The tag acquisition unit 232 acquires one or more tags for the video captured by the imaging unit 231 and associates them with the video.

[0228] The tag acquisition unit 232, for example, analyzes the video captured by the imaging unit 231 and acquires one or more tags for the video.

[0229] The tag acquisition unit 232 acquires one or more tags, for example, using one or more moving object attribute values ​​acquired during image capture by the image capture unit 231. The moving object attribute values ​​are, for example, CAN data.

[0230] The tag acquisition unit 232 may, for example, determine one or more still images that match the video tag condition and acquire a tag that pairs with the video tag condition. The tag acquisition unit 232 may also associate the tag with the one or more still images. Note that the still images are frames that make up the video.

[0231] For example, if the video tag condition is "the number of vehicles with a distance between them within a threshold is equal to or greater than a threshold and the vehicle's travel speed is equal to or less than a threshold," and the tag paired with the video tag condition is "traffic jam," the tag acquisition unit 232 analyzes frames of the video, recognizes multiple vehicles, and acquires the distances between each of the multiple vehicles. The tag acquisition unit 232 also acquires the number of vehicles with a distance between them within a threshold. The tag acquisition unit 232 also acquires the travel distance and frame rate of a single vehicle in multiple frames, and acquires the vehicle's speed. The tag acquisition unit 232 then determines whether the video tag condition is met using the number of vehicles with a distance between them within a threshold and the vehicle's speed. If the video tag condition is met, the tag acquisition unit 232 acquires the tag "traffic jam" that is paired with the video tag condition. The tag acquisition unit 232 may also associate the tag "traffic jam" with the analyzed frame.

[0232] The tag acquisition unit 232 may, for example, determine one or more moving object attribute values ​​that match the attribute value tag condition, and acquire a tag that is paired with the attribute value tag condition. The tag acquisition unit 232 may also associate the tag with a video that is paired with the one or more moving object attribute values.

[0233] For example, if the attribute value tag condition is "driving time at a speed of less than 30 km for 10 minutes or more and the proportion of times during that driving time that the speed is less than 30% is 80% or more," and the tag paired with the attribute value tag condition is "traffic jam," the tag acquisition unit 232 uses the speed history contained in the CAN data associated with each field of the video to detect CAN data that matches the attribute value tag condition, acquires the tag "traffic jam" that matches the attribute value tag condition, and associates the tag with the field associated with the CAN data. Note that the CAN data associated with each field of the video is CAN data acquired at the same time that the field was captured.

[0234] The movement information acquisition unit 233 detects movement of the mobile terminal 2 and acquires movement information, for example, at the start of the movement. The movement information acquisition unit 233 acquires movement information, which is, for example, a rights holder identifier of the movement storage unit 21. The movement information acquisition unit 233 acquires, for example, movement information of the movement storage unit 21. Note that the movement information is, for example, a rights holder identifier and information indicating that movement has started. Furthermore, detection of movement of the mobile terminal 2 is, for example, detection of the engine being turned on or detection of the power being turned on of the mobile terminal 2.

[0235] The movement transmitting unit 24 transmits various types of information to the information processing device 1. The various types of information are, for example, movement information, movement video, and a movement attribute value set.

[0236] The movement information transmitting unit 241 normally transmits the movement information acquired by the movement information acquiring unit 233 to the information processing device 1 at the start of movement.

[0237] The movement video sending unit 242 sends the movement video captured by the shooting unit 231 to the information processing device 1. The timing at which the movement video sending unit 242 sends the video does not matter. For example, after the movement processing unit 23 acquires a video paired with a movement attribute value set that matches the query received, the movement video sending unit 242 sends the video to the information processing device 1. For example, when the movement video sending unit 242 detects the end of movement, it sends the video that has been accumulated since the start of movement was detected to the information processing device 1. For example, when the movement video sending unit 242 determines that the preservation conditions are met, it sends the video corresponding to this determination to the information processing device 1.

[0238] It is preferable that the movement transmitting unit 24 does not transmit the sound information acquired by the movement processing unit 23 even if it transmits the movement video captured by the imaging unit 231 to the information processing device 1. The sound information may be, for example, audio information of the driver or passengers in the moving object, and transmitting such sound information to the information processing device 1 and providing it to the user terminal 4 or the like may threaten the privacy of the driver or passengers, and is therefore not appropriate.

[0239] When the movement processing unit 23 inquires of the user of the mobile terminal 2 whether or not to transmit the movement video based on the usage mode flag, the movement video transmitting unit 242 transmits the movement video only when information indicating that the movement video will be transmitted is received in response to the inquiry. Note that the information indicating that the movement video will be transmitted or that the movement video will not be transmitted is received by a movement receiving unit (not shown) that the mobile terminal 2 has.

[0240] The set sending unit 243 sends the set of movement attribute values ​​in the movement storage unit 21 to the information processing device 1. For example, when the movement of the mobile terminal 2 ends, the set sending unit 243 sends the set of movement attribute values ​​in the movement storage unit 21 to the information processing device 1. Here, the set of movement attribute values ​​in the movement storage unit 21 is a set of movement attribute values ​​for the video stored in the movement storage unit 21.

[0241] Note that, once the set transmitting unit 243 acquires the set of moved attribute values, it may immediately transmit the set of moved attribute values ​​to the information processing device 1. In other words, the set transmitting unit 243 may transmit the set of moved attribute values ​​at any time. <Details of Components of Fixed Terminal 3>

[0242] Various types of information are stored in the fixed storage unit 31 constituting the fixed terminal 3. The various types of information include, for example, a fixed video, a fixed attribute value set, a rights holder identifier, camera information, a set of attribute value tag conditions and tags, a set of video tag conditions and tags, one or more conservation conditions, and one or more pieces of acquired information. The fixed storage unit 31 typically stores one or more sets of video tag conditions and tags. It is preferable that the fixed attribute value set include location information of the fixed terminal 3.

[0243] For example, one or more moving image attribute values ​​constituting a fixed attribute value set correspond to one or more still images (which may also be called fields or frames) constituting a fixed moving image. The one or more moving image attribute values ​​may correspond to all of the still images, some of the still images, or multiple still images.

[0244] A video tag condition is a condition related to a video. A video tag condition is a condition based on the analysis results of a video. Examples of video tag conditions include "having a still image of multiple cars colliding (it is an accident)," "having a frame in which the distance between multiple cars is 0 (cars are colliding)," and "the cumulative change in the distance between the center of gravity of the car in front and the lane per unit time is greater than or equal to a threshold (the car in front is swerving)." Tags that pair with a video tag condition are, for example, "accident" and "abnormal driving."

[0245] The fixed receiving unit 32 receives various types of information, such as inquiries and various instructions.

[0246] The fixed processing unit 33 performs various types of processing. The various types of processing are, for example, processing performed by the fixed camera 331. The fixed processing unit 33 configures information to be transmitted in response to an inquiry received by the fixed receiving unit 32. The fixed processing unit 33 configures information to be transmitted by the fixed transmitting unit 34. The transmitted information includes a fixed video. It is preferable that the transmitted information includes a fixed attribute value set and a rights holder identifier.

[0247] The fixed processing unit 33 acquires a fixed attribute value set during shooting, for example. The fixed processing unit 33 stores the acquired fixed attribute value set in the fixed storage unit 31. Furthermore, the fixed processing unit 33 associates the acquired fixed attribute value set with a moving image, for example. Associating with a moving image usually means associating with frames that make up the moving image. It is preferable that the fixed attribute value set and the frames are synchronized in time.

[0248] The set of fixed attribute values ​​may be, for example, one or more pieces of environmental information, such as time information, weather information, temperature information, and season information.

[0249] The fixed processing unit 33 acquires time information from a clock (not shown), for example, during shooting. The fixed processing unit 33 acquires time information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.

[0250] For example, the fixed processing unit 33 acquires time information from a clock (not shown) during shooting, and acquires season information corresponding to the time information.

[0251] The fixed processing unit 33 acquires weather information, for example, during shooting. The fixed processing unit 33 acquires weather information corresponding to the location information of the fixed terminal 3, for example, from a server (not shown). The fixed processing unit 33 acquires weather information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.

[0252] The fixed processing unit 33 acquires temperature information, for example, during shooting. The fixed processing unit 33 acquires temperature information corresponding to the location information of the fixed terminal 3, for example, from a server (not shown). The fixed processing unit 33 acquires temperature information, for example, from a temperature sensor installed in a mobile object. The fixed processing unit 33 acquires temperature information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.

[0253] The fixed processing unit 33, for example, determines whether the acquired set of fixed attribute values ​​matches the preservation conditions. The fixed processing unit 33, for example, determines whether the acquired time-series set of fixed attribute values ​​matches the preservation conditions. For example, if the preservation conditions are met, the fixed processing unit 33 acquires a video corresponding to the set of fixed attribute values. For example, if the preservation conditions are met, the fixed processing unit 33 acquires a video corresponding to the preservation conditions. For example, if the preservation conditions are met, the fixed processing unit 33 acquires acquisition information paired with the preservation conditions and acquires a video in accordance with the acquisition information.

[0254] The fixed processing unit 33 preferably includes, for example, a microphone, acquires sound information, and stores the sound information in association with the fixed moving image acquired by the fixed camera 331 .

[0255] The fixed processing unit 33 analyzes the fixed video and acquires one or more tags for the fixed video. The process of acquiring the tags is the same as the process performed by the tag acquisition unit 232 described above.

[0256] The fixed camera 331 captures and acquires video. Such video is a fixed video. The fixed camera 331 is a camera that does not move. The fixed camera 331 is a camera that is installed at a fixed position. The fixed camera 331 is a camera whose shooting position is fixed. A fixed shooting position does not necessarily mean that the shooting direction is also fixed, and includes the case where the shooting direction changes.

[0257] The fixed transmission unit 34 transmits various types of information to the information processing device 1. The various types of information include, for example, fixed moving images, fixed attribute value sets, and sound information. <Details of Components of the User Terminal 4>

[0258] Various types of information are stored in the user storage unit 41 of the user terminal 4. The various types of information include, for example, a user identifier and a video.

[0259] The user reception unit 42 receives various instructions and information, such as inquiries and purchase instructions.

[0260] It should be noted that a purchase instruction is an instruction to purchase a video. The purchase instruction is associated with a user identifier. The purchase instruction typically includes information that identifies the video. The purchase instruction has, for example, a video identifier. The purchase instruction includes, for example, an inquiry. The purchase instruction includes, for example, purchase conditions. The purchase conditions are, for example, a purchase price. The purchase conditions include, for example, information that identifies the rights period.

[0261] The means for inputting various instructions and information may be any means, such as a touch panel, keyboard, mouse, or menu screen.

[0262] The user processing unit 43 performs various processes, such as creating a data structure for transmitting various instructions and information received by the user receiving unit 42. The various processes include creating a structure for transmitting information received by the user receiving unit 45.

[0263] The user transmission unit 44 transmits various instructions and information to the information processing device 1. The various instructions and information include, for example, an inquiry, a purchase instruction, location information received from the target terminal 5, and route information received from the navigation terminal 6.

[0264] The user receiving unit 45 receives various information and instructions, such as inquiries, videos, and status information.

[0265] The user output unit 46 outputs various types of information, such as video and status information.

[0266] Here, output is a concept that includes displaying on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and delivery of processing results to other processing devices or other programs.

[0267] The target terminal 5 acquires and transmits location information that identifies the location of the target terminal 5. The target terminal 5 transmits the location information to, for example, the information processing device 1 or the user terminal 4. It is preferable that the target terminal 5 transmits the location information in pair with a target person identifier. The target person identifier is information that identifies the person being monitored. The target person identifier is, for example, an ID, name, telephone number, email address, or MAC address of the target terminal 5. The target terminal 5 acquires the location information using, for example, a GPS receiver, but the method for acquiring the location information is not important.

[0268] The navigation terminal 6 has the functions of a known navigation terminal. The navigation terminal 6 accepts input of a destination, acquires the current location, and acquires route information from the current location to the destination. The navigation terminal 6, for example, transmits the accepted destination. The navigation terminal 6, for example, transmits the acquired route information. The destination to which the destination or route information is transmitted is, for example, the information processing device 1 or the user terminal 4. The navigation terminal 6 may receive the destination from the user terminal 4 and acquire route information from the current location to the destination.

[0269] The storage unit 11, mobile terminal management unit 111, fixed terminal management unit 112, mobile storage unit 21, fixed storage unit 31, and user storage unit 41 are preferably non-volatile recording media, but can also be realized as volatile recording media.

[0270] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0271] The receiving unit 12, the mobile information receiving unit 121, the inquiry receiving unit 122, the collective receiving unit 123, the mobile receiving unit 22, and the user receiving unit 45 are typically realized by wireless or wired communication means, but may also be realized by means for receiving broadcasts.

[0272] The processing unit 13, movement information storage unit 131, collection storage unit 132, video acquisition unit 133, video composition unit 134, video processing unit 135, rights holder processing unit 136, movement video acquisition means 1331, fixed video acquisition means 1332, first security means 1361, second security means 1362, third security means 1363, fourth security means 1364, reward means 1365, movement processing unit 23, tag acquisition unit 232, movement information acquisition unit 233, fixed processing unit 33, and user processing unit 43 can typically be realized using a processor, memory, etc. The processing procedures of the processing unit 13, etc. are typically realized using software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized using hardware (dedicated circuits). The processor may be a CPU, MPU, GPU, etc., and the type does not matter.

[0273] The system includes a transmission unit 14, a video transmission unit 141, a status transmission unit 142, and a needs transmission unit 143.

[0274] In the mobile terminal 2, the mobile transmitting unit 24, the mobile information transmitting unit 241, the mobile video transmitting unit 242, the collective transmitting unit 243, the fixed transmitting unit 34, and the user transmitting unit 44 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0275] The photographing unit 231 is realized by a camera. The fixed camera 331 is a fixed camera. The camera may be, for example, a CCD camera, a CMOS camera, a 3D camera, a LiDAR, or an omnidirectional camera, but the type is not limited thereto.

[0276] The user reception unit 42 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0277] The user output unit 46 may or may not include an output device such as a display or speaker. The user output unit 46 may be realized by driver software for an output device, or by a combination of driver software for an output device and the output device. <Operation>

[0278] Next, an example of the operation of the information system A will be described. First, an example of the operation of the information processing device 1 will be described using the flowchart in Fig. 5. Note that "S" in each flowchart used in the following description indicates a step.

[0279] (S501) The movement information receiving unit 121 determines whether movement information associated with a rights holder identifier has been received from the mobile terminal 2. If movement information has been received, the process proceeds to S502; if not, the process proceeds to S503.

[0280] (S502) The movement information receiving unit 121 associates the movement information received in S501 with the rights holder identifier and stores it in the storage unit 11. Return to S501. Note that if the movement information is a rights holder identifier, the movement information receiving unit 121 stores the rights holder identifier.

[0281] (S503) The set receiving unit 123 determines whether or not a moved attribute value set associated with the right holder identifier has been received from the mobile terminal 2. If a moved attribute value set has been received, the process proceeds to S504; if not, the process proceeds to S506.

[0282] (S504) The set accumulation unit 132 accumulates the moved attribute value set received in S503 in the set storage unit 111 in association with the right holder identifier.

[0283] (S505) The processing unit 13 deletes the movement information associated with the right holder identifier from the storage unit 11. Return to S501. Note that deleting the movement information is equivalent to "rewriting the movement information to information indicating that the video is not in a state where it can be transmitted."

[0284] (S506) The receiving unit 12 determines whether or not a video or the like associated with the rights holder identifier has been received from the mobile terminal 2 or the fixed terminal 3. If a video or the like has been received, the process proceeds to S507; if not, the process proceeds to S510.

[0285] (S507) The processing unit 13 performs processing to acquire an attribute value set corresponding to the moving image received in S506. An example of such attribute value set acquisition processing will be described using the flowchart in FIG. 6. The attribute value set acquisition processing here may also be processing to acquire an attribute value set associated with the received moving image. Note that the moving image here is a moving moving image or a fixed moving image. The attribute value set acquired here is a moving attribute value set or a fixed attribute value set.

[0286] (S508) The first security means 1361 stores the video received in S506 in association with the attribute value set acquired in S507. Note that the first security means 1361 stores the video in the storage unit 11, for example, but may store the video in another device such as a device constituting a blockchain. The process of storing the video by the first security means 1361 is the first security process.

[0287] (S509) The fourth maintenance means 1364 performs the fourth maintenance process. The process returns to S501. An example of the fourth maintenance process will be described with reference to the flowchart in FIG.

[0288] (S510) The inquiry receiving unit 122 determines whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S511; if not, the process proceeds to S519. The inquiry receiving unit 122 receives an inquiry from, for example, the user terminal 4 or the navigation terminal 6. The navigation terminal 6 that sends the inquiry may be the user terminal 4.

[0289] (S511) The video acquisition unit 133 determines whether the received query is a real-time video search. If it is a real-time video search, the process proceeds to S512. If it is not a real-time video search, the process proceeds to S515. This determination can be made by inspecting the received query. A real-time video search is a process in which the mobile terminal 2 or the fixed terminal 3 searches for videos that it currently holds or is currently shooting.

[0290] (S512) The video acquisition unit 133 determines whether the received query requires spatial fusion of multiple videos. If spatial fusion is required, the process proceeds to S513. If spatial fusion is not required, the process proceeds to S514. Note that this determination can be made by examining the received query.

[0291] (S513) The video acquisition unit 133, video composition unit 134, video processing unit 135, etc. perform video fusion processing. Proceed to S518. The video fusion processing is a process of spatially fusing multiple videos to acquire and transmit a video having one or more frames. An example of the video fusion processing will be described using the flowchart in FIG. 10. Note that the video to be transmitted may be a stereoscopic three-dimensional video.

[0292] (S514) The video acquisition unit 133, video composition unit 134, video processing unit 135, etc. perform video combination processing. Go to S518. The video combination processing is a process of transmitting a single video in which multiple videos are combined in time. An example of the video combination processing will be described using the flowchart in FIG. 12.

[0293] (S515) The video acquisition unit 133 and the like perform a registered video search process. An example of the registered video search process will be described with reference to the flowchart in Fig. 13. Note that there may be cases where the video acquisition unit 133 is unable to acquire a video.

[0294] (S516) The video acquisition unit 133 determines whether or not the video was acquired in S515. If the video was acquired, the process proceeds to S517, and if the video was not acquired, the process proceeds to S518.

[0295] (S517) The video acquisition unit 133 performs unregistered video search processing. An example of the unregistered video search processing will be described using the flowcharts in Figures 14 and 15. Note that there may be cases where the video acquisition unit 133 is unable to acquire a video.

[0296] (S518) The rights holder processing unit 136 performs a security process. Return to S501. An example of the security process will be described with reference to the flowchart in FIG.

[0297] (S519) The receiving unit 12 determines whether or not a purchase instruction has been received from the user terminal 4. If a purchase instruction has been received, the process proceeds to S520; if not, the process returns to S501.

[0298] (S520) The video acquisition unit 133 acquires a video corresponding to the purchase instruction. Note that the video acquisition unit 133 receives the video corresponding to the purchase instruction from, for example, the mobile terminal 2 or the fixed terminal 3. The video acquisition unit 133 also acquires the video corresponding to the purchase instruction from, for example, registered videos. The video acquisition unit 133 may also acquire the video corresponding to the purchase instruction from, for example, unregistered videos.

[0299] (S521) The fourth security unit 1364 acquires the user identifier corresponding to the user terminal 4 that sent the purchase instruction. This user identifier becomes the rights holder identifier for the purchased video.

[0300] (S522) The fourth security means 1364 performs the fourth security process using the user identifier acquired in S516. An example of the fourth security process will be described with reference to the flowchart in FIG.

[0301] (S523) The remuneration means 1365 performs remuneration processing for the original rights holder of the purchased video. Return to S501. An example of the remuneration processing will be described with reference to the flowchart in FIG.

[0302] In the flowchart of FIG. 5, the process ends when the power is turned off or an interrupt occurs to end the process.

[0303] 5, it is preferable that the receiver 12 receives and temporarily stores location information that identifies the current location of each of one or more mobile terminals 2, paired with an identifier (e.g., a rights holder identifier) ​​of the mobile terminal 2. The frequency or time interval at which the receiver 12 receives and temporarily stores the location information from the mobile terminal 2 is not important. For example, the receiver 12 continuously receives and temporarily stores the location information. For example, the receiver 12 periodically receives and temporarily stores the location information.

[0304] 5 , the receiving unit 12 may, for example, continuously receive location information from the user terminal 4, the target terminal 5, or the navigation terminal 6. In such a case, the information processing device 1 may, for example, use the received location information to receive a video acquired by the mobile terminal 2 or a video acquired by the fixed terminal 3, and transmit the video to the user terminal 4 or the navigation terminal 6. Note that the frequency or time interval at which the location information is continuously received does not matter.

[0305] 5, the receiving unit 12 may receive the fixed attribute value set and the fixed video from the fixed terminal 3, paired with an identifier (e.g., a rights holder identifier) ​​of the fixed terminal 3. In this case, the processing unit 13 stores the fixed attribute value set and the fixed video in the fixed terminal management unit 112, paired with the identifier of the fixed terminal 3.

[0306] Next, an example of the attribute value set acquisition process in S507 will be described with reference to the flowchart in FIG.

[0307] (S601) The movement processing unit 23, the collection storage unit 132, or the fixed processing unit 33 (hereinafter referred to as the movement processing unit 23, etc.) acquires one or more pieces of environmental information, etc. associated with a target video. The target video is a video for which a video attribute value is to be acquired, such as a received video, a captured video, or a video being captured. The target video is a moving video or a fixed video. The environmental information, etc. may be only environmental information, or may be environmental information and one or more moving object attribute values.

[0308] (S602) The movement processing unit 23 etc. acquires the rights holder identifier associated with the received video.

[0309] (S603) The movement processing unit 23 etc. determines whether or not to perform annotation processing on the video. If annotation processing is to be performed, the process proceeds to S604; if not, the process proceeds to S611. It is assumed that whether or not to perform annotation processing is determined in advance. It is also preferable to annotate the video on either the information processing device 1 or a terminal. The terminal here is the mobile terminal 2 or the fixed terminal 3.

[0310] (S604) The movement processing unit 23 and the like assign 1 to the counter i.

[0311] (S605) The movement processing unit 23 etc. determines whether the i-th annotation unit exists in the video. If the i-th annotation unit exists, the process proceeds to S606; if not, the process proceeds to S611. Note that the annotation unit is the target for which annotations are acquired, and is, for example, one frame, a set of a certain number of frames, or a video of a certain duration (for example, 5 seconds or 10 seconds).

[0312] (S606) The movement processing unit 23 etc. determines whether or not to use the moving object attribute value when annotating. If the moving object attribute value is to be used, the process proceeds to S607, and if not, the process proceeds to S608. It is assumed that whether or not to use the moving object attribute value when annotating is determined in advance. Furthermore, the movement processing unit 23 etc. can use the moving object attribute value only when the target video is a moving video.

[0313] (S607) The movement processing unit 23 and the like perform a moving object attribute value tag acquisition process. An example of the moving object attribute value tag acquisition process will be described with reference to the flowchart of FIG.

[0314] (S608) The movement processing unit 23 etc. determines whether or not to use a moving image when annotating. If a moving image is to be used, the process proceeds to S609, and if not, the process proceeds to S610. It is assumed that whether or not a moving image is to be used when annotating is determined in advance.

[0315] (S609) The movement processing unit 23 and the like perform a video tag acquisition process. An example of the video tag acquisition process will be described with reference to the flowchart of FIG.

[0316] (S610) The movement processing unit 23 and the like increment the counter i by 1. The process returns to S605.

[0317] (S611) The movement processing unit 23 etc. constructs an attribute value set having one or more pieces of environment information, one or more tags, and a rights holder identifier, and then returns to the upper-level processing.

[0318] In the flowchart of FIG. 5, the moving image may be divided into a plurality of inspection units, and the processes from S601 to S611 may be performed for each inspection unit.

[0319] Next, an example of the mobile object attribute value tag acquisition process in S607 will be described with reference to the flowchart of FIG.

[0320] (S701) The movement processing unit 23 or the set accumulation unit 132 assigns 1 to a counter i.

[0321] (S702) The movement processing unit 23 etc. determines whether or not the i-th attribute value tag condition exists. If the i-th attribute value tag condition exists, the process proceeds to S703, and if not, the process returns to the upper process.

[0322] (S703) The movement processing unit 23 etc. acquires the i-th attribute value tag condition.

[0323] (S704) The movement processing unit 23 etc. acquires one or more moving object attribute values ​​to be used for determining the i-th attribute value tag condition.

[0324] (S705) The movement processing unit 23 etc. determines whether or not one or more moving object attribute values ​​acquired in S704 satisfy the i-th attribute value tag condition. If the attribute value tag condition is satisfied, the process proceeds to S706; if not, the process proceeds to S707.

[0325] (S706) The movement processing unit 23 or the like acquires a tag that is paired with the i-th attribute value tag condition and associates the tag with the corresponding video. Associating a tag with a video usually means associating the tag with a field that constitutes the video.

[0326] (S707) The movement processing unit 23 and the like increment the counter i by 1. The process returns to S702.

[0327] Next, an example of the video tag acquisition process in S609 will be described with reference to the flowchart in FIG.

[0328] (S801) The movement processing unit 23, the set storage unit 132, or the fixed processing unit 33 assigns 1 to a counter i.

[0329] (S802) The movement processing unit 23 etc. determines whether or not the i-th moving image tag condition exists. If the i-th moving image tag condition exists, the process proceeds to S803, and if not, the process returns to the upper process.

[0330] (S803) The movement processing unit 23 and the like acquire the i-th moving image tag condition.

[0331] (S804) The movement processing unit 23 etc. acquires one or more moving image attribute values ​​to be used for determining the i-th moving image tag condition.

[0332] (S805) The movement processing unit 23 etc. determines whether or not the one or more video attribute values ​​acquired in S804 satisfy the i-th attribute value tag condition. If the attribute value tag condition is satisfied, the process proceeds to S806; if not, the process proceeds to S807.

[0333] (S806) The movement processing unit 23 or the like acquires a tag that pairs with the i-th attribute value tag condition and associates the tag with the corresponding video. Associating a tag with a video usually means associating the tag with one or more frames that pair with one or more video attribute values ​​used to determine the attribute value tag condition.

[0334] (S807) The movement processing unit 23 etc. increments the counter i by 1. The process returns to S802.

[0335] Next, an example of the fourth maintenance process of S509 will be described with reference to the flowchart of FIG.

[0336] (S901) The fourth security unit 1364 acquires access information that identifies a storage location of moving images, such as fusion moving images, combined moving images, and stereoscopic three-dimensional moving images.

[0337] (S902) The fourth maintenance unit 1364 acquires a set of attribute values ​​corresponding to the stored video. If the stored video is composed of multiple original videos, the set of attribute values ​​corresponding to the video is the set of attribute values ​​of each of the multiple original videos.

[0338] (S903) The fourth security means 1364 constructs security information including the access information acquired in S901, the attribute value set acquired in S902, and the rights holder identifier of the video. If a new rights holder identifier has been acquired, the fourth security means 1364 constructs security information including, for example, the new rights holder identifier and the original rights holder identifier. (S904) The fourth security means 1364 stores the security information constructed in S903. The process returns to the upper level process.

[0339] In S904, if preservation information for a video corresponding to the preservation information to be stored has already been stored, the preservation information configured in S903 is overwritten with the preservation information. This allows, for example, the transition of rights holders of the video to be managed. Furthermore, the fourth preservation unit 1364 stores the preservation information in a blockchain, for example.

[0340] Next, an example of the moving image fusion process in S513 will be described with reference to the flowchart in FIG.

[0341] (S1001) The video acquisition unit 133 acquires reference location information. The reference location information is, for example, location information included in the received inquiry. The reference location information is, for example, location information of the user terminal 4 or the target terminal 5. The reference location information is, for example, one piece of location information included in the route information included in the inquiry. The one piece of location information is the first piece of location information in the route information. The location information is, for example, (latitude, longitude), but may also be a location name (for example, the name of a parking lot), a place name, etc.

[0342] (S1002) The moving image acquiring unit 133 assigns 1 to a counter i.

[0343] (S1003) The video acquisition unit 133 determines whether or not there is an i-th terminal to which video can be transmitted. If there is an i-th terminal, the process proceeds to S1004; if there is no i-th terminal, the process proceeds to S1008. Note that a terminal to which video can be transmitted is usually a fixed terminal 3 or a mobile terminal 2 that is moving.

[0344] Furthermore, the mobile terminal 2 in motion is the mobile terminal 2 corresponding to the movement information stored in the storage unit 11. Furthermore, the fixed terminal 3 is the fixed terminal 3 corresponding to the fixed terminal information managed by the fixed terminal management unit 112. In other words, the video acquisition unit 133 determines, for example, whether or not the ith information exists among the movement information in the storage unit 11 or the fixed terminal information in the fixed terminal management unit 112.

[0345] (S1004) Video obtaining unit 133 obtains the latest location information transmitted by mobile terminal 2, which is the i-th terminal, or location information included in the fixed terminal information, which is the i-th terminal.

[0346] (S1005) The video acquisition unit 133 determines whether the location information acquired in S1004 satisfies a second location condition. If the second location condition is satisfied, the process proceeds to S1006. If the second location condition is not satisfied, the process proceeds to S1007.

[0347] (S1006) The video acquiring unit 133 acquires an image from the i-th terminal, pairs the image with the rights holder identifier and location information of the i-th terminal, and temporarily stores the image in a buffer (not shown).

[0348] The video acquisition unit 133 sends, for example, an inquiry to the i-th terminal 2, including a type identifier that specifies the type of image requested (e.g., "video," "still image," or "around-view image"), and receives an image corresponding to the inquiry from the i-th mobile terminal 2.

[0349] (S1007) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S1003.

[0350] (S1008) The moving image composition unit 134 calculates the difference (distance) between the position information for each image temporarily stored in a buffer (not shown) and the reference position information. The moving image composition unit 134 sorts the images in ascending order using the difference as a key.

[0351] (S1009) The video composition unit 134 acquires a reference image. The reference image is an image corresponding to the reference location information. The reference image is, for example, an image transmitted from a user terminal 4 located at the location indicated by the reference location information. Note that this image is of a type identified by a type identifier that specifies the type of image being requested.

[0352] (S1010) The moving image composition unit 134 assigns 1 to a counter j.

[0353] (S1011) The moving image composition unit 134 determines whether or not a jth other image exists among the images sorted in S1008. If a jth other image exists, the process proceeds to S1012; if not, the process proceeds to S1014. The other images are the images sorted in S1008.

[0354] (S1012) If the jth other image is an image of a moving image that does not correspond to the priority type (for example, a moving image), the moving image composition unit 134 determines whether or not a priority image that can be substituted for the jth other image exists. If a priority image that can be substituted exists, the process proceeds to S1013; if not, the process proceeds to S1014.

[0355] A replaceable priority image is an image captured of an area equivalent to the area of ​​the jth other image, and is an image paired with location information of a position whose distance from the position indicated by the location information of the jth other image is within a threshold. A priority image is an image of a moving image (e.g., a static moving image) corresponding to a priority type. Note that an equivalent area is, for example, an image in which the proportion of overlapping area is equal to or greater than a threshold, or the size of the overlapping area is equal to or greater than a threshold.

[0356] The video composition unit 134 checks whether an image is a priority image, for example, among images paired with position information of a position whose distance from the position indicated by the position information of the jth other image is within a threshold, in order of shortest distance.

[0357] (S1013) The moving image composition unit 134 acquires a replaceable priority image.

[0358] (S1014) The moving image composition unit 134 performs image fusion processing to fuse the reference image acquired in S1009 with the j-th other image or the priority image acquired in S1013. An example of the image fusion processing will be described with reference to the flowchart in FIG. 11.

[0359] (S1015) The moving image composition unit 134 increments the counter j by α. The process returns to S1011. Note that if the jth other image is used in the image fusion process, α=1, and if a priority image is used, α is the number by which the counter j is advanced to the next other image after the priority image.

[0360] (S1016) The moving image constructing unit 134 constructs an image that clearly indicates the position indicated by the reference position information in the fusion image finally constructed in S1014, and temporarily stores the image in a buffer (not shown). Note that this image is a fusion image.

[0361] (S1017) The video sending unit 141 sends the fusion image constructed in S1016. Note that the video sending unit 141 may send the fusion image finally constructed in S1014. The destination of the fusion image is usually the user terminal 4 that sent the inquiry.

[0362] (S1018) The moving image transmitting unit 141 determines whether or not to end the transmission of the fusion image. If the transmission of the fusion image is to be ended, the process proceeds to S1019, and if not, the process returns to S1001.

[0363] When an end instruction is received from the user terminal 4 that has sent the inquiry, the video sending unit 141 determines to end the transmission of the fusion image.

[0364] (S1019) The moving image composition unit 134 temporally combines the transmitted fusion images to compose a fusion moving image, and temporarily stores the fusion moving image in a buffer (not shown).

[0365] (S1020) The video processing unit 135 performs a 3D process on the fusion video created in S1019. The process returns to the upper level process. The 3D process is a process of performing image processing on videos that satisfy certain conditions to create a stereoscopic 3D video. An example of the 3D process will be described using the flowchart in FIG. 18.

[0366] 10, the video transmission unit 141 transmits the fusion image. However, instead of the process of S1017, the video transmission unit 141 may transmit the video that has been subjected to the three-dimensional processing in S1020.

[0367] Next, an example of the image fusion process in S1014 will be described with reference to the flowchart in FIG.

[0368] (S1101) The moving image composition unit 134 acquires a base image. The initial base image is the reference image acquired in S1009. The base image is also the base image updated in S1105 in the loop immediately preceding S1101 to S1105.

[0369] (S1102) The moving image composition unit 134 acquires area information that identifies the area covered by the base image acquired in S1101. The area information is, for example, a set of position information of vertices that form a polygon, or a set of position information of two diagonal corners that form a rectangle.

[0370] (S1103) The moving image composition unit 134 acquires an image to be used for image fusion. The other image to be used for image fusion is the j-th other image in S1011 or the priority image in S1013.

[0371] (S1104) The moving image composing unit 134 acquires area information that identifies the area covered by the image acquired in S1103.

[0372] (S1105) The moving image composition unit 134 uses the area information of the base image and the area information acquired in S1104 to determine an area in the base image where the image acquired in S1103 will be placed, arranges the other image in that area, and acquires an updated base image. The process then returns to the upper level process.

[0373] The two images to be subjected to the image fusion process (the base image and the image acquired in S1103) each have an overlapping area. The moving image composition unit 134 then fuses the two images based on the overlapping area to compose a single image. This image is the updated base image. Furthermore, image fusion processing of two images having an overlapping area can be performed using publicly known technology.

[0374] Next, an example of the moving image combining process in S514 will be described with reference to the flowchart in FIG.

[0375] (S1201) The video acquisition unit 133 acquires location information of a target. Note that the location information of the target is typically location information included in a query. The location information of the target is, for example, information that identifies a location close to the location indicated by the location information corresponding to the acquired video.

[0376] (S1202) The moving image acquiring unit 133 assigns 1 to a counter i.

[0377] (S1203) The video acquisition unit 133 determines whether or not there is an i-th terminal to which video can be transmitted, in order from a terminal corresponding to the priority type (for example, the fixed terminal 3) to a terminal not corresponding to the priority type (for example, the mobile terminal 2). If there is an i-th terminal, the process proceeds to S1204; if there is not, the process proceeds to S1214. Note that the terminals that can transmit video are usually the fixed terminal 3 and the mobile terminal 2 that is moving.

[0378] (S1204) Video acquisition unit 133 acquires location information of the i-th terminal. Note that if the i-th terminal is fixed terminal 3, video acquisition unit 133 normally acquires location information contained in the i-th fixed terminal information of fixed terminal management unit 112. If the i-th terminal is mobile terminal 2, video acquisition unit 133 acquires the latest location information transmitted by the i-th terminal.

[0379] (S1205) The video acquisition unit 133 determines whether the location information of the i-th terminal satisfies the first location condition for the target location information. If the first location condition is satisfied, the process proceeds to S1206. If the first location condition is not satisfied, the process proceeds to S1210.

[0380] (S1206) The video acquiring unit 133 acquires the video from the i-th terminal, pairs the video with the rights holder identifier and location information of the i-th terminal, and temporarily stores the video in a buffer (not shown).

[0381] In addition, the video acquisition unit 133 may, for example, send an inquiry to the i-th terminal including a type identifier (e.g., "video" or "still image") that specifies the type of image requested, and receive an image corresponding to the inquiry from the i-th terminal.

[0382] (S1207) The video sending unit 141 sends the video acquired in S1206. The video sending unit 141 usually sends the video to the user terminal 4 that has sent the inquiry.

[0383] (S1208) The video acquisition unit 133 acquires the target location information. Note that, for example, if the target terminal 5 that transmits the location information is moving, the target location information acquired here differs from the location information acquired in S1201. However, if the target terminal 5 is not moving, or if an inquiry having a fixed set of location information is received, the target location information acquired here is the same as the location information acquired in S1201.

[0384] (S1209) The video acquisition unit 133 acquires the location information of the i-th terminal. The process returns to S1205. Note that if the i-th terminal is a mobile terminal 2, the location information of the i-th terminal here is updated location information. If the i-th terminal is a fixed terminal 3, the location information of the i-th terminal here remains unchanged.

[0385] (S1210) The video composition unit 134 determines whether or not a transmitted video exists among the videos acquired by the i-th terminal. If a transmitted video exists, the process proceeds to S1211, and if a transmitted video does not exist, the process proceeds to S1213.

[0386] (S1211) The video composition unit 134 composes a combined video by combining the video acquired by the i-th terminal with the video already transmitted. The video composition unit 134 also adds the rights holder identifier of the i-th terminal to the combined video and temporarily stores the combined video in a buffer (not shown).

[0387] In the initial process of S1211, the combined moving image is the moving image acquired by the i-th terminal.

[0388] (S1212) The moving image acquiring unit 133 assigns 1 to the counter i. The process returns to S1203.

[0389] (S1213) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S1203.

[0390] (S1214) The moving image acquisition unit 133 determines whether or not to end the moving image combination process. If so, proceed to S1215; if not, proceed to S1216.

[0391] For example, if the next location information included in the received inquiry does not exist, or if an end instruction is received, the video acquisition unit 133 determines to end the video combination process.

[0392] (S1215) The video acquiring unit 133 acquires the location information of the next target. For example, the video acquiring unit 133 acquires the next location information included in the route information included in the inquiry. The process returns to S1203.

[0393] (S1216) The moving image composition unit 134 links the multiple combined moving images composed in S1211 in the order in which they were acquired to compose a combined moving image to be stored. Next, the moving image composition unit 134 pairs the combined moving image with the multiple rights holder identifiers and location information of the moving images that are the basis of the combined moving image, and temporarily stores the combined moving image in a buffer (not shown).

[0394] If there is only one combined moving image constructed in S1211, the moving image constructing unit 134 does not perform the process of combining the combined moving images.

[0395] (S1217) The moving image processing unit 135 performs a three-dimensional process on the combined moving image created in S1216. The process returns to the upper level process. An example of the three-dimensional process will be described with reference to the flowchart in FIG.

[0396] Next, an example of the registered moving image search process in S515 will be described with reference to the flowchart in FIG.

[0397] (S1301) The moving image acquiring unit 133 assigns 1 to a counter i.

[0398] (S1302) The video acquisition unit 133 determines whether the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1303; if not, the process proceeds to S1317.

[0399] The target location information is location information used to search for videos, such as one piece of location information in the received query or each piece of location information included in the route information in the received query.

[0400] (S1303) The video acquisition unit 133 acquires the i-th target location information from the received inquiry.

[0401] (S1304) The video acquisition unit 133 acquires all the security information. For example, the video acquisition unit 133 acquires the security information that exists on the blockchain.

[0402] (S1305) The moving image acquiring unit 133 assigns 1 to a counter j.

[0403] (S1306) The video acquisition unit 133 determines whether the j-th security information exists among the security information acquired in S1304. If the j-th security information exists, the process proceeds to S1307; if not, the process proceeds to S1316.

[0404] (S1307) The video acquisition unit 133 checks the jth security information and determines whether the video corresponding to the jth security information can be provided to a third party. If it can be provided, the process proceeds to S1308, and if it cannot be provided, the process proceeds to S1315. Note that "providable" means, for example, that it can be viewed, that it can be sold, or that it can be viewed and sold.

[0405] (S1308) The moving image acquiring unit 133 assigns 1 to a counter k.

[0406] (S1309) The video acquisition unit 133 determines whether the kth search unit exists in the jth maintenance information or the video corresponding to the jth maintenance information. If the kth search unit exists, the process proceeds to S1310; if not, the process proceeds to S1315.

[0407] The search unit is a quantity of video used to determine whether a query is satisfied. The search unit video is, for example, a video from the start to the end of filming, a video of a certain duration, a video until a predetermined event occurs (for example, a video from when the speed, which is a mobile object attribute value, is 0 to the next speed 0, a video from departing from a regular residence position (for example, a parking lot at home) to returning to the regular residence position), or a video of a predetermined period before and after a predetermined tag (for example, "accident" or "traffic jam") is attached. However, the search unit is not important.

[0408] (S1310) The moving image acquiring unit 133 acquires a set of moving image attribute values ​​of the kth search unit from the jth maintenance information.

[0409] (S1311) The video acquisition unit 133 determines whether the set of video attribute values ​​of the kth search unit for the i-th target location information satisfies the first location condition. If the first location condition is satisfied, the process proceeds to S1312; if not, the process proceeds to S1314.

[0410] (S1312) The video acquisition unit 133 acquires a video or the like corresponding to the j-th security information and temporarily stores it in a buffer (not shown). The video or the like is, for example, a set of video and video attribute values, or a video.

[0411] (S1313) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S1302.

[0412] (S1314) The moving image acquiring unit 133 increments the counter k by 1. The process returns to S1309.

[0413] (S1315) The moving image acquiring unit 133 increments the counter j by 1. The process returns to S1306.

[0414] (S1316) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S1302.

[0415] (S1317) The video composition unit 134 determines whether or not multiple videos, etc. have been acquired in S1312. If multiple videos, etc. have been acquired, the process proceeds to S1318, and if only one video, etc. has been acquired, the process proceeds to S1319.

[0416] (S1318) The moving image composition unit 134 combines the multiple moving images stored in a buffer (not shown) in the order in which they were stored to compose a combined moving image, and stores the combined moving image in a buffer (not shown). Note that it is preferable that the moving image composition unit 134 also associates the rights holder identifier of the moving images that are the basis of the combined moving image and one or more moving image attribute values ​​with the combined moving image and stores them in a buffer (not shown).

[0417] (S1319) The video sending unit 141 sends the combined video or the acquired video. The process returns to the upper level process. Note that it is preferable that the video be associated with a rights holder identifier and one or more video attribute values.

[0418] 13, the video transmission unit 141 may transmit the video every time the video is acquired in S1312. In this case, the process of S1319 is not necessary.

[0419] In the flowchart of FIG. 13, the video sending unit 141 normally sends the combined video or video to the user terminal 4 that sent the inquiry.

[0420] Next, a first example of the unregistered video search process of S517 will be described with reference to the flowchart of Fig. 14. The first example of the unregistered video search process is a case where attribute value sets of the mobile terminal management unit 111 and the fixed terminal management unit 112 are used.

[0421] (S1401) The moving image acquiring unit 133 assigns 1 to a counter i.

[0422] (S1402) The video acquisition unit 133 determines whether the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1403; if not, the process proceeds to S1420.

[0423] (S1403) The video acquisition unit 133 acquires the i-th target location information from the received inquiry.

[0424] (S1404) The moving image acquiring unit 133 assigns 1 to a counter j.

[0425] (S1405) Video acquisition unit 133 determines whether the jth attribute value set exists in the fixed attribute value set of fixed terminal management unit 112 or the mobile attribute value set of mobile terminal management unit 111. If the jth attribute value set exists, the process proceeds to S1406; if it does not exist, the process proceeds to S1419. Note that here, it is preferable for video acquisition unit 133 to check the fixed attribute value set of fixed terminal management unit 112.

[0426] (S1406) The moving image acquiring unit 133 assigns 1 to a counter k.

[0427] (S1407) The video acquisition unit 133 determines whether the kth search unit corresponding to the jth attribute value set exists. If the kth search unit exists, the process proceeds to S1408; if not, the process proceeds to S1414.

[0428] (S1408) The moving image obtaining unit 133 obtains a set of moving image attribute values ​​of the kth search unit in the jth attribute value set from the set storage unit 111. Note that the number of search units in the attribute value set may be one.

[0429] (S1409) The moving image acquiring unit 133 determines whether the set of moving image attribute values ​​acquired in S1408 satisfies the query. If the set satisfies the query, the process proceeds to S1410, and if not, the process proceeds to S1414.

[0430] (S1410) The moving image obtaining unit 133 obtains a rights holder identifier corresponding to the moving image.

[0431] (S1411) The video acquisition unit 133 determines whether or not it is possible to acquire the video corresponding to the rights holder identifier acquired in S1410. If it is possible to acquire the video, the process proceeds to S1412, and if it is not possible to acquire the video, the process proceeds to S1415.

[0432] In addition, video acquisition is possible when the terminal corresponding to the jth attribute value set is a fixed terminal 3, or when the terminal corresponding to the jth attribute value set is a mobile terminal 2 and movement information for that terminal exists (for example, when the power of mobile terminal 2 is ON).

[0433] (S1412) The video acquisition unit 133 acquires the video corresponding to the k-th search unit from the terminal corresponding to the rights holder identifier acquired in S1410, and temporarily stores the video in a buffer (not shown). Then, the process proceeds to S1419.

[0434] For example, the video acquisition unit 133 transmits a request for a video corresponding to the kth search unit to the terminal and receives the video corresponding to the request from the terminal. The range of videos corresponding to the kth search unit is not important. The videos corresponding to the kth search unit may be all videos stored in the terminal or only some of the videos.

[0435] (S1413) The moving image acquiring unit 133 increments the counter k by 1. The process returns to S1407.

[0436] (S1414) The moving image acquiring unit 133 increments the counter j by 1. The process returns to S1405.

[0437] (S1415) Video acquisition unit 133 acquires status information etc. Note that status information etc. includes, for example, status information indicating that the terminal is mobile terminal 2 and that its power is OFF.

[0438] (S1416) The status transmission unit 142 transmits the status information and the like acquired in S1415 to the user who transmitted the inquiry.

[0439] (S1417) The video acquisition unit 133 acquires needs information.

[0440] (S1418) The needs transmission unit 143 transmits the needs information acquired in S1417 to the rights holder of the video. Return to S1414.

[0441] (S1419) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S1402.

[0442] (S1420) The moving image composing unit 134 determines whether or not multiple moving images were acquired in S1412. If multiple moving images were acquired, the process proceeds to S1421, and if only one moving image was acquired, the process proceeds to S1422.

[0443] (S1421) The moving image composing unit 134 combines the multiple moving images stored in the buffer (not shown) in S1412 in the order in which they were stored to compose one combined moving image, and stores the combined moving image in the buffer (not shown).

[0444] (S1422) The video sending unit 141 sends the combined video or video. The process returns to the upper level process. Note that the video sending unit 141 normally sends the combined video or video to the user terminal 4 that sent the inquiry.

[0445] Next, a second example of the unregistered video search process of S517 will be described with reference to the flowchart of Fig. 15. The second example of the unregistered video search process is a case where an inquiry is made to the fixed terminal 3 or the mobile terminal 2.

[0446] (S1501) The moving image acquiring unit 133 assigns 1 to a counter i.

[0447] (S1502) The video acquisition unit 133 determines whether the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1503; if not, the process proceeds to S1512.

[0448] (S1503) The video acquisition unit 133 acquires the i-th target location information from the received inquiry.

[0449] (S1504) The moving image acquiring unit 133 assigns 1 to a counter j.

[0450] (S1505) The video acquisition unit 133 determines whether or not a jth terminal to which an inquiry can be sent exists. If a jth terminal exists, the process proceeds to S1503; if a jth terminal does not exist, the process proceeds to S1511.

[0451] The terminals that can send inquiries are terminals that are in a state where they can send moving images, and are fixed terminals 3 or mobile terminals 2 whose movement information exists in the storage unit 11 .

[0452] It is preferable that the video acquisition unit 133 selects as the j-th terminal only the fixed terminal 3 that is installed at a position indicated by location information that matches the first location condition for the i-th target location information. Also, it is preferable that the video acquisition unit 133 searches for a terminal that sends an inquiry from terminals that correspond to the priority type (usually the fixed terminal 3), and if there is no terminal that corresponds to the priority type that can send an inquiry, it searches for a terminal that does not correspond to the priority type (usually the mobile terminal 2).

[0453] (S1506) The video acquisition unit 133 transmits a query to the j-th terminal. Note that the video acquisition unit 133 preferably transmits the query from a terminal corresponding to the priority type (usually, the fixed terminal 3), and if there is no terminal that can respond to the query, transmits the query to a terminal that is not of the priority type (usually, the mobile terminal 2).

[0454] (S1507) The video acquisition unit 133 determines whether or not information has been received from the j-th terminal. If information has been received, the process proceeds to S1508, and if not, the process returns to S1507.

[0455] (S1508) The video acquisition unit 133 determines whether or not a video is included in the information received in S1507. If a video is included, the process proceeds to S1509, and if not, the process proceeds to S1510.

[0456] (S1509) The video acquisition unit 133 temporarily stores the video etc. received in S1507 in a buffer (not shown). Then, the process proceeds to S1511. The video etc. is, for example, a video, a set of attribute values, and a rights holder identifier.

[0457] (S1510) The moving image acquiring unit 133 increments the counter j by 1. The process returns to S1505.

[0458] (S1511) The moving image acquiring unit 133 increments the counter i by 1. The process returns to S1502.

[0459] (S1512) The moving image composing unit 134 determines whether or not multiple moving images were stored in S1509. If multiple moving images were stored, the process proceeds to S1513, and if only one moving image was acquired, the process proceeds to S1514.

[0460] (S1513) The moving image composing unit 134 combines the multiple moving images stored in the buffer (not shown) in S1509 in the order in which they were stored to compose one combined moving image, and stores the combined moving image in the buffer (not shown).

[0461] (S1514) The video sending unit 141 sends the combined video or video. The process returns to the upper level process. Note that the video sending unit 141 normally sends the combined video or video to the user terminal 4 that sent the inquiry.

[0462] Next, an example of the security process in S518 will be described with reference to the flowchart in FIG.

[0463] (S1601) The rights holder processing unit 136 stores the transmitted moving image in association with a set of attribute values ​​associated with one or more moving images that are the basis of the transmitted moving image.

[0464] It is preferable that the rights holder processing unit 136 stores each video in pairs with a rights holder identifier that identifies one or more rights holders of the video. The rights holder identifier here is, for example, one or more rights holder identifiers of the videos that are the source of the stored video. The rights holder identifier here is, for example, a single rights holder identifier that identifies the user who made the inquiry.

[0465] Furthermore, the rights holder processing unit 136 stores the video in, for example, the storage unit 11 or a device other than the information processing device 1. The device other than the information processing device 1 may be a device that constitutes a blockchain.

[0466] (S1602) The fourth maintenance means 1364 performs the fourth maintenance process. An example of the fourth maintenance process has been described with reference to the flowchart in FIG.

[0467] (S1603) The rights holder processing unit 136 assigns 1 to the counter i.

[0468] (S1604) The rights holder processing unit 136 determines whether the i-th video that was the source of the stored video exists. If the i-th video exists, the process proceeds to S1605; if not, the process returns to the upper process.

[0469] (S1605) Reward unit 1365 performs reward processing. An example of the reward processing will be described with reference to the flowchart in Fig. 17. The reward processing here is a process of paying reward to the rights holder of the i-th video that is the basis for the accumulated video.

[0470] (S1606) The rights holder processing unit 136 determines whether or not to change the rights holder of the i-th video that is the source of the stored video. If the rights holder is to be changed, the process proceeds to S1607; if the rights holder is not to be changed, the process proceeds to S1608.

[0471] Whether or not to change the rights holder may be based on a flag corresponding to the i-th video, may be predetermined, or the rights holder may be changed if the inquiry includes "information indicating a request to change the rights holder."

[0472] (S1607) The rights holder processing unit 136 acquires the user identifier of the user terminal 4. This user identifier becomes the new rights holder identifier.

[0473] (S1608) The rights holder processing unit 136 determines whether the i-th video that is the source of the stored video has already been registered. If it has already been registered, the process proceeds to S1611; if it has not yet been registered, the process proceeds to S1609.

[0474] (S1609) The first maintenance unit 1361 stores the i-th moving image that was the source of the stored moving images.

[0475] (S1610) The fourth maintenance unit 1364 performs the fourth maintenance process on the i-th video that was the source of the stored video. An example of the fourth maintenance process was described using the flowchart in FIG.

[0476] (S1611) The rights holder processing unit 136 increments the counter i by 1. The process returns to S1604.

[0477] In the flowchart of Figure 16, when a combined video is constructed, a process (S1609) of storing the multiple videos that formed the combined video and a fourth conservation process (S1610) are performed, but such processes do not have to be performed.

[0478] Next, an example of the reward process in S523 etc. will be described with reference to the flowchart in FIG.

[0479] (S1701) The reward unit 1365 acquires one or more right holder identifiers of the target video. The reward unit 1365 may also acquire the right holder identifiers of the previous right holders of the target video.

[0480] (S1702) The reward means 1365 acquires a set of attribute values ​​of the target video.

[0481] (S1703) The reward unit 1365 acquires a service identifier that identifies the service provided for the target video. The service identifier is, for example, “view” or “purchase.”

[0482] (S1704) The reward unit 1365 acquires a reward amount using one or more pieces of information from the attribute value set acquired in S1702 and the service identifier acquired in S1703.

[0483] If multiple right holder identifiers are acquired, the remuneration unit 1365 acquires a remuneration amount for each right holder identifier. If history information of right holders including multiple right holder identifiers is acquired, the remuneration unit 1365 may acquire a remuneration amount for each right holder identifier.

[0484] Furthermore, it is preferable that the remuneration means 1365, for example, acquires a video attribute value corresponding to each of the multiple videos that were the basis for the video transmitted by the video transmission unit 141, and uses the video attribute value to determine a remuneration amount for each of the multiple rights holders. It is preferable that the remuneration means 1365, for example, determines a larger remuneration amount the greater the data amount, video duration, or number of frames of the original video used in the video transmitted by the video transmission unit 141. It is preferable that the remuneration means 1365, for example, determines a larger remuneration amount the greater the resolution of the original video used in the video transmitted by the video transmission unit 141.

[0485] (S1705) The reward unit 1365 performs processing to give a reward of the amount of reward acquired in S1704 to the right holder identified by the right holder identifier acquired in S1701.

[0486] (S1706) The remuneration means 1365 performs processing to charge a remuneration to the user who has enjoyed the service related to the target video. The processing returns to the upper level processing. Note that the target video is usually a video transmitted to the user terminal 4.

[0487] In the flowchart of FIG. 17, the profits obtained by the operator of the information processing device 1 may be acquired and accumulated.

[0488] Next, an example of the three-dimensional processing in steps S1019 and S1217 will be described with reference to the flowchart in FIG.

[0489] (S1801) The moving image processing unit 135 assigns 1 to a counter i.

[0490] (S1802) The moving image processing unit 135 determines whether the i-th inspection unit exists in the target moving image. If the i-th inspection unit exists, the process proceeds to S1803, and if the i-th inspection unit does not exist, the process returns to the upper process.

[0491] The inspection unit here refers to the inspection target of the moving image for determining at once whether or not the captured moving image to be processed should be converted into a stereoscopic three-dimensional moving image. The inspection unit may be, for example, a predetermined number of frames, a predetermined time, each of the moving images that are the source of the captured moving image to be processed, or the entire object to be processed. The captured moving image to be processed may be, for example, a fused moving image or a combined moving image.

[0492] (S1803) The moving image processing unit 135 acquires one or more moving image attribute values ​​that are used to determine processing conditions and that are paired with the moving image of the i-th inspection unit.

[0493] The one or more video attribute values ​​are, for example, tags, time information, weather information, temperature information, and season information.

[0494] (S1804) The moving image processing unit 135 determines whether or not the one or more moving image attribute values ​​acquired in S1803 satisfy the processing conditions. If the processing conditions are satisfied, the process proceeds to S1805, and if the processing conditions are not satisfied, the process proceeds to S1811.

[0495] (S1805) The moving image processing unit 135 assigns 1 to a counter j.

[0496] (S1806) The moving image processing unit 135 determines whether or not the jth frame corresponding to the ith inspection unit exists. If the jth frame exists, the process proceeds to S1807, and if the jth frame does not exist, the process proceeds to S1811.

[0497] Note that the frame corresponding to the i-th inspection unit is usually a frame included in the i-th inspection unit, but it may also be a frame temporally before or after the i-th inspection unit, etc. For example, if the processing condition is "tag = accident," the frames to be made into the stereoscopic 3D video may also include frames before and after the accident occurs.

[0498] (S1807) The moving image processing unit 135 acquires the j-th frame corresponding to the i-th inspection unit.

[0499] (S1808) The moving image processing unit 135 acquires a stereoscopic 3D image from the frames acquired in S1807.

[0500] (S1809) The moving image processing unit 135 replaces the j-th frame with the stereoscopic 3D image acquired in S1808.

[0501] (S1810) The moving image processing unit 135 increments the counter j by 1. The process returns to S1806.

[0502] (S1811) The moving image processing unit 135 increments the counter i by 1. The process returns to S1802.

[0503] Next, an example of the operation of the mobile terminal 2 will be described with reference to the flowchart of FIG.

[0504] (S1901) The movement processing unit 23 determines whether or not the start of movement has been detected. If the start of movement has been detected, the process proceeds to S1902, and if not, the process proceeds to S1917. The start of movement is determined, for example, by the engine of the moving object on which the mobile terminal 2 is installed being turned on or the power of the mobile terminal 2 being turned on.

[0505] (S1902) The movement information acquisition unit 233 acquires movement information. The movement information transmission unit 241 transmits the movement information to the information processing device 1. Note that the movement information is stored in the movement storage unit 21, for example.

[0506] (S1903) The photographing unit 231 starts photographing.

[0507] (S1904) The image capturing unit 231 acquires a video.

[0508] (S1905) The movement processing unit 23 acquires one or more moving image attribute values, and associates the one or more moving image attribute values ​​with the moving images acquired in S1904.

[0509] The movement processing unit 23 acquires, for example, location information, time information, weather information, temperature information, and season information. The movement processing unit 23 also acquires, for example, one or more moving object attribute values ​​(for example, CAN data).

[0510] (S1906) The tag acquisition unit 232 determines whether or not to acquire a moving object attribute value tag. If a moving object attribute value tag is to be acquired, the process proceeds to S1907; if not, the process proceeds to S1909. Note that whether or not to acquire a moving object attribute value tag is determined in advance.

[0511] (S1907) The tag acquisition unit 232 performs a mobile object attribute value tag acquisition process. An example of the mobile object attribute value tag acquisition process has been described with reference to the flowchart in Fig. 7. Note that there may be cases where a tag cannot be acquired at this stage.

[0512] (S1908) If one or more tags are acquired in S1907, the tag acquiring unit 232 associates the one or more tags with the video acquired in S1904.

[0513] It is preferable that the tag acquisition unit 232 associates one or more tags with the video and stores them in the mobile storage unit 21. This allows the user of the mobile terminal 2 to search for videos using the tags as a key, for example.

[0514] (S1909) The tag acquisition unit 232 determines whether or not to acquire a moving image tag. If a moving image tag is to be acquired, the process proceeds to S1910; if not, the process proceeds to S1912. Note that whether or not to acquire a moving image tag is determined in advance.

[0515] (S1910) The tag obtaining unit 232 performs a video tag obtaining process. An example of the video tag obtaining process has been described with reference to the flowchart in Fig. 8. Note that there may be cases where a tag cannot be obtained at this stage.

[0516] (S1911) If one or more tags are acquired in S1910, the tag acquiring unit 232 associates the one or more tags with the video acquired in S1904.

[0517] It is preferable that the tag acquisition unit 232 associates one or more tags with the video and stores them in the mobile storage unit 21. This allows the user of the mobile terminal 2 to search for videos using the tags as a key, for example.

[0518] (S1912) The movement processing unit 23 determines whether or not the end of movement has been detected. If the end of movement has been detected, the process proceeds to S1913, and if not, the process returns to S1904.

[0519] The movement processing unit 23 detects the end of movement by, for example, detecting that the engine is turned off, the power of the mobile terminal 2 is turned off, or by detecting arrival at the destination.

[0520] (S1913) The movement processing unit 23 acquires the rights holder identifier from the movement storage unit 21.

[0521] (S1914) The movement processing unit 23 generates an attribute value set that includes one or more moving object attribute values ​​acquired in S1905, one or more moving object attribute values ​​and tags acquired in S1907, and one or more tags acquired in S1910. Note that the attribute value set does not include information that was not acquired above.

[0522] (S1915) The set sending unit 243 sends the attribute value set constructed in S1914 to the information processing device 1 in association with the rights holder identifier.

[0523] (S1916) The moving video transmission unit 242 etc. performs video transmission processing. Return to S1901. An example of the video transmission processing will be described with reference to the flowchart in Fig. 20. The video transmission processing is processing for automatically transmitting videos that meet the preservation conditions.

[0524] (S1917) The mobile receiving unit 22 determines whether or not an inquiry has been received from the information processing device 1. If an inquiry has been received, the process proceeds to S1918, and if not, the process returns to S1901.

[0525] (S1918) The movement processing unit 23 performs a terminal image acquisition process corresponding to the inquiry received in S1917. An example of the terminal image acquisition process will be described using the flowchart in Fig. 21. The terminal image acquisition process is a process of acquiring a video that matches the inquiry from the videos stored in the movement storage unit 21.

[0526] (S1919) The movement processing unit 23 determines whether or not an image was acquired in S1918. If an image was acquired, the process proceeds to S1920, and if not, the process proceeds to S1921.

[0527] (S1920) The moving video sending unit 242 sends images and the like to the information processing device 1. Return to S1901. Note that the images and the like are, for example, an image, a set of attribute values, and a rights holder identifier. The images are, for example, videos, still images, and around-view images.

[0528] (S1921) The moving video transmission unit 242 transmits error information. The error information is information indicating that a video could not be acquired.

[0529] In the flowchart of Figure 19, it is preferable to transmit a set of video attribute values, which are one or more video attribute values ​​acquired in S1905, moving object attribute values ​​and tags acquired in S1907, and all or part of the tags acquired in S1910, to the information processing device 1 each time a video is acquired, at predetermined intervals, or when a predetermined condition is met (for example, the tag ``accident'' or ``traffic jam'' is acquired).

[0530] 19, the video transmission process (processing of S1916) is performed after the end of movement is detected, but the video transmission process may be performed continuously. Continuously performing the video transmission process means, for example, performing the video transmission process (S1916) immediately after S1911. This allows videos that meet the preservation conditions to be preserved even if the storage area for videos in the storage unit 11 is small.

[0531] Furthermore, in the flowchart of FIG. 19, the process ends when the power is turned off or an interrupt occurs to end the process.

[0532] Next, an example of the video transmission process in S1916 will be described with reference to the flowchart in FIG.

[0533] (S2001) The movement processing unit 23 acquires a set of attribute values ​​from the movement storage unit 21.

[0534] (S2002) The movement processing unit 23 assigns 1 to the counter i.

[0535] (S2003) The movement processing unit 23 determines whether or not the i-th search unit exists. If the i-th search unit exists, the process proceeds to S2004, and if not, the process returns to the upper process.

[0536] (S2004) The movement processing unit 23 acquires a set of video attribute values ​​of the i-th search unit.

[0537] (S2005) The movement processing unit 23 assigns 1 to the counter j.

[0538] (S2006) The movement processing unit 23 determines whether or not the j-th maintenance condition exists. If the j-th maintenance condition exists, the process proceeds to S2007, and if not, the process proceeds to S2010.

[0539] (S2007) The movement processing unit 23 determines whether the set of video attribute values ​​acquired in S2004 satisfies the j-th conservation condition. If the j-th conservation condition is satisfied, the process proceeds to S2008; if not, the process proceeds to S2011.

[0540] (S2008) The movement processing unit 23 acquires the video etc. of the i-th search unit. Note that the video etc. of the i-th search unit is, for example, the video of the i-th search unit and a set of attribute values ​​corresponding to the video. Furthermore, the video of the i-th search unit may include, for example, the video before the i-th search unit and the video after the i-th search unit.

[0541] (S2009) The moving video transmission unit 242 transmits the video etc. acquired in S2008 to the information processing device 1.

[0542] (S2010) The movement processing unit 23 increments the counter i by 1. The process returns to S2003.

[0543] (S2011) The movement processing unit 23 increments the counter j by 1. The process returns to S2006.

[0544] Next, an example of the terminal image acquisition process in S1918 will be described with reference to the flowchart in FIG.

[0545] (S2101) The movement processing unit 23 uses the received inquiry to determine whether or not to acquire a video. If a video is to be acquired, the process proceeds to S2102, and if a still image or the like is to be acquired, the process proceeds to S2108.

[0546] The movement processing unit 23 checks, for example, the type identifier (for example, "video", "still image", or "around-view image") included in the received inquiry, and determines whether or not to acquire a video.

[0547] (S2102) The movement processing unit 23 assigns 1 to the counter i.

[0548] (S2103) The movement processing unit 23 determines whether or not the i-th search unit exists. If the i-th search unit exists, the process proceeds to S2104, and if not, the process returns to the upper process.

[0549] (S2104) The movement processing unit 23 acquires a set of video attribute values ​​of the i-th search unit.

[0550] (S2105) The movement processing unit 23 determines whether the set of video attribute values ​​of the i-th search unit matches the query. If it matches the query, the process proceeds to S2106, and if it does not match, the process proceeds to S2107.

[0551] (S2106) The movement processing unit 23 acquires the video etc. of the i-th search unit. Note that the video etc. is, for example, a set of video and video attribute values.

[0552] (S2107) The movement processing unit 23 increments the counter i by 1. The process returns to S2103.

[0553] (S2108) The movement processing unit 23 acquires the latest composite image or the latest still image, and returns to the upper-level processing.

[0554] The composite image is an image synthesized using still images captured by multiple cameras installed on a moving object. The composite image is, for example, an around-view image. The still image is an image captured by one camera installed on the moving object. The technology for constructing around-view images is well known, and therefore a detailed description thereof will be omitted.

[0555] Next, an example of the operation of the fixed terminal 3 will be described with reference to the flowchart of FIG.

[0556] (S2201) The fixed camera 331 starts capturing images. Note that the trigger for starting capturing images may be, for example, power-on or receipt of an instruction from the user.

[0557] (S2202) The fixed processing unit 33 acquires the video captured by the fixed camera 331 and temporarily stores it in a buffer (not shown).

[0558] (S2203) The fixed processing unit 33 acquires one or more moving image attribute values, such as time information, weather information, temperature information, and season information.

[0559] (S2204) The fixed processing unit 33 associates the one or more moving image attribute values ​​acquired in S2203 with the moving image acquired in S2202.

[0560] (S2205) The fixed processing unit 33 determines whether or not to acquire a moving image tag. If a moving image tag is to be acquired, the process proceeds to S2206. If a moving image tag is not to be acquired, the process proceeds to S2208. Note that whether or not to acquire a moving image tag is determined in advance.

[0561] (S2206) The fixed processing unit 33 acquires a moving image tag. Note that an example of such moving image tag acquisition processing is the same as the processing in the flowchart in FIG.

[0562] (S2207) The fixed processing unit 33 associates the one or more tags acquired in S2206 with the moving image acquired in S2202.

[0563] (S2208) The fixed processing unit 33 determines whether or not to end the process. If it is determined that the process should be ended, the process ends. If it is determined that the process should not be ended, the process proceeds to S2209. Note that the fixed processing unit 33 determines that the process should be ended, for example, when the power is turned off or when an end instruction is received.

[0564] (S2209) Fixed receiving unit 32 determines whether or not an inquiry has been received from information processing device 1. If an inquiry has been received, the process proceeds to S2210, and if not, the process proceeds to S2202.

[0565] (S2210) The fixed processing unit 33 acquires a moving image corresponding to the inquiry. An example of such fixed moving image acquisition processing will be described with reference to the flowchart in FIG.

[0566] (S2211) The fixed transmission unit 34 transmits the fixed video etc. acquired in S2210 to the information processing device 1. Note that the fixed video etc. is, for example, a fixed video, one or more video attribute values, and a rights holder identifier.

[0567] (S2212) The fixed processing unit 33 determines whether or not to end the transmission of the fixed video. If the transmission is to be ended, the process returns to S2202, and if the transmission is not to be ended, the process returns to S2210.

[0568] Next, an example of the fixed video acquisition process of S2210 will be described with reference to the flowchart of Fig. 23. In the flowchart of Fig. 23, the description of the same steps as in the flowchart of Fig. 22 will be omitted.

[0569] (S2301) The fixed processing unit 33 determines whether or not to acquire past fixed videos, which are fixed videos that have already been stored. If past fixed videos are to be acquired, the process proceeds to S2302; if past fixed videos are not to be acquired, the process proceeds to S2202. The past videos are stored in a buffer (not shown).

[0570] It should be noted that, for example, when the received inquiry includes time information indicating a time in the past, the fixed processing unit 33 determines to acquire the past fixed video.

[0571] (S2302) The fixed processing unit 33 assigns 1 to the counter i.

[0572] (S2303) The fixed processing unit 33 determines whether or not the i-th search unit exists in the fixed moving images stored in the fixed storage unit 31. If the i-th search unit exists, the process proceeds to S2304, and if the i-th search unit does not exist, the process returns to the upper process.

[0573] (S2304) The fixed processing unit 33 acquires the attribute value set corresponding to the i-th search unit from the fixed storage unit 31.

[0574] (S2305) The fixed processing unit 33 determines whether the attribute value set acquired in S2304 matches the received query. If it matches the query, the process proceeds to S2306, and if it does not match the query, the process proceeds to S2307.

[0575] (S2306) The fixed processing unit 33 acquires a fixed video, etc. corresponding to the i-th search unit. The fixed video, etc. here may be, for example, a fixed video, an attribute value set, and a rights holder identifier. The fixed video here may be the video of the i-th search unit, or may include videos that are temporally before and after the video of the i-th search unit based on the video of the i-th search unit.

[0576] (S2307) The fixed processing unit 33 increments the counter i by 1. The process returns to S2303.

[0577] Next, an example of the operation of the user terminal 4 will be described with reference to the flowchart of FIG.

[0578] (S2401) The user reception unit 42 determines whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S2402, and if not, the process proceeds to S2407.

[0579] The reception of an inquiry includes, for example, reception of a user's input, reception of location information from the target terminal 5, and reception of destination or route information from the navigation terminal 6.

[0580] (S2402) The user processing unit 43 composes a query to be transmitted. Next, the user transmitting unit 44 transmits the query to the information processing device 1 in association with the user identifier.

[0581] (S2403) The user receiving unit 45 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S2404, and if information has not been received, the process returns to S2403.

[0582] (S2404) The user processing unit 43 determines whether or not the information received in S2403 includes a moving image. If a moving image is included, the process proceeds to S2405, and if a moving image is not included, the process proceeds to S2406.

[0583] (S2405) User output unit 46 outputs the received video. Return to S2401.

[0584] (S2406) User output unit 46 outputs the received other information. Return to S2401. Note that the other information may be, for example, information indicating that the video could not be acquired.

[0585] (S2407) The user processing unit 43 determines whether it is time to make an inquiry. If it is time to make an inquiry, the process proceeds to S2408, and if it is not time to make an inquiry, the process proceeds to S2409.

[0586] The timing of the inquiry is, for example, when a destination is set in the user terminal 4, which is a navigation terminal, or when the user terminal 4 installed in the vehicle detects congestion.

[0587] (S2408) The user processing unit 43 performs an inquiry generation process. The process proceeds to S2402. An example of the inquiry generation process will be described with reference to the flowchart in FIG.

[0588] (S2409) The user receiving unit 42 determines whether or not a purchase instruction has been received from the user. If a purchase instruction has been received, the process proceeds to S2410, and if not, the process returns to S2401. The purchase instruction may include, for example, an inquiry.

[0589] (S2410) The user processing unit 43 composes a purchase instruction to be transmitted. Next, the user transmitting unit 44 transmits the purchase instruction to the information processing device 1 in association with the user identifier.

[0590] (S2411) The user receiving unit 45 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S2412, and if information has not been received, the process returns to S2411. The information may be, for example, a video, information indicating that the transfer of rights holders has been completed, and a set of attribute values ​​of the video.

[0591] (S2412) User processing unit 43 uses the received information to compose information to be output. User output unit 46 outputs the information. Return to S2401.

[0592] In the flowchart of FIG. 24, the process ends when the power is turned off or an interrupt occurs to end the process.

[0593] Next, an example of the inquiry generation process in S2408 will be described with reference to the flowchart in FIG.

[0594] (S2501) The user processing unit 43 acquires a destination.

[0595] (S2502) The user processing unit 43 acquires the current location.

[0596] (S2503) The user processing unit 43 searches for a route using the destination and the current location, and acquires route information.

[0597] (S2504) The user processing unit 43 acquires one or more pieces of point information that are points on the route specified by the route information acquired in S2503 and that satisfy the extraction conditions. Note that the point information is usually location information included in the route information.

[0598] (S2505) The user processing unit 43 composes a query using one or more pieces of location information acquired in S2504. The process returns to the upper level process. For example, the user processing unit 43 composes a query such as "Search for videos that were taken within the next hour, have a tag of either 'traffic jam' or 'accident' added to them, and are paired with an attribute value set that includes one or more pieces of location information."

[0599] 25, the user processing unit 43 may acquire the current location and generate a query including the current location. The query is a request to transmit videos corresponding to a predetermined tag (e.g., "accident" or "traffic jam") in a surrounding area including the current location (e.g., a distance within a threshold).

[0600] Next, an example of the operation of the target terminal 5 will be described. The target terminal 5 acquires location information. Next, the target terminal 5 transmits the location information to the information processing device 1 or a user terminal 4 registered in the target terminal 5. Note that the target terminal 5 acquires and transmits the location information, for example, by receiving an instruction from the holder of the target terminal 5 (for example, a child being watched over). Note that the target terminal 5 has, for example, a GPS receiver and acquires the location information using the GPS receiver, but the method for acquiring the location information is not important. Furthermore, the user terminal 4 registered in the target terminal 5 is, for example, the guardian of the holder of the target terminal 5.

[0601] Next, an example of the operation of the navigation terminal 6 will be described. The navigation terminal 6 accepts a destination from a user. Next, the navigation terminal 6 performs the inquiry generation process described in FIG. 25. Next, the navigation terminal 6 performs the processes from S2402 to S2405 and S2406 in FIG. 24. In other words, the navigation terminal 6 accepts the destination, acquires route information, uses the route information to make an inquiry to the information processing device 1, and receives and outputs videos of one or more points on the route indicated by the route information from the information processing device 1. <Specific Example of Operation>

[0602] A specific example of the operation of the information system A in this embodiment will be described below.

[0603] Currently, a mobile terminal management table having the structure shown in FIG. 26 is stored in the mobile terminal management unit 111 of the information processing device 1. The mobile terminal management table is a table that manages information on one or more mobile terminals. The mobile terminal management table is a table that manages mobile terminals 2 that transmit moving videos. Note that a mobile terminal 2 that is not managed in the mobile terminal management table may also transmit videos to the information processing device 1.

[0604] The mobile terminal management table manages one or more records each having an "ID," "terminal identifier," "video information," "movement information," "registration flag," and "provision flag." "Video information" is information about the video that has been shot, and includes a "frame identifier" and a "video attribute value." The "video attribute value" includes "environmental information" and "tag." "Environmental information" includes "location information," "direction information," "camera information," "time information," "weather information," and "temperature information." "Camera information" includes "angle of view" and "resolution." "Environmental information" is information about the environment around the mobile terminal 2 when the video was captured, etc. "Tags" include "accident," "traffic jam," and "dangerous driving." In other words, here, a video can be assigned one or more tags, for example, "accident," "traffic jam," and "dangerous driving."

[0605] "ID" is information for identifying a record. "Terminal Identifier" is an identifier for the mobile terminal 2, and is the same as the rights holder identifier that identifies the rights holder of the video when the video is transmitted. "Frame Identifier" is the ID of a frame that constitutes the video. A frame may also be called a field or a still image. "Location Information" here is (latitude, longitude). "Direction Information" is the camera's shooting direction, which is the angle clockwise from true north. In other words, if the camera's shooting direction is due east, the direction information is "90 degrees." "Angle of View" is the camera's angle of view. "Resolution" is the camera's resolution. "Time Information" here is the year, month, day, hour, minute, and second. "Weather Information" is, for example, "Sunny," "Rainy," "Cloudy," or "Snowy." "Temperature Information" is the temperature (°C) outside the moving object. A value of "1" for "Accident" indicates that a tag indicating an accident has occurred is attached to the corresponding frame. A value of "1" for "Traffic Jam" indicates that a tag indicating a traffic jam has occurred is attached to the corresponding frame. A value of "1" for "Dangerous Driving" indicates that a tag indicating that the car or the like in front is driving dangerously has been attached to the corresponding frame. "Movement Information = 1" indicates that the mobile terminal 2 is currently in a state where it can transmit video. "Movement Information = 0" indicates that the mobile terminal 2 is currently in a state where it cannot transmit video because it is turned off, etc. "Registration Flag = 1" indicates that the video has already been registered and can be obtained from the registration destination (e.g., storage unit 11 or another device). "Provision Flag" of "1" indicates that viewing of the video is permitted. "Provision Flag" of "2" indicates that sale of the video (transfer of rights holder) is permitted.

[0606] The fixed terminal management unit 112 of the information processing device 1 also stores a fixed terminal management table having the structure shown in Fig. 27. The fixed terminal management table is a table for managing information on one or more fixed terminals. The fixed terminal management table is a table for managing fixed terminals 3 that transmit fixed videos.

[0607] The fixed terminal management table manages one or more records each having an "ID," "terminal identifier," "location information," "direction information," and "camera information." The "camera information" here includes an "angle of view" and a "resolution."

[0608] Furthermore, a registered video management table having the structure shown in FIG. 28 is stored on the blockchain. The registered video management table is a table for managing registered videos. The registered video management table is a table for managing preservation information. Registered videos are, for example, videos provided to users in response to inquiries from the users, and videos that satisfy preservation conditions. In other words, here, videos that match inquiries received by the information processing device 1 and videos that satisfy preservation conditions are automatically accumulated.

[0609] The registered video management table manages one or more records having an "ID," "video identifier," "access information," "rights holder identifier," "rights registration date," "video information," and "provision flag." Registered videos are stored in the information processing device 1, another device, or a blockchain. Each record in the registered video management table is considered to be preservation information.

[0610] In this context, five specific examples will be described. Specific examples 1 to 4 use real-time video. Specific example 5 uses video captured in the past. <Specific Examples>

[0611] Specific example 1 is a case where, in order to monitor a target person (e.g., a child or a wandering elderly person), a combined video is created by combining fixed images taken by one or more fixed terminals 3 with moving videos taken by mobile terminals 2 installed on one or more mobile bodies, and the combined video is sent to a user terminal 4 of a user (e.g., a guardian) related to the target person.

[0612] Specific example 2 is a case where a combined video combining one or more fixed images and one or more moving videos is output to the user terminal 4 in response to an inquiry using route information corresponding to a destination set on the user terminal 4 or the navigation terminal 6.

[0613] Specific example 3 is a case where the user terminal 4 or navigation terminal 6 detects a traffic jam, acquires a combined video that allows the user to understand the reason for the traffic jam, and outputs the combined video to the user terminal 4 or navigation terminal 6. Note that the combined video here is a video that combines one or more fixed images with one or more moving videos.

[0614] Specific example 4 is a case where a fused video is output that allows a user to understand the parking lot situation so that they can find an available parking space in a parking lot. Note that the fused video here is a video that is composed of one or more fixed images and one or more moving videos. In other words, one or more fixed terminals 3 are installed in the parking lot.

[0615] Specific example 5 is a case where route information specifying a route the user has traveled in the past (e.g., a route they have traveled) is used to output a combined video that combines registered videos to the user terminal 4. Note that the combined video here is a video that combines one or more fixed images with one or more moving videos.

[0616] (Specific Example 1) The storage unit 11 of the information processing device 1 stores management information having a user identifier (e.g., the IP address of the user terminal 4) for sending a video to a user terminal 4 of a guardian P and a target identifier "T001" for identifying a target terminal 5 of a child A of the guardian P.

[0617] Next, assume that child A turns on the target terminal 5 to return home from school. Next, the target terminal 5 periodically acquires location information and sends a query (for example, "Video transmission instruction, target identifier=T001, location information (x ｔ１ , y ｔ１ )) to the information processing device 1. It is assumed that the target terminal 5 stores a target user identifier "T001" and communication destination information (e.g., IP address) of the information processing device 1 for transmitting information to the information processing device 1.

[0618] In addition, the receiving unit 12 of the information processing device 1 periodically receives location information that identifies the location of the mobile terminal 2, paired with the terminal identifier, from each of one or more mobile terminals 2, and stores the location information paired with the terminal identifier in the mobile terminal management table (Figure 26).

[0619] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry “video transmission instruction target person identifier=T001, location information (x ｔ１ , y ｔ１ ) from the target terminal 5 and stores the location information (xｔ１ , y ｔ１ ) and the target user identifier "T001" are temporarily stored. It is assumed that the location information of the target terminal 5 is received periodically, and the latest location information is stored in a buffer (not shown).

[0620] Next, the video acquisition unit 133 determines the fixed terminal 3 or the first mobile terminal 2 corresponding to the location information that satisfies the first location condition with respect to the location indicated by the latest location information of the target terminal 5, by the operation described in the flowchart of FIG. 12 . Note that here, it is assumed that the priority video is a "fixed video." Then, the video acquisition unit 133 checks the fixed terminal management table ( FIG. 27 ) to determine whether or not there is location information, directional information, and angle of view that satisfy the first location condition (that is, the location indicated by the latest location information of the target terminal 5 has been captured) with respect to the latest location information of the target terminal 5. Then, if there is no location information, etc. that satisfies the first location condition in the fixed terminal management table ( FIG. 27 ), it determines whether or not the latest location information, directional information, and angle of view of each mobile terminal 2 in the mobile terminal management table satisfy the first location condition with respect to the latest location information of the target terminal 5.

[0621] Then, the video acquisition unit 133 determines that the fixed terminal management table (Figure 27) does not contain location information that satisfies the first location condition, and therefore no videos from any of the fixed terminals 3 can be used.

[0622] Next, it is assumed that the video acquisition unit 133 has determined a first terminal (here, a mobile terminal 2 mounted on an automobile). The video acquisition unit 133 then receives a video from the first mobile terminal 2. Note that the video contains a child A. It is determined that the video satisfies the first location condition based on the location information of the target terminal 5 of child A, etc. The video is usually associated with the location information and the first rights holder identifier of the mobile terminal 2, which is the first terminal. It is also preferable that the received video is a video that was transmitted immediately after being shot by the mobile terminal 2.

[0623] Next, the video sending unit 141 sends the video received by the video acquiring unit 133 to the user terminal 4 of the guardian P who is identified by the user identifier paired with the subject identifier "T001".

[0624] Then, the video acquisition unit 133 temporarily stores the video received from the first terminal in a buffer (not shown) in a pair with the first rights holder identifier and location information of the first terminal.

[0625] In addition, the video acquisition unit 133 receives at least the video received from the first terminal while the location information of child A's target terminal 5 and the location information of the first terminal, etc., satisfy the first location condition, and the video transmission unit 141 transmits the video to parent P's user terminal 4.

[0626] Next, the video acquisition unit 133 determines that the location information of the target terminal 5 of child A and the location information of the first terminal, etc., no longer satisfy the first location condition (child A is no longer in the image), because the mobile terminal 2 has moved and is no longer in a position to capture an image of child A, who is walking home.

[0627] Next, the video acquisition unit 133 again tries to determine the location information of the second terminal that satisfies the first location condition with respect to the latest location information of the target terminal 5 by first referring to the fixed terminal management table (FIG. 27). Here, the video acquisition unit 133 uses the location information (FX １ , FY １ ) and direction information "Fd １ " and angle of view "FD １ " and determines that the area being photographed satisfies the first location condition for the latest location information of the target terminal 5.

[0628] Next, the video acquisition unit 133 transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier "U101" and receives the fixed video from the fixed terminal 3. Next, the video acquisition unit 133 pairs the video with the second rights holder identifier "U101" of the second terminal (the fixed terminal 3) and location information, and temporarily stores the video in a buffer (not shown).

[0629] Next, the video sending unit 141 sends the fixed video received by the video acquiring unit 133 from the second terminal to the user terminal 4 of the guardian P.

[0630] Then, as time passes (for example, 10 seconds), the video acquisition unit 133 determines that the location information of Child A's target terminal 5 and the location information, direction information, and angle of view of the second terminal no longer satisfy the first position condition (Child A is no longer visible).

[0631] Next, the video acquisition unit 133 again attempts to determine the location information of a third terminal whose latest location information satisfies the first location condition by referring to the fixed terminal management table in Figure 27, based on the latest location information of the target terminal 5, but it turns out that there is no location information of a third terminal (fixed terminal 3) that satisfies the first location condition.

[0632] 26 to determine the location information, etc. of a mobile terminal 2 whose latest location information satisfies the first location condition with respect to the latest location information of the target terminal 5, but there is no location information, etc. of a mobile terminal 2 that satisfies the first location condition. In such a case, the video sending unit 141 may send information to the user terminal 4 of the parent / guardian P that the video containing child A cannot be sent, or may stop sending videos and information to the user terminal 4.

[0633] By the above video transmission, the user terminal 4 sequentially receives and outputs the moving video acquired by the mobile terminal 2 and the fixed video acquired by the fixed terminal 3. This allows the guardian P to see how the child A returns home.

[0634] Then, as time passes, a third terminal, mobile terminal 2, approaches child A, and the video acquisition unit 133 is able to determine the location information, etc. of the third terminal that satisfies the first location condition with respect to the latest location information of the target terminal 5 by referring to Fig. 26. However, it is assumed that the video acquisition unit 133 was unable to detect the location information, etc. of the fixed terminal 3 that satisfies the first location condition with respect to the latest location information of the target terminal 5.

[0635] Next, the video acquisition unit 133 acquires the video from the third terminal, and then temporarily stores the video in a buffer (not shown) in association with the third rights holder identifier and location information of the third terminal.

[0636] Next, the video sending unit 141 sends the video received by the video acquisition unit 133 from the third terminal to the user terminal 4 of the guardian P, who is identified by the user identifier that pairs with the subject identifier "T001".

[0637] Next, the user terminal 4 receives and outputs the video acquired by the third terminal. As a result, parent P is able to see Child A's return home again, although parent P was unable to see Child A's return home for a short time.

[0638] The above process continues until the target terminal 5 of child A is powered off (until child A returns home), and parent B can monitor how child A returns home from school.

[0639] In addition, the reward means 1365 performs the above-mentioned reward processing for the rights holders identified by the respective rights holder identifiers of the first mobile terminal 2, the second terminal (fixed terminal 3), and the third mobile terminal 2 that provided the video to guardian B.

[0640] Furthermore, the moving image composing unit 134 combines the moving images transmitted from each mobile terminal 2 and fixed terminal 3 in the order in which they were transmitted to compose a combined moving image.

[0641] The rights holder processing unit 136 also stores the combined video in pairs with the rights holder identifier, which is the identifier of the guardian P.

[0642] The rights holder processing unit 136 also stores the combined moving image in association with a set of attribute values ​​associated with one or more moving images that are the basis of the combined moving image.

[0643] Next, the fourth security means 1364 acquires access information that specifies the storage destination of the combined video. The fourth security means 1364 also acquires a set of attribute values ​​corresponding to the stored combined video. Next, the fourth security means 1364 constructs security information having the acquired access information, the acquired set of attribute values, and the rights holder identifier of the video. Next, the fourth security means 1364 stores the constructed security information in the blockchain. An example of such security information is the record "ID=2" in Figure 28.

[0644] As described above, according to this specific example, the combined video can be used to monitor the subject who owns the target terminal 5. In addition, a reward can be given to the rights holder who provided the video that is the basis for the combined video used for monitoring. In addition, the combined video can be managed appropriately.

[0645] (Specific Example 2) It is assumed that a user inputs a destination into a user terminal 4 having a navigation function. Next, the user accepting unit 42 of the user terminal 4 accepts the destination. Furthermore, the user processing unit 43 acquires the current location. Next, the user processing unit 43 acquires route information from the current location to the destination. It is assumed here that the route information includes multiple pieces of location information.

[0646] Next, in response to the acquisition of the route information by the user processing unit 43 , the user transmitting unit 44 of the user terminal 4 automatically transmits an inquiry including the route information to the information processing device 1 .

[0647] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry. Next, the moving image acquiring unit 133 and the moving image composing unit 134 perform moving image combining processing as follows.

[0648] That is, first, the video acquisition unit 133 acquires, from among the multiple pieces of position information contained in the route information, position information (first position information) of the first intersection on the route specified by the route information. That is, it is preferable that the video acquisition unit 133 acquires a video using only a portion of the position information contained in the received route information. It is preferable that the video acquisition unit 133 acquires a video using only a portion of the position information contained in the received route information that satisfies a specific condition. Note that the specific condition is, for example, that the position information is information indicating the position of an intersection and that the distance from the previously used position information is equal to or greater than a threshold.

[0649] Next, the video acquisition unit 133 determines the position information, direction information, and angle of view of a first terminal that satisfies the first position condition based on the acquired position information. The first terminal is the fixed terminal 3 or the mobile terminal 2. Furthermore, the video acquisition unit 133 preferably searches the fixed terminal management table ( FIG. 27 ) according to the priority type “fixed terminal,” and if the position information, etc. of the first terminal cannot be determined, it searches the mobile terminal management table ( FIG. 26 ).

[0650] The video acquisition unit 133 then acquires the video from the determined first terminal, pairs the video with the first rights holder identifier of the first terminal, the first location information, etc., and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.

[0651] Next, the video acquisition unit 133 acquires second position information of the next intersection that is closer to the destination than the position indicated by the first position information of the intersection that has been previously acquired.

[0652] Next, the video acquisition unit 133 determines the location information of the second terminal that satisfies the first location condition for the second location information. The second terminal is the fixed terminal 3 or the mobile terminal 2.

[0653] The video acquisition unit 133 then acquires the video from the determined second terminal, pairs the video with the second rights holder identifier of the second terminal, second location information, etc., and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.

[0654] Next, the video acquisition unit 133 acquires third position information of the next intersection that is closer to the destination than the position indicated by the second position information.

[0655] Next, the video acquisition unit 133 determines the location information, etc. of a third terminal that satisfies the first location condition for the third location information. Then, the video acquisition unit 133 acquires the video from the determined third terminal, pairs the video with the third rights holder identifier of the third terminal and the third location information, etc., and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.

[0656] The information processing device 1 repeats the above process until transmitting a video corresponding to the nth location information that identifies the destination.

[0657] The user receiving unit 45 of the user terminal 4 sequentially receives the first video, the second video, the third video, ..., and the nth video. The user output unit 46 sequentially outputs the first video, the second video, the third video, ..., and the nth video.

[0658] Furthermore, the reward means 1365 of the information processing device 1 performs a reward process of rewarding the first right holder, the second right holder, ..., and the nth right holder identified by the first right holder identifier for providing the video.

[0659] Furthermore, the video composition unit 134 combines the videos transmitted from the multiple terminals in the order in which they were transmitted to compose a combined video, which is, for example, a video in which the mobile video transmitted from the mobile terminal 2 and the fixed video transmitted from the fixed terminal 3 are combined in time in the order in which they were received.

[0660] The rights holder processing unit 136 also stores the combined moving image in pair with a rights holder identifier, which is the identifier of the user of the user terminal 4. In other words, in this case, the rights holder of the combined moving image is the user.

[0661] The rights holder processing unit 136 also stores the combined moving image in association with a set of attribute values ​​associated with one or more moving images that are the basis of the combined moving image.

[0662] Next, fourth security means 1364 acquires access information that specifies the storage destination of the combined video. Fourth security means 1364 also acquires a set of attribute values ​​corresponding to the stored combined video. Next, fourth security means 1364 constructs security information that includes the acquired access information, the acquired set of attribute values, and the rights holder identifier of the video. Next, fourth security means 1364 stores the constructed security information.

[0663] As described above, according to this specific example, videos transmitted from multiple mobile terminals 2 are combined in time, at least in appearance, to form a combined video, which allows the user to check the route to the destination in order of proximity to the current location, thereby providing support for driving using a mobile object such as an automobile.

[0664] Furthermore, it is possible to give a reward to the rights holder who provided the original video that is the basis for the combined video, and it is also possible to appropriately store and manage the combined video.

[0665] (Specific Example 3) For example, assume that the navigation terminal 6 detects a traffic jam on the route to the destination. Note that the function of detecting a traffic jam is possible using publicly known technology. Next, assume that the navigation terminal 6 acquires route information having one or more pieces of location information identifying the congested road, and transmits an inquiry including the route information and a user identifier stored in the navigation terminal 6 to the information processing device 1. Note that the user identifier is an identifier, such as an IP address, of the user terminal 4 that receives the video transmitted from the fixed terminal 3 or mobile terminal 2 located at the location of the congested road. Furthermore, assume that the route information is information that identifies one or more congested roads. The user terminal 4 is, for example, a terminal of a passenger in the front passenger seat.

[0666] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry including the route information and the user identifier. Next, the video acquisition unit 133 and the video composition unit 134 perform video combination processing as follows.

[0667] First, the video acquisition unit 133 acquires the location information closest to the current location from among the location information included in the route information. Note that the location information closest to the current location is, for example, the first location information among the location information included in the route information.

[0668] Next, the video acquisition unit 133 acquires the last piece of position information that follows the position information and is within the threshold value of the adjacent position information. The video acquisition unit 133 sets this position information as the target position information to be used for video acquisition.

[0669] Next, the video acquisition unit 133 determines a first terminal having location information, etc., that satisfies a first location condition for the target location information. The first terminal is the fixed terminal 3 or the mobile terminal 2. Here, it is preferable that the video acquisition unit 133 searches from the fixed terminal 3.

[0670] Next, the video acquisition unit 133 acquires the video from the terminal, pairs the video with the first rights holder identifier of the first terminal, location information, etc., and temporarily stores the video in a buffer (not shown).

[0671] Then, the video acquisition unit 133 receives video from the first terminal and temporarily stores it in a buffer (not shown) until the location information, etc. of the first terminal no longer satisfies the first location condition.

[0672] Furthermore, the video sending unit 141 sequentially sends the acquired videos to the user terminal 4 .

[0673] Next, it is assumed that the mobile terminal 2 leaves the traffic jam and the video acquisition unit 133 determines that the location information etc. of the first terminal no longer satisfies the first location condition.

[0674] Next, the video acquisition unit 133 determines a second terminal having location information that satisfies the first location condition for the target location information. The first terminal is the fixed terminal 3 or the mobile terminal 2.

[0675] Next, the video acquisition unit 133 acquires the video from the second terminal, pairs the video with the second rights holder identifier of the second terminal and location information, etc., and temporarily stores the video in a buffer not shown.

[0676] Then, the video acquisition unit 133 receives video from the second mobile terminal 2 and temporarily stores it in a buffer (not shown) until the location information of the second terminal no longer satisfies the first location condition.

[0677] Furthermore, the video sending unit 141 sequentially sends the acquired videos to the user terminal 4 .

[0678] The user terminal 4 then receives the video from the information processing device 1 and outputs it.

[0679] By repeating the above process, the user of the user terminal 4 can continuously know the state of the traffic jam. In addition, the user can know the point where the traffic jam will be resolved.

[0680] Furthermore, when the route information includes information specifying two or more congested roads, the information processing device 1 performs the same process using the route information specifying the second and subsequent locations. As a result, the user terminal 4 can also receive and output videos for understanding the congestion situation at the second and subsequent locations. Note that when there are two or more congested locations, it is preferable that the user terminal 4 automatically or in response to a user instruction switch between the locations and receive and output the videos.

[0681] In this specific example, although explanation is omitted, reward processing, various security processing, etc. may also be performed.

[0682] As described above, according to this example, the state of traffic congestion on the route to the destination can be grasped using video transmitted from a plurality of terminals including the fixed terminal 3 and the mobile terminal 2 .

[0683] (Specific Example 4) Assume that a user terminal 4 installed in a vehicle that has entered a large parking lot transmits an inquiry including location information of the user terminal 4 to information processing device 1 in order to understand the status of the parking lot. The inquiry includes "type identifier = around-view image," location information of the user terminal 4 (reference location information), and an around-view image acquired by the user terminal 4.

[0684] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry. Next, the video acquisition unit 133 determines from the received inquiry that spatial fusion of multiple videos is necessary. Next, the video acquisition unit 133 and the video composition unit 134 perform video fusion processing as follows.

[0685] That is, the video acquisition unit 133 acquires the reference location information contained in the received inquiry. Next, the video acquisition unit 133 determines one or more mobile terminals 2 corresponding to location information that satisfies a second location condition with respect to the reference location information. Next, the video acquisition unit 133 transmits an instruction to one or more terminals to transmit current around-view images. Note that each of the multiple terminals is a fixed terminal 3 or a mobile terminal 2. Also, the second location condition here is that the location indicated by the location information is located in an area of ​​the parking lot that includes the reference location information. Note that it is assumed that each of the one or more mobile terminals 2 is continuously acquiring around-view images. It is also assumed that one or more fixed terminals 3 are installed on the ceiling of the parking lot, capture an image directly below, and acquire around-view images.

[0686] Then, the video acquisition unit 133 receives, from each of one or more terminals, around-view images associated with the position information of the terminal.

[0687] Next, the moving image composer 134 calculates the difference (distance) between the position information paired with each received around-view image and the reference position information, and then sorts the around-view images in ascending order using the difference as a key.

[0688] Next, the moving image composer 134 acquires a reference image, which is the around-view image included in the query.

[0689] Next, the moving image composition unit 134 performs the processing described using the flowchart in FIG. 11 , and arranges each of the around-view images received from the other terminals in an appropriate position relative to the reference image in the sorted order using the position information, thereby composing a fusion image.

[0690] Furthermore, the video composition unit 134 composes an image that clearly indicates the position indicated by the reference position information (the position of the user terminal 4 that sent the query) in the finally constructed fusion image. Note that such an image is also a fusion image.

[0691] Next, the video sending unit 141 sends the constructed fusion image to the user terminal 4 .

[0692] Next, the user terminal 4 receives and outputs the fused image.

[0693] Furthermore, the information processing device 1 repeats the above process, and as a result, transmits a fusion moving image including a plurality of fusion images to the user terminal 4. Then, the user terminal 4 receives and outputs the fusion moving image.

[0694] As described above, according to this example, images transmitted from one or more other mobile terminals 2 or fixed terminals 3 can be fused to assist in finding an empty space in a parking lot.

[0695] In this specific example, the information processing device 1 fuses around-view images. However, the information processing device 1 may fuse multiple normal camera images or multiple images acquired by an omnidirectional camera using position information corresponding to each image to construct a fused image and transmit the fused image to the user terminal 4. Furthermore, the information processing device 1 may fuse images from a fixed terminal 3 installed at an event venue such as a live performance with images from user terminals 4 held by audience members gathered at the event venue, and the images of areas that could not be captured by the user terminals 4 held by the audience members may be supplemented with the images from the fixed terminals 3.

[0696] (Specific Example 5) It is assumed that user B drives a driving course on a rainy day, and route information specifying the route that user B drove by car is stored in the user terminal 4.

[0697] Next, assume that after returning home, user B inputs a query including the route information stored in user terminal 4 and environmental information "weather information = sunny" into user terminal 4. Then, user terminal 4 accepts the query and transmits the query to information processing device 1. Note that this query is a query to obtain a video of the driving course traveled by the user on a sunny day.

[0698] Next, the inquiry receiving unit 122 of the information processing device 1 receives the inquiry. Next, the video acquisition unit 133 determines that the received inquiry is not a real-time video search. Next, the video acquisition unit 133 and the like perform a registered video search process as follows.

[0699] That is, the video acquisition unit 133 acquires the first location information contained in the route information in the received inquiry.

[0700] Next, the video acquisition unit 133 acquires all the security information from the registered video management table (FIG. 28). That is, the video acquisition unit 133 acquires, for example, the security information that exists on the blockchain.

[0701] Next, the video acquisition unit 133 determines, from the acquired security information, security information that has a provision flag of "1," has location information that satisfies the first location condition for the acquired location information, and includes "weather information = sunny." Here, it is assumed that the video acquisition unit 133 has determined the security information of "ID = 1" in FIG. 28, for example.

[0702] Next, the video acquisition unit 133 acquires the access information "Address 01" that pairs with the determined security information. Next, the video acquisition unit 133 acquires a video using the access information "Address 01." Next, the video acquisition unit 133 cuts out from the video corresponding to the access information "Address 01" a video up to the point where the second or subsequent location information included in the route information included in the received inquiry satisfies (is the same as or close to) the first location condition as the location information, etc., of the security information. In other words, the video acquisition unit 133 cuts out and acquires from the video corresponding to the access information "Address 01" a video up to the point where the user deviates from the route of the driving course traveled by the user.

[0703] Next, the video acquisition unit 133 acquires second location information, which is location information when the video corresponding to the access information "address 01" deviates from the driving course and is contained in the route information contained in the received inquiry.

[0704] Then, the video acquisition unit 133 determines, from the acquired maintenance information, maintenance information that includes a provision flag of "1", has location information that satisfies the first location condition for the acquired second location information, and includes "weather information = sunny".

[0705] Next, the video acquisition unit 133 acquires access information (for example, "address X") that pairs with the determined security information. Next, it is assumed that the video acquisition unit 133 acquires a second video using the access information "address X." Next, the video acquisition unit 133 cuts out from the second video corresponding to the access information "address X" the video up to the point where the location information beyond the second location information contained in the route information contained in the received inquiry satisfies the location information of the security information. In other words, the video acquisition unit 133 cuts out and acquires from the video corresponding to the access information "address X" the video up to the point where the user deviates from the route of the driving course traveled by the user.

[0706] The video acquisition unit 133 performs the above process until it uses the last location information included in the route information in the received inquiry.

[0707] Next, the video composition unit 134 combines the multiple videos acquired by the video acquisition unit 133 in the order in which they were acquired (in the order of the route) to compose a combined video. Note that it is preferable that the multiple videos that form the basis of the combined video include a fixed video and a moving video.

[0708] Next, the video sending unit 141 sends the combined video to the user terminal 4 of user B.

[0709] Next, the user terminal 4 receives and outputs the combined video.

[0710] In addition, the reward means 1365 of the information processing device 1 acquires the rights holder identifiers (e.g., "U001") of each of the original multiple videos used in the combined video, and performs reward processing to reward the rights holders identified by each of the multiple rights holder identifiers.

[0711] The rights holder processing unit 136 also performs the various preservation processes described above on the combined moving image.

[0712] As described above, according to this specific example, a combined video can be constructed by combining multiple videos taken on a route in the past by using route information of the route traveled by the user, and can be output.

[0713] As described above, according to this embodiment, a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be used to create and provide a useful video.

[0714] Furthermore, according to this embodiment, a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be temporally combined to form and provide a single useful video.

[0715] Furthermore, according to this embodiment, a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be spatially merged to form and provide a single useful video.

[0716] Furthermore, according to this embodiment, it is possible to select an appropriate video from the videos taken by the mobile terminal 2 and the videos taken by the fixed terminal 3, and to compose and provide a useful video.

[0717] Furthermore, according to this embodiment, a useful stereoscopic three-dimensional video can be constructed and provided by using the video captured by the mobile terminal 2 and the video captured by the fixed terminal 3 .

[0718] Furthermore, according to this embodiment, moving videos and fixed videos based on the intention of the user of the mobile terminal 2 can be used to create and provide a useful single video.

[0719] According to this embodiment, the camera used to capture the video may be an omnidirectional camera or the like, and the type of camera is not important.

[0720] The processing in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may also be recorded on a recording medium such as a CD-ROM and distributed. This also applies to the other embodiments in this specification. The software that realizes information system A in this embodiment is the following program. In other words, this program causes a computer to execute a video acquisition step of acquiring a mobile video, which is a video taken by a mobile terminal and corresponds to an attribute value set including one or more pieces of environmental information including location information that identifies the shooting location or time information that identifies the time of shooting, and transmitted from the mobile terminal, and acquiring a fixed video, which is a video taken by a fixed camera whose shooting location is fixed and corresponds to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera; a video composition step of temporally combining the mobile video and the fixed video to compose a single video, or compose a single video having two or more frames by spatially fusing at least some of the frames of the mobile video and at least some of the frames of the fixed video; and a video transmission step of transmitting the single video composed in the video composition step.

[0721] 29 shows the appearance of a computer that executes the programs described herein to realize the information processing device 1, user terminal 4, target terminal 5, navigation terminal 6, etc., of the various embodiments described above. The above-described embodiments can be realized by computer hardware and computer programs executed thereon. FIG. 29 is an overview of this computer system 300, and FIG. 30 is a block diagram of the system 300. The mobile terminal 2 and fixed terminal 3 are equipped with cameras.

[0722] In FIG. 29, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0723] 30, the computer 301 includes, in addition to a CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012 etc., a ROM 3015 for storing programs such as a boot-up program, a RAM 3016 connected to the MPU 3013 for temporarily storing instructions for application programs and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card for providing connection to a LAN.

[0724] A program that causes computer system 300 to execute the functions of information processing device 1 and the like of the above-described embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 when executed. The program may also be loaded directly from CD-ROM 3101 or the network.

[0725] The program does not necessarily have to include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the information processing device 1 of the above-described embodiment. The program only needs to include instructions that call appropriate functions (modules) in a controlled manner and achieve the desired results. How the computer system 300 operates is well known, and a detailed description thereof will be omitted.

[0726] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0727] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0728] Furthermore, in each of the above embodiments, it goes without saying that a plurality of communication means present in one device may be physically realized by one medium.

[0729] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0730] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention.

[0731] As described above, the information processing device 1 of the present invention has the effect of being able to use videos taken by a mobile terminal and videos taken by a fixed camera to create and provide a useful single video, and is useful as a server that provides videos, etc.

Claims

1. an acquisition unit that acquires mobile images, which are images captured by a mobile terminal, corresponding to an attribute value set including one or more pieces of environmental information including location information specifying a location of the image capture or time information specifying a time of the image capture, and transmitted from the mobile terminal, and acquires fixed images, which are images captured by a fixed camera having a fixed location of the image capture, corresponding to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera; a configuration unit that configures a single image by combining the moving image and the fixed image in time, or that configures a single image having a plurality of frames by spatially fusing at least a portion of a frame of the moving image and at least a portion of a frame of the fixed image; a transmission unit that transmits the one image constructed by the construction unit, The acquisition unit is An information processing device acquires an image corresponding to a priority type indicating a priority type of image among the fixed images and the moving images, when there are a plurality of images that satisfy adoption conditions, which are conditions for the original images that constitute the one image, and when there are fixed images and moving images corresponding to position information indicating positions close enough to satisfy a first position condition, the image corresponding to the priority type indicating a priority type of image among the fixed images and the moving images is acquired.

2. The present invention further includes a receiving unit for receiving the moving image from the mobile terminal and for receiving the fixed image from a fixed terminal equipped with the fixed camera, The acquisition unit is acquiring the moving image and the fixed image received by the receiving unit; The receiving unit is 2. The information processing apparatus according to claim 1, further comprising: a function for receiving the moved image from the mobile terminal that has accepted a user's permission in accordance with a usage mode flag for the image stored in the mobile terminal.

3. The usage state flag is information indicating whether or not the mobile image transmitted from the mobile terminal has a "non-provisional use request," The receiving unit is An information processing device as described in claim 2, wherein when the usage mode flag stored in the mobile terminal is information indicating "there is a desire for non-temporary use", the mobile image is received from the mobile terminal only if the mobile terminal accepts the user's permission.

4. The moving image or the fixed image is associated with a rights holder identifier that identifies a rights holder of the moving image or the fixed image, 2. The information processing apparatus according to claim 1, further comprising a right holder processing unit for performing right holder processing relating to a right holder identified by a right holder identifier corresponding to the one image constructed by the construction unit.

5. An information processing method realized by an acquisition unit, a configuration unit, and a transmission unit, comprising: an acquisition step in which the acquisition unit acquires mobile images, which are images taken by a mobile terminal, corresponding to an attribute value set including one or more pieces of environmental information including location information specifying a location of the image or time information specifying a time of the image, and transmitted from the mobile terminal, and acquires fixed images, which are images taken by a fixed camera having a fixed shooting location, corresponding to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera; a constructing step in which the constructing unit constructs one image by combining the moving image and the fixed image in time, or constructs one image having a plurality of frames by spatially fusing at least a portion of a frame of the moving image and at least a portion of a frame of the fixed image; a transmission step in which the transmission unit transmits the one image constructed in the image construction step, The acquiring step includes: An information processing method in which, when there are multiple images that satisfy adoption conditions, which are conditions for the original images that constitute the one image, and when there are fixed images and moving images whose position information corresponds to position information indicating positions close enough to satisfy a first position condition, an image corresponding to a priority type indicating a priority type of image among the fixed images and the moving images is obtained.