Information processor, information processing method, and program

Abstract

To make it possible to diminish resources of a portable terminal needed to assist a user in avoiding an object such as an obstacle existing in a route along which the user moves.SOLUTION: An information processor 1 in accordance with an embodiment of the present invention includes an acquisition unit 131 that acquires position information representing the position of a first portable terminal owned by a user and route information representing a route along which the user is scheduled to move, a determination unit 133 that determines based on a trajectory of a position of a second portable terminal owned by another user different from the user whether there is a possibility that a predetermined object may exist in the route, and on the condition that the determination unit has determined that an object may be present on the route, a transmitting unit 132 that transmits initiation information, which initiates at least one of that an imaging unit associated with the first portable terminal takes an image and that a shot image taken by the imaging unit is transmitted to a predetermined transmission destination, to the first portable terminal.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program for performing processing related to imaging using a mobile terminal owned by a user. [Background technology]

[0002] Patent document 1 describes a navigation system for helping users, such as visually impaired people, reach their destination, in which images of the surroundings are transmitted from the user's mobile terminal and advice can be given to the user by others based on the transmitted images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-117094 A Summary of the Invention [Problem to be solved by the invention]

[0004] When an object such as an obstacle is present on the path along which the user is moving, it is desirable to give the user an instruction to avoid the object. In the system described in Patent Document 1, in order to give the user an instruction to avoid the object based on the image, the mobile terminal must constantly capture and transmit images, which causes a problem in that resources such as the battery capacity and communication capacity of the mobile terminal are consumed in large amounts.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to make it possible to reduce the resources of a mobile terminal required to assist a user in avoiding objects such as obstacles that exist on the route the user is traveling. [Means for solving the problem]

[0006] An information processing device of a first aspect of the present invention includes an acquisition unit that acquires location information indicating a location of a first portable terminal held by a user and route information indicating a route along which the user plans to travel, a determination unit that determines whether or not a predetermined object is likely to exist on the route based on a trajectory of the location of a second portable terminal held by another user different from the user, and a transmission unit that transmits to the first portable terminal start information to cause an imaging unit associated with the first portable terminal to start at least one of capturing an image and transmitting an image captured by the imaging unit to a predetermined destination, on condition that the determination unit has determined that the object is likely to exist on the route.

[0007] The transmission unit may transmit the start information on a condition that the determination unit determines that there is a possibility that the object is present on the path and the distance between the position of the first mobile terminal and the position of the object becomes less than or equal to a threshold.

[0008] After transmitting the start information to the first portable terminal, the transmission unit may transmit end information to the first portable terminal to cause the imaging unit to terminate at least one of capturing an image and transmitting an image captured by the imaging unit to the destination, on the condition that a distance between a position of the first portable terminal and a position of the object becomes greater than the threshold value.

[0009] The determination unit may determine whether or not the object is captured in the captured image, and the transmission unit may transmit information to the first portable terminal informing the user that the object is outside the imaging range of the imaging unit, on condition that the determination unit determines that the object is not captured in the captured image.

[0010] The determination unit may determine whether or not the object is captured in the captured image, and the transmission unit may transmit, to the first portable terminal, control information for controlling the imaging unit to expand an imaging range of the imaging unit, on condition that the determination unit determines that the object is not captured in the captured image.

[0011] The determination unit may determine whether or not the object is captured in the captured image, and the transmission unit may transmit, to the first portable terminal, control information for controlling the imaging unit to change the direction of imaging by the imaging unit, on condition that the determination unit determines that the object is not captured in the captured image.

[0012] The determination unit may determine whether the first portable terminal, which has received the start information, has been able to start at least one of capturing images using the imaging unit and transmitting the captured image, and the transmission unit may transmit information to the first portable terminal notifying the user that the captured image cannot be obtained, on condition that the determination unit determines that the first portable terminal has not been able to start at least one of capturing images using the imaging unit and transmitting the captured image.

[0013] When the determination unit determines that there is a possibility that the object is present on the path, and the number or density of the multiple trajectories within a specified range based on the object is equal to or greater than a reference value, the transmission unit may not need to transmit the start information.

[0014] The determination unit may determine whether the object determined to possibly exist on the path is a fixed object fixed on the path, and if the determination unit determines that the object is the fixed object at a first point in time, it may determine that the object may exist on the path at a second point in time after the first point in time without based on the trajectory.

[0015] When the determination unit determines that the object is the fixed object at the first time point, the transmission unit may transmit information to the first mobile terminal notifying the user who plans to travel the route at a second time point after the first time point that the object may be present on the route.

[0016] The determination unit may determine whether or not the object is the fixed object based on the captured image captured by the imaging unit associated with the first portable terminal.

[0017] The determination unit may determine whether or not the object is the fixed object based on a captured image captured by an imaging unit associated with a portable terminal different from the first portable terminal.

[0018] An information processing method of a second aspect of the present invention includes the steps of: acquiring, executed by a processor, location information indicating a location of a first portable terminal held by a user and route information indicating a route along which the user plans to travel; determining whether or not a predetermined object is likely to exist on the route based on a trajectory of the location of a second portable terminal held by another user different from the user; and, on condition that it has been determined in the step of determining that the object is likely to exist on the route, transmitting to the first portable terminal start information for causing an imaging unit associated with the first portable terminal to start at least one of capturing an image and transmitting an image captured by the imaging unit to a predetermined destination.

[0019] A third aspect of the program of the present invention causes a processor to execute the steps of: acquiring location information indicating the location of a first portable terminal held by a user and route information indicating a route along which the user plans to travel; determining whether or not a predetermined object is likely to exist on the route based on a trajectory of the location of a second portable terminal held by another user other than the user; and, on condition that it has been determined in the step of determining that the object is likely to exist on the route, initiating at least one of capturing an image with an imaging unit associated with the first portable terminal and transmitting the captured image captured by the imaging unit to a predetermined destination. Effect of the Invention

[0020] According to the present invention, it is possible to reduce the resources of a mobile terminal required to assist a user in avoiding objects such as obstacles that exist on a route along which the user is traveling. [Brief description of the drawings]

[0021] [Figure 1] 1 is a schematic diagram of an information processing system according to an embodiment. [Diagram 2] 1 is a block diagram of an information processing system according to an embodiment. [Diagram 3] 4 is a schematic diagram for explaining a method in which an acquisition unit acquires information; FIG. [Figure 4] 5 is a schematic diagram for explaining a method in which a determination unit determines whether or not an object is likely to exist on a path. FIG. [Diagram 5] 11 is a schematic diagram for explaining start information transmitted by a transmission unit; FIG. [Figure 6] 10 is a schematic diagram for explaining notification information and control information transmitted by a transmission unit. FIG. [Figure 7] 11 is a schematic diagram for explaining information indicating the position of an object determined to be a fixed object. FIG. [Figure 8] FIG. 1 is a flowchart illustrating an exemplary information processing method executed by an information processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] First Embodiment [Outline of Information Processing System S] 1 is a schematic diagram of an information processing system S according to this embodiment. The information processing system S includes an information processing device 1, a user terminal 2, another user's terminal 3, and a monitoring user terminal 4. The information processing system S may include a plurality of user terminals 2, a plurality of other user's terminals 3, and a plurality of monitoring user terminals 4. The information processing system S may also include other devices such as servers and terminals.

[0023] The information processing device 1 is a computer that performs processing related to imaging by the user terminal 2. When a predetermined condition is satisfied, the information processing device 1 starts at least one of imaging by an imaging unit of the user terminal 2 and transmitting the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination.

[0024] The user terminal 2, the other user terminal 3, and the monitoring user terminal 4 are computers such as smartphones or tablet terminals. The user terminal 2, the other user terminal 3, and the monitoring user terminal 4 have an operation unit such as a touch panel or a keyboard for receiving operations, and a display unit such as a liquid crystal display for displaying information. The user terminal 2, the other user terminal 3, and the monitoring user terminal 4 are capable of communicating with the information processing device 1 via a network.

[0025] The user terminal 2 (first mobile terminal) is a mobile terminal used by a user. The user is a person traveling along a route R toward a predetermined destination. The user is, for example, a visually impaired person who travels while receiving remote support from a monitoring user. A visually impaired user is a person who has no eyesight or a person with weak eyesight. The user is not limited to a visually impaired person, and may be any other person who travels while receiving remote support from a monitoring user.

[0026] The user terminal 2 has an imaging unit that generates a captured image by capturing an image of the user's surroundings, and a measurement unit that measures the position of the user terminal 2 by GNSS (Global Navigation Satellite System) or the like. The imaging unit is an imaging unit that is associated in advance with the user terminal 2, and is, for example, an imaging element built into the user terminal 2, or an imaging device connected to the user terminal 2 by short-range wireless communication or the like. The user terminal 2 transmits the captured image and position information indicating the measured position to the information processing device 1 and the monitoring user terminal 4.

[0027] The other user terminal 3 (second mobile terminal) is a mobile terminal used by another user different from the user. The other user is a person who has previously traveled through an area including the route R along which the user plans to travel. The other user terminal 3 has a measurement unit that measures the position of the other user terminal 3 by GNSS or the like. The other user terminal 3 records a trajectory T of the position of the other user terminal 3 measured by the measurement unit, and transmits trajectory information indicating the recorded trajectory T to the information processing device 1.

[0028] The monitoring user terminal 4 (third mobile terminal) is a mobile terminal used by a monitoring user different from the user. The monitoring user is a person who remotely supports the user's movement. For example, the monitoring user visually checks the captured image displayed on the monitoring user terminal 4, and gives the user instructions to avoid an object O, such as an obstacle, that exists on the route R.

[0029] An overview of the process executed by the information processing device 1 according to this embodiment will be described below. The information processing device 1 acquires location information indicating the location of the user terminal 2 and route information indicating the route R along which the user plans to travel.

[0030] The information processing device 1 determines whether or not there is a possibility that a predetermined object O, such as an obstacle, exists on the route R, based on the trajectory T of the positions of the other user terminals 3. For example, when the trajectory T of the positions of multiple other user terminals 3 indicates a shape that avoids something on the route R, the information processing device 1 determines that an object O exists.

[0031] On the condition that it is determined that an object O may exist on the path R, the information processing device 1 transmits to the user terminal 2 start information for causing the imaging unit of the user terminal 2 to start at least one of capturing an image and transmitting the captured image captured by the imaging unit of the user terminal 2 in accordance with the start information received from the information processing device 1. The user terminal 2 captures an image using the imaging unit and transmits the generated captured image to the monitoring user terminal 4 in accordance with the start information received from the information processing device 1.

[0032] The monitoring user terminal 4 displays on a display unit the captured image received from the user terminal 2. Thereafter, the monitoring user visually recognizes the captured image displayed on the monitoring user terminal 4, and causes the monitoring user terminal 4 to transmit an instruction to the user terminal 2 to avoid the object O.

[0033] In this way, the information processing device 1 causes the user terminal 2 to start at least one of capturing images by the imaging unit and transmitting the captured image on the condition that it has determined that there is a possibility that an object O exists on the route R based on the trajectory T of the positions of the other user terminals 3. This eliminates the need for the user terminal 2 to constantly capture images by the imaging unit and transmit the captured images, and therefore the information processing device 1 can reduce the resources of the user terminal 2 required to support the user in avoiding the object O existing on the route R along which the user moves.

[0034] [Configuration of Information Processing System S] FIG. 2 is a block diagram of an information processing system S according to this embodiment. In FIG. 2, arrows indicate main data flows, and data flows other than those shown in FIG. 2 may exist. In FIG. 2, each block indicates a configuration in functional units, not a configuration in hardware (device) units. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between blocks via any means, such as a data bus, a network, or a portable storage medium.

[0035] The information processing device 1 includes a communication unit 11, a storage unit 12, and a control unit 13. The information processing device 1 may be configured by connecting two or more physically separate devices by wire or wirelessly. The information processing device 1 may also be configured by a cloud that is a collection of computer resources.

[0036] The communication unit 11 has a communication controller for transmitting and receiving data between the user terminal 2, the other user terminal 3, and the monitoring user terminal 4 via the network. The communication unit 11 notifies the control unit 13 of data received via the network from the user terminal 2, the other user terminal 3, or the monitoring user terminal 4. In addition, the communication unit 11 transmits data output from the control unit 13 to the user terminal 2, the other user terminal 3, or the monitoring user terminal 4 via the network.

[0037] The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, etc. The storage unit 12 stores in advance a program to be executed by the control unit 13. The storage unit 12 also stores trajectory information indicating a trajectory T of positions to which each of the multiple other user terminals 3 has moved in the past. The storage unit 12 may be provided outside the information processing device 1, in which case data may be exchanged between the storage unit 12 and the control unit 13 via a network.

[0038] The control unit 13 has an acquisition unit 131, a transmission unit 132, a determination unit 133, and a generation unit 134. The control unit 13 is a processor such as a CPU (Central Processing Unit), and functions as the acquisition unit 131, the transmission unit 132, the determination unit 133, and the generation unit 134 by executing a program stored in the storage unit 12.

[0039] The following is a detailed description of the processes executed by the information processing device 1. The acquisition unit 131 acquires various types of information required for performing processes relating to imaging by the user terminal 2 owned by the user.

[0040] 3(a) is a schematic diagram for explaining a method in which the acquisition unit 131 acquires destination information and position information. When a user starts moving toward a destination, the user inputs the destination in the user terminal 2 held by the user. The user inputs, for example, a voice or characters indicating the destination to the user terminal 2. The user terminal 2 transmits the destination information indicating the destination input by the user to the information processing device 1. In the information processing device 1, the acquisition unit 131 acquires the destination information transmitted by the user terminal 2.

[0041] The user terminal 2 transmits location information indicating the location of the user terminal 2 measured by the measurement unit to the information processing device 1. The user terminal 2 transmits location information indicating the current location (latest location) measured by the measurement unit at a predetermined time interval. In the information processing device 1, the acquisition unit 131 acquires the location information transmitted by the user terminal 2.

[0042] When the acquisition unit 131 cannot acquire the position information, or when the accuracy of the position information acquired by the acquisition unit 131 is lower than a predetermined reference value, the transmission unit 132 may transmit alert information indicating that highly accurate position information cannot be acquired to the monitoring user terminal 4, without transmitting start information described below to the user terminal 2. The monitoring user terminal 4 displays the alert information received from the information processing device 1 on the display unit. This allows the information processing device 1 to alert the monitoring user when it cannot acquire position information with sufficient accuracy to determine the presence or absence of an object O.

[0043] 3(b) is a schematic diagram for explaining a method for acquiring route information and reference point information by the acquisition unit 131. When the user starts moving toward the destination, the acquisition unit 131 acquires route information indicating a route R for moving from the position of the user terminal 2 indicated by the position information to the destination indicated by the destination information.

[0044] The acquisition unit 131, for example, determines a route R by executing a known route search process on a map pre-stored in the storage unit 12, and acquires route information indicating the determined route R. In addition, the acquisition unit 131 may, for example, transmit position information and destination information to a device different from the information processing device 1, and receive route information indicating the route R for moving from the position of the user terminal 2 to the destination from the device.

[0045] The acquisition unit 131 acquires reference point information indicating one or more reference points located on the route R from the position of the user terminal 2 indicated by the position information to the destination indicated by the destination information. The storage unit 12 stores reference point information indicating multiple reference points on a map in advance. The multiple reference points are points for partitioning an area in which the determination unit 133 described below determines the presence or absence of an object O. The multiple reference points include, for example, road intersections, corners, points arranged at predetermined intervals (e.g., 50 meters), and the like. The acquisition unit 131 acquires, for example, from the storage unit 12, reference point information indicating one or more reference points on the route R from the position of the user terminal 2 to the destination.

[0046] The acquisition unit 131 acquires trajectory information indicating a trajectory T of past positions of the other user terminals 3 owned by each of the multiple other users. The storage unit 12 pre-stores trajectory information transmitted by each of the multiple other user terminals 3. The acquisition unit 131 acquires, for example, one or more pieces of trajectory information from the multiple pieces of trajectory information stored in the storage unit 12, which indicate a trajectory T that passed through a range between the position of the user terminal 2 and the next reference point on the route R (the reference point closest in the direction in which the user terminal 2 moves).

[0047] When the number of pieces of trajectory information acquired by the acquisition unit 131 is equal to or less than a predetermined first reference value, the transmission unit 132 may transmit alert information indicating that the number of pieces of trajectory information is insufficient to the monitoring user terminal 4, without transmitting start information described below to the user terminal 2. The first reference value is pre-stored in the storage unit 12. The monitoring user terminal 4 displays the alert information received from the information processing device 1 on the display unit. When the number of pieces of trajectory information is insufficient, there is a possibility that the presence or absence of an object O cannot be correctly determined based on the trajectory information of other users because the other users did not happen to approach the object O. When the information processing device 1 cannot acquire a sufficient number of pieces of trajectory information to determine the presence or absence of the object O, it can alert the monitoring user.

[0048] As another method, when the number of pieces of trajectory information acquired by the acquisition unit 131 is equal to or less than a first reference value, the transmission unit 132 may transmit start information, described later, to the user terminal 2, regardless of the determination result of the determination unit 133, described later, as to whether or not there is a possibility of an object O being present. In this way, when the number of pieces of trajectory information between specific reference points is small, the information processing device 1 can cause the user terminal 2 to start at least one of capturing images and transmitting the captured images, regardless of the presence or absence of an object O.

[0049] When the number of pieces of trajectory information acquired by the acquisition unit 131 is equal to or greater than a predetermined second reference value, the transmission unit 132 may transmit alert information indicating that the number of pieces of trajectory information is excessive to the monitoring user terminal 4 without transmitting start information described below to the user terminal 2. The second reference value is pre-stored in the storage unit 12. The monitoring user terminal 4 displays the alert information received from the information processing device 1 on the display unit. Since a place where the number of pieces of trajectory information is excessive is a crowded place, there is a possibility that the presence or absence of an object O cannot be correctly determined based on the trajectory information of other users due to the occurrence of many avoidance actions against other people rather than the object O. When the information processing device 1 cannot determine the presence or absence of an object O in a crowded place, it can alert the monitoring user.

[0050] When the number of pieces of trajectory information acquired by the acquisition unit 131 is greater than the above-mentioned first reference value, the determination unit 133 determines whether or not there is a possibility that a predetermined object O, such as an obstacle, exists on the route R indicated by the route information acquired by the acquisition unit 131, based on the trajectory T of the positions of the other user terminals 3 indicated by the trajectory information acquired by the acquisition unit 131. The determination unit 133 performs the determination for the range between the current position of the user terminal 2 and the next reference point on the route R, for example.

[0051] 4(a) and 4(b) are schematic diagrams for explaining a method in which the determination unit 133 determines whether or not there is a possibility that an object O exists on the route R. The determination unit 133 calculates a value T″ of a second-order differential with respect to the shape of the trajectory T at each position of the trajectory T that passes through the range between the position of the user terminal 2 and the next reference point on the route R, which is indicated by one or more pieces of trajectory information acquired by the acquisition unit 131, for example. The value T″ takes a large value when the trajectory T changes suddenly (for example, when another user avoids the object O) and takes a small value when the trajectory T does not change suddenly (for example, when another user walks straight).

[0052] When the value T" of another user at any position is equal to or greater than the first threshold, the judgment unit 133 judges that the other user is an avoidable user who has taken avoidant action (Figure 4(a)). On the other hand, when the value T" of another user at any position is less than the first threshold, the judgment unit 133 judges that the other user is not an avoidable user who has taken avoidant action (Figure 4(b)). The first threshold for the value T" is stored in advance in the memory unit 12.

[0053] Next, the determination unit 133 calculates an avoidance ratio, which is the ratio of the number of other users determined to be avoided users to the number of all other users corresponding to the trajectory T indicated by one or more pieces of trajectory information acquired by the acquisition unit 131. When the avoidance ratio is equal to or greater than the second threshold, the determination unit 133 determines that there is a possibility that an object O exists on the route R. On the other hand, when the avoidance ratio is less than the second threshold, the determination unit 133 determines that there is no possibility that an object O exists on the route R. The second threshold of the avoidance ratio is stored in the storage unit 12 in advance.

[0054] This allows the information processing device 1 to determine whether or not there is a possibility that an object O is present on the route R based on the trajectories T moved by multiple other users, without incurring the cost of building a database that records the positions of objects O, such as obstacles, on a map.

[0055] The generation unit 134 generates start information for starting at least one of the following: capturing an image by the imaging unit of the user terminal 2 and transmitting the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination, on the condition that the determination unit 133 has determined that there is a possibility that an object O is present on the path R.

[0056] The transmission unit 132 transmits the start information generated by the generation unit 134 to the user terminal 2. In accordance with the start information received from the information processing device 1, the user terminal 2 starts at least one of capturing an image with the imaging unit of the user terminal 2 and transmitting the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination.

[0057] On the other hand, on condition that the determination unit 133 determines that there is no possibility that an object O exists on the route R, the transmission unit 132 may transmit information indicating that there is no possibility that an object O exists on the route R to the monitoring user terminal 4 without transmitting start information to the user terminal 2.

[0058] 5(a) and 5(b) are schematic diagrams for explaining start information transmitted by the transmission unit 132. In the example of FIG. 5(a), the start information is information for starting imaging by the imaging unit of the user terminal 2. In this case, the user terminal 2 does not perform imaging by the imaging unit before receiving the start information, but starts imaging by the imaging unit on the condition that the start information is received. This eliminates the need for the user terminal 2 to constantly perform imaging by the imaging unit, thereby reducing consumption of battery capacity.

[0059] In the example of Fig. 5(b), the start information is information for starting the transmission of captured images captured by the imaging unit of the user terminal 2 to a predetermined destination. The destinations of the captured images are, for example, the information processing device 1 and the monitoring user terminal 4. In this case, the user terminal 2 does not transmit the captured images before receiving the start information, but starts transmitting the captured images on the condition that the start information is received. This eliminates the need for the user terminal 2 to constantly transmit captured images, thereby reducing the consumption of communication capacity.

[0060] The start information may be information that causes a user terminal 2 that is not performing both image capture by the imaging unit and transmission of the captured image to start both image capture by the imaging unit and transmission of the captured image. The start information may be information that causes a user terminal 2 that is always capturing images by the imaging unit to start only transmission of captured images. The start information may be information that causes a user terminal 2 that is always transmitting data to the information processing device 1 and the monitoring user terminal 4 to start only image capture by the imaging unit.

[0061] The transmission unit 132 may transmit the start information on the condition that the determination unit 133 determines that there is a possibility that an object O exists on the route R and the distance between the position of the user terminal 2 and the position of the object O is equal to or less than a predetermined threshold (10 meters, etc.). The distance threshold is stored in advance in the storage unit 12. The position of the object O is, for example, a position where the above-mentioned value T'' for any other user is equal to or greater than the first threshold. This allows the information processing device 1 to automatically start at least one of imaging by the imaging unit of the user terminal 2 and transmission of the captured image on the condition that the user approaches a position where there is a possibility that the object O exists, thereby reducing resources required for processing related to imaging in the user terminal 2.

[0062] The monitoring user terminal 4 receives the captured image transmitted by the user terminal 2 and displays the received captured image on a display unit. The monitoring user judges whether or not an object O actually exists on the route R along which the user is traveling by visually checking the captured image displayed on the monitoring user terminal 4. When the monitoring user judges that an object O exists, he or she causes the monitoring user terminal 4 to transmit an instruction to the user terminal 2 to avoid the object O.

[0063] The monitoring user may, for example, give the user an instruction to avoid the object O through a voice call between the monitoring user terminal 4 and the user terminal 2. The monitoring user may, for example, use the monitoring user terminal 4 to transmit a voice or text message indicating an instruction to avoid the object O to the user terminal 2. This allows the information processing device 1 to have the monitoring user check captured images in which the object O may appear based on the trajectory T of the other user's movement, making it easier for the monitoring user to support the user.

[0064] After the transmission unit 132 transmits the start information, the determination unit 133 determines whether the user terminal 2 that has received the start information has been able to start at least one of capturing images by the imaging unit and transmitting the captured images. For example, if the determination unit 133 is able to receive captured images from the user terminal 2 within a predetermined time after the transmission of the start information by the transmission unit 132, the determination unit 133 determines that the user terminal 2 has been able to start at least one of capturing images by the imaging unit and transmitting the captured images, and if the determination unit 133 is unable to receive captured images within the predetermined time, the determination unit 133 determines that the user terminal 2 has not been able to start at least one of capturing images by the imaging unit and transmitting the captured images.

[0065] The transmission unit 132 transmits information notifying the user that the captured image cannot be acquired to the user terminal 2 on the condition that the determination unit 133 determines that the user terminal 2 has failed to start at least one of capturing an image by the imaging unit and transmitting the captured image. The user terminal 2 displays information notifying the user that the captured image cannot be acquired on the display unit according to the information received from the information processing device 1. In this way, the information processing device 1 can notify the user that the captured image cannot be normally acquired from the user terminal 2, and can prevent the user from moving without assistance from the monitoring user.

[0066] After transmitting the start information to the user terminal 2, the transmission unit 132 may transmit end information to the user terminal 2 to terminate at least one of the capturing of images by the imaging unit of the user terminal 2 and the transmission of the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination, on the condition that the distance between the position of the user terminal 2 and the position of the object O becomes greater than a predetermined threshold (e.g., 10 meters).

[0067] The user terminal 2 terminates at least one of capturing images by the imaging unit of the user terminal 2 and transmitting the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination in accordance with the termination information received from the information processing device 1. This allows the information processing device 1 to automatically terminate at least one of capturing images by the imaging unit of the user terminal 2 and transmitting the captured image on condition that the user has moved away from a position where the object O may be present, thereby reducing resources required for processing related to imaging in the user terminal 2.

[0068] After the transmission unit 132 transmits the start information, the acquisition unit 131 acquires the captured image transmitted by the user terminal 2. The determination unit 133 determines whether or not an object O appears in the captured image acquired by the acquisition unit 131. The determination unit 133 determines whether or not an object O appears in the captured image by, for example, executing a known image recognition process on the captured image. The determination unit 133 may also receive information indicating whether or not an object O appears in the captured image, which is input by the monitoring user at the monitoring user terminal 4, and determine whether or not an object O appears in the captured image based on the information received from the monitoring user terminal 4.

[0069] The generation unit 134 generates, on condition that the determination unit 133 has determined that the object O is not captured in the captured image, at least one of notification information notifying the user that the object O is outside the imaging range of the imaging unit of the user terminal 2 and control information for controlling the imaging unit of the user terminal 2.

[0070] The transmission unit 132 transmits at least one of the notification information and the control information generated by the generation unit 134 to the user terminal 2. The user terminal 2 outputs the notification information received from the information processing device 1 to the user. In addition, the user terminal 2 controls the imaging unit in accordance with the control information received from the information processing device 1.

[0071] FIG. 6(a) is a schematic diagram for explaining the notification information transmitted by the transmission unit 132. In the example of FIG. 6(a), the notification information includes information indicating that the object O is outside the imaging range of the imaging unit of the user terminal 2, and an instruction to the user to change the orientation of the imaging unit. The notification information may include other information. The user terminal 2 outputs the notification information as sound from the speaker, but may also output the information as characters on the display unit. In this way, the information processing device 1 supports the user so that the imaging unit of the user terminal 2 can capture the object O when the object O is not captured in the captured image, and makes it easier for the monitoring user to check the object O in the captured image.

[0072] 6(b) and 6(c) are schematic diagrams for explaining the control information transmitted by the transmission unit 132. In the example of FIG. 6(b), the control information includes information for controlling the imaging unit of the user terminal 2 so as to widen the imaging range of the imaging unit. For example, the user terminal 2 changes the imaging unit from a mode capable of imaging a first imaging range (normal imaging mode in FIG. 6(b)) to a mode capable of imaging a second imaging range wider than the first imaging range (wide-angle imaging mode in FIG. 6(b)) in accordance with the control information. Thereby, when the object O is not captured in the captured image, the information processing device 1 automatically widens the imaging range of the imaging unit of the user terminal 2, making it easier for the monitoring user to confirm the object O in the captured image.

[0073] In the example of FIG. 6(c), the control information includes information for controlling the imaging unit of the user terminal 2 to change the orientation of imaging by the imaging unit of the user terminal 2. In this case, the imaging unit of the user terminal 2 is configured to be able to change its orientation by a motor or the like. For example, the user terminal 2 changes the imaging unit from a first orientation to a second orientation different from the first orientation in accordance with the control information. In this way, the information processing device 1 can automatically change the orientation of the imaging unit of the user terminal 2 when the object O is not captured in the captured image, making it easier for the surveillance user to confirm the object O in the captured image.

[0074] The determination unit 133 may determine whether or not the object O shown in the captured image captured by the imaging unit of the user terminal 2 is a fixed object fixed on the route R. The determination unit 133 may also receive a captured image of the object O captured by an imaging unit associated with a mobile terminal (another user terminal 3 or other terminal) different from the user terminal 2, and determine whether or not the object O is a fixed object based on the received captured image. A fixed object is, for example, a structure fixed to the ground, rather than luggage or the like temporarily placed on the ground. The determination unit 133 determines whether or not the object O is a fixed object, for example, by executing a known image recognition process on the captured image.

[0075] In addition, the determination unit 133 may receive information indicating whether or not the object O is a fixed object input by the monitoring user at the monitoring user terminal 4, and determine whether or not the object O is a fixed object based on the information received from the monitoring user terminal 4.

[0076] When the determination unit 133 determines that the object O is a fixed object, the determination unit 133 causes the storage unit 12 to store information indicating the position (coordinates, etc.) of the object O.

[0077] Fig. 7 is a schematic diagram for explaining information indicating the position of an object O determined to be a fixed object. In the example of Fig. 7, the position of the object O determined to be a fixed object is recorded on a map. In the next or subsequent determinations, the determination unit 133 may determine whether or not there is a possibility that the object O exists on the route R based on the position of the object O determined to be a fixed object.

[0078] For example, when the determination unit 133 determines that the object O is a fixed object at a first time point, the determination unit 133 determines that the object O may be present on the route R at a second time point after the first time point, without being based on the trajectory T of the positions of the other user terminals 3. This allows the information processing device 1 to reduce the calculation load for performing a determination process using the trajectory T from the next time onwards for the object O registered as a fixed object.

[0079] Furthermore, when the determination unit 133 determines that the object O is a fixed object at the first time point, the transmission unit 132 may transmit, to the user terminal 2, notification information notifying the user who plans to move along the route R at a second time point after the first time point that the object O may be present on the route R. The user terminal 2 outputs the notification information received from the information processing device 1 to the user before the user starts moving along the route R. This allows the information processing device 1 to alert the user in advance that the object O, which is a fixed object registered in the past, is present on the route R.

[0080] [Information processing method flow] 8 is a diagram showing a flowchart of an exemplary information processing method executed by the information processing device 1 according to this embodiment. When a user starts moving toward a destination, the user inputs the destination into the user terminal 2 owned by the user. For example, the user inputs voice or characters indicating the destination into the user terminal 2.

[0081] The user terminal 2 transmits destination information indicating the destination input by the user to the information processing device 1. The user terminal 2 also transmits position information indicating the position of the user terminal 2 measured by a measurement unit to the information processing device 1 at predetermined time intervals. In the information processing device 1, the acquisition unit 131 acquires the destination information and position information transmitted by the user terminal 2 (S11).

[0082] If the acquisition unit 131 cannot acquire location information, or if the accuracy of the location information acquired by the acquisition unit 131 is lower than a predetermined reference value (NO in S12), the transmission unit 132 transmits alert information indicating that high-accuracy location information cannot be acquired to the monitoring user terminal 4 without transmitting start information to the user terminal 2 (S13).

[0083] If the acquiring unit 131 can acquire location information with an accuracy equal to or higher than a predetermined reference value (YES in S12), the acquiring unit 131 acquires route information indicating a route R for moving from the position of the user terminal 2 indicated by the location information to the destination indicated by the destination information, and reference point information indicating one or more reference points located on the route R between the position of the user terminal 2 indicated by the location information and the destination indicated by the destination information (S14).

[0084] The information processing device 1 selects, as the next reference point, from among one or more reference points indicated by the reference point information, a reference point that is closest in the direction in which the user terminal 2 moves. The acquisition unit 131 acquires one or more pieces of trajectory information indicating a trajectory T of past positions of the other user terminals 3 owned by each of the multiple other users, the trajectory T passing through the range between the position of the user terminal 2 and the next reference point on the route R (S15).

[0085] When the number of pieces of trajectory information acquired by the acquisition unit 131 is equal to or smaller than a predetermined first reference value (NO in S16), the transmission unit 132 transmits alert information indicating that the number of pieces of trajectory information is insufficient to the monitoring user terminal 4 (S13) without transmitting start information to the user terminal 2. In addition, when the number of pieces of trajectory information acquired by the acquisition unit 131 is equal to or larger than a predetermined second reference value, the transmission unit 132 may transmit alert information indicating that the number of pieces of trajectory information is excessive to the monitoring user terminal 4 without transmitting start information to the user terminal 2.

[0086] When the number of pieces of trajectory information acquired by the acquisition unit 131 is greater than a predetermined first reference value (YES in S16), it is determined whether or not there is a possibility that a predetermined object O such as an obstacle exists on the route R indicated by the route information acquired by the acquisition unit 131, based on the trajectory T of the positions of the other user terminals 3 indicated by the trajectory information acquired by the acquisition unit 131 (S17).

[0087] If the determination unit 133 determines that there is no possibility that the object O exists on the path R (NO in S18), the information processing device 1 proceeds to step S20.

[0088] When the determination unit 133 determines that there is a possibility that an object O exists on the path R (YES in S18), the generation unit 134 generates start information for starting at least one of capturing an image by the imaging unit of the user terminal 2 and transmitting an image captured by the imaging unit of the user terminal 2 to a predetermined destination. The transmission unit 132 transmits the start information generated by the generation unit 134 to the user terminal 2 (S19).

[0089] In accordance with the start information received from the information processing device 1, the user terminal 2 starts at least one of capturing images using the imaging unit of the user terminal 2 and transmitting the captured images captured by the imaging unit of the user terminal 2 to a predetermined destination.

[0090] If the user has arrived at the destination, for example if the next reference point is the destination (YES in S20), the information processing device 1 ends the process. If the user has not arrived at the destination, for example if the next reference point is not the destination (NO in S20), the information processing device 1 selects a further next reference point from one or more reference points indicated by the reference point information, and repeats steps S15 to S20.

[0091] [Effects of the embodiment] According to the information processing system S of this embodiment, the information processing device 1 causes the user terminal 2 to start at least one of capturing images by the imaging unit and transmitting the captured image on the condition that it has determined that there is a possibility that an object O exists on the route R based on the trajectory T of the positions of the other user terminals 3. This eliminates the need for the user terminal 2 to constantly capture images by the imaging unit and transmit the captured images, and therefore the information processing device 1 can reduce the resources of the user terminal 2 required to support the user in avoiding the object O existing on the route R along which the user moves.

[0092] [Variations] In the above embodiment, depending on the route R to the destination input by the user, there may be a case where the amount of trajectory information stored in the storage unit 12 is small, and therefore the information processing device 1 cannot start at least one of imaging and transmission of the captured image to the user terminal 2. Therefore, when the amount of trajectory information related to the entire route R is small, the information processing device 1 according to this modification starts at least one of imaging and transmission of the captured image to the user terminal 2 from the beginning of the route R.

[0093] When the acquisition unit 131 acquires route information based on a destination input by a user, the acquisition unit 131 acquires one or more pieces of trajectory information indicating a trajectory T that has passed through a route R (for example, a trajectory T at least a portion of which intersects with the route R) from among the multiple trajectory information stored in the memory unit 12, as trajectory information related to the entire route R.

[0094] When the number of pieces of trajectory information related to the entire route R acquired by the acquisition unit 131 is greater than a predetermined third reference value, the information processing device 1 executes the process according to the above-described embodiment. The third reference value is stored in the storage unit 12 in advance.

[0095] On the other hand, when the number of trajectory information related to the entire route R acquired by the acquisition unit 131 is equal to or less than a predetermined third reference value, the generation unit 134 generates start information for starting at least one of the following: capturing an image by the imaging unit of the user terminal 2; and transmitting the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination.

[0096] The transmission unit 132 transmits the start information generated by the generation unit 134 to the user terminal 2. In accordance with the start information received from the information processing device 1, the user terminal 2 starts at least one of capturing an image with the imaging unit of the user terminal 2 and transmitting the captured image captured by the imaging unit of the user terminal 2 to a predetermined destination.

[0097] As a result, when the amount of trajectory information regarding the route R to the destination input by the user is small, the information processing device 1 can cause the user terminal 2 to start at least one of imaging and transmission of the captured image, regardless of the amount of trajectory information between individual reference points.

[0098] <Second embodiment> In the first embodiment, the information processing device 1 determines whether or not there is a possibility that an object O exists on the route R, whereas in this embodiment, the user terminal 2 determines whether or not there is a possibility that an object O exists on the route R. Below, differences from the first embodiment will be mainly described.

[0099] The processor of the user terminal 2 according to this embodiment executes a program stored in the storage unit of the user terminal 2 to perform processing corresponding to the processing performed by the information processing device 1 of the first embodiment.

[0100] The processor of the user terminal 2, like the acquisition unit 131, acquires position information indicating the position of the user terminal 2 and route information indicating the route R along which the user plans to move. Like the determination unit 133, the processor of the user terminal 2 determines whether or not there is a possibility that an object O exists on the route R, based on the trajectory T of the positions of other user terminals 3.

[0101] Instead of the process of transmitting the start information generated by the generation unit 134 by the transmission unit 132, the processor of the user terminal 2 starts at least one of capturing an image with the imaging unit of the user terminal 2 and transmitting the image captured by the imaging unit of the user terminal 2 to a predetermined destination, on the condition that it is determined that there is a possibility that an object O exists on the route R.

[0102] [Effects of the embodiment] According to the information processing system S of this embodiment, the user terminal 2 does not need to constantly capture images using the imaging unit and transmit the captured images, thereby reducing the resources of the user terminal 2 required to assist the user in avoiding an object O that is present on the route R along which the user is traveling.

[0103] <Third embodiment> In the first embodiment, the information processing device 1 determines whether or not the object O appears in the captured image after transmitting start information to the user terminal 2, whereas in this embodiment, the information processing device 1 determines whether or not the object O appears in the captured image without transmitting start information to the user terminal 2. Below, differences from the first embodiment will be mainly described.

[0104] In this embodiment, the user terminal 2 always captures images using the imaging unit and transmits the captured images. In the information processing device 1, the acquisition unit 131 acquires the captured images transmitted by the user terminal 2. The determination unit 133 determines whether or not an object O appears in the captured image acquired by the acquisition unit 131. The determination unit 133 determines whether or not an object O appears in the captured image by, for example, executing a known image recognition process on the captured image. The determination unit 133 may also receive information indicating whether or not an object O appears in the captured image, which is input by the monitoring user at the monitoring user terminal 4, and determine whether or not an object O appears in the captured image based on the information received from the monitoring user terminal 4.

[0105] The generation unit 134 generates at least one of notification information notifying the user that the object O shown in Fig. 6(a) is outside the imaging range of the imaging unit of the user terminal 2 and control information for controlling the imaging unit of the user terminal 2 shown in Fig. 6(b) and Fig. 6(c), on the condition that the determination unit 133 has determined that the object O is not captured in the captured image. The transmission unit 132 transmits at least one of the notification information and the control information generated by the generation unit 134 to the user terminal 2.

[0106] The user terminal 2 outputs the notification information received from the information processing device 1 to the user. The notification information includes, for example, information indicating that the object O is outside the imaging range of the imaging unit of the user terminal 2, and an instruction to the user to change the orientation of the imaging unit. In this way, the information processing device 1 supports the user so that the imaging unit of the user terminal 2 can capture the image of the object O when the object O is not captured in the captured image, making it easier for the monitoring user to confirm the object O in the captured image.

[0107] The user terminal 2 controls the imaging unit in accordance with the control information received from the information processing device 1. The control information includes, for example, information for controlling the imaging unit of the user terminal 2 to expand the imaging range of the imaging unit. This allows the information processing device 1 to automatically expand the imaging range of the imaging unit of the user terminal 2 when the object O is not captured in the captured image, making it easier for the surveillance user to confirm the object O in the captured image.

[0108] Furthermore, the control information includes, for example, information for controlling the imaging unit of the user terminal 2 to change the orientation of imaging by the imaging unit of the user terminal 2. In this way, the information processing device 1 can automatically change the orientation of the imaging unit of the user terminal 2 when the object O is not captured in the captured image, making it easier for the surveillance user to confirm the object O in the captured image.

[0109] [Effects of the embodiment] Conventionally, when the object O is not captured in the captured image, the monitoring user cannot grasp the state of the route R along which the user moves, and therefore cannot smoothly support the user's movement. According to the information processing system S according to the present embodiment, when the object O is not captured in the captured image captured by the imaging unit of the user terminal 2, the information processing device 1 notifies the imaging unit so that the imaging unit can capture the object O, or controls the imaging unit so that the imaging unit can capture the object O. This allows the information processing device 1 to easily give the monitoring user instructions to avoid the object O based on the captured image.

[0110] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote industry, innovation and infrastructure."

[0111] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0112] S Information Processing System 1. Information processing device 11 Communications Department 12 Storage section 13 Control section 131 Acquisition Department 132 Transmitter 133 Judgment section 134 Generation part 2. User terminal 3. Other users' devices 4. Monitoring user terminal

Claims

1. An acquisition unit that acquires location information indicating a location of a first portable terminal held by a user and route information indicating a route along which the user plans to travel; a determination unit that determines whether or not there is a possibility that a predetermined object exists on the route based on a trajectory of a position of a second mobile terminal owned by a user different from the user; a transmission unit that transmits to the first portable terminal start information for starting at least one of an image capture by an imaging unit associated with the first portable terminal and a transmission of an image captured by the imaging unit to a predetermined destination, on a condition that the determination unit has determined that there is a possibility that the object exists on the path; and An information processing device having the above configuration.

2. the transmitting unit transmits the start information on the condition that the determining unit determines that there is a possibility that the object exists on the path and the distance between a position of the first portable terminal and a position of the object becomes equal to or less than a threshold. The information processing device according to claim 1 .

3. After transmitting the start information to the first portable terminal, the transmission unit transmits to the first portable terminal end information for ending at least one of the imaging by the imaging unit and the transmission of the captured image captured by the imaging unit to the transmission destination, on a condition that a distance between a position of the first portable terminal and a position of the object becomes greater than the threshold value. The information processing device according to claim 2 .

4. The determination unit determines whether or not the object is captured in the captured image, The transmission unit transmits, to the first portable terminal, information notifying the user that the object is outside an imaging range of the imaging unit, on a condition that the determination unit determines that the object is not captured in the captured image. The information processing device according to claim 1 .

5. The determination unit determines whether or not the object is captured in the captured image, The transmission unit transmits, to the first portable terminal, control information for controlling the imaging unit to expand an imaging range of the imaging unit, on a condition that the determination unit determines that the object is not captured in the captured image. The information processing device according to claim 1 .

6. The determination unit determines whether or not the object is captured in the captured image, The transmission unit transmits, to the first portable terminal, control information for controlling the imaging unit to change an imaging direction of the imaging unit, on a condition that the determination unit determines that the captured image does not include the object. The information processing device according to claim 1 .

7. the determination unit determines whether or not the first portable terminal that has received the start information has been able to start at least one of capturing an image by the imaging unit and transmitting the captured image; The transmission unit transmits, to the first portable terminal, information notifying the user that the captured image cannot be acquired, on condition that the determination unit determines that the first portable terminal has not been able to start at least one of capturing the captured image by the imaging unit and transmitting the captured image. The information processing device according to claim 1 .

8. the transmission unit does not transmit the start information when the determination unit determines that there is a possibility that the object is present on the path and when a number or density of the plurality of trajectories within a predetermined range based on the object is equal to or greater than a reference value. The information processing device according to claim 1 .

9. the determination unit determines whether the object determined to possibly exist on the path is a fixed object fixed on the path; when the determination unit determines that the object is the fixed object at a first time point, the determination unit determines that the object is likely to be present on the path at a second time point after the first time point, without relying on the trajectory. The information processing device according to claim 1 .

10. When the determination unit determines that the object is the fixed object at the first time point, the transmission unit transmits, to the first portable terminal, information notifying the user who plans to travel along the route at a second time point after the first time point that the object may be present on the route. The information processing device according to claim 9.

11. The determination unit determines whether or not the object is the fixed object based on the captured image captured by the imaging unit associated with the first portable terminal. The information processing device according to claim 9.

12. The determination unit determines whether or not the object is the fixed object based on a captured image captured by an imaging unit associated with a portable terminal different from the first portable terminal. The information processing device according to claim 9.

13. The processor executes acquiring location information indicating a location of a first portable terminal held by a user and route information indicating a route along which the user plans to travel; determining whether or not there is a possibility that a predetermined object exists on the route based on a trajectory of a position of a second mobile terminal owned by a user different from the user; transmitting, to the first portable terminal, start information for starting at least one of capturing an image by an imaging unit associated with the first portable terminal and transmitting an image captured by the imaging unit to a predetermined destination, on the condition that it has been determined in the step of determining that there is a possibility that the object exists on the path; An information processing method comprising the steps of:

14. The processor: acquiring location information indicating a location of a first portable terminal held by a user and route information indicating a route along which the user plans to travel; determining whether or not there is a possibility that a predetermined object exists on the route based on a trajectory of a position of a second mobile terminal owned by a user different from the user; a step of starting at least one of capturing an image by an imaging unit associated with the first portable terminal and transmitting an image captured by the imaging unit to a predetermined destination, on the condition that it has been determined in the step of determining that there is a possibility that the object exists on the path; A program to execute.

Images