Trajectory generation method and trajectory generation system

Abstract

A trajectory generation method according to the present disclosure is executed by at least one processor of an information processing device equipped with at least one processor, and includes: detecting a user operation related to generation of a trajectory of a moving body in a target region; relaxing a condition under which two or more points included in the trajectory are regarded as the same point on the basis of the user operation; and correcting the trajectory if there is a set of points satisfying the relaxed condition.

Description

Trajectory generation method and trajectory generation system

[0001] The present disclosure relates to a trajectory generation method and a trajectory generation system.

[0002] Conventionally, techniques such as SLAM (Simultaneous Localization and Mapping) and VSLAM (Visual SLAM) are known that simultaneously estimate position information (e.g., self-position) from three-dimensional information of the surrounding real space and create an environmental map based on the three-dimensional positions of multiple feature points. In these techniques, the same feature point is extracted from the three-dimensional information of the surrounding real space, and the position information and the three-dimensional position of the feature point are alternately estimated. Therefore, position estimation and map creation are interdependent. Therefore, in an unknown environment, errors may accumulate (cumulate) in the trajectory of the estimated position. When this cumulative error occurs, an inconsistency occurs between the position information and the environmental map, such as not being able to recognize that the vehicle has returned to its original position on a circular route (loop).

[0003] A known method for correcting accumulated errors is loop closing, which corrects a trajectory when the same position is observed around a loop (loop detection) using the continuity of the loop as a constraint. For example, Patent Literature 1 discloses a technique for increasing the frequency of loop closing to reduce accumulated errors in the aircraft's position, etc. Furthermore, Patent Literature 2 discloses a technique for reducing the complexity of creating highly accurate map information by evaluating the degree of accumulation of errors in position information and determining the need for correction processing of accumulated errors in map information based on the evaluation results.

[0004] JP 2017-111688 A JP 2023-069019 A

[0005] However, in an environment where a sufficient number of feature points cannot be obtained from three-dimensional information of the surrounding real space, there is a risk that the accumulated error of the trajectory cannot be properly corrected, such as inability to properly detect loops.

[0006] One of the problems that the present disclosure aims to solve is to appropriately correct accumulated errors in a trajectory even in an environment where a sufficient number of feature points cannot be obtained.

[0007] The trajectory generation method according to the present disclosure is a method executed by at least one processor in an information processing device having the at least one processor, which detects user operations related to the generation of a trajectory of a moving body in a target area, relaxes the conditions for considering two or more points included in the trajectory as the same point based on the user operations, and corrects the trajectory if there is a pair of points that meets the relaxed conditions.

[0008] FIG. 1 is a diagram illustrating an example of the configuration of a trajectory generation system according to an embodiment. FIG. 2 is a diagram illustrating an overview of a trajectory generation method according to an embodiment. FIG. 3 is a diagram illustrating an example of the functional configuration of the trajectory generation system according to an embodiment. FIG. 4 is a diagram illustrating an example of the hardware configuration of an information processing device that realizes each device of the trajectory generation system according to an embodiment. FIG. 5 is a flowchart illustrating an example of the flow of edge-side trajectory generation processing executed in the trajectory generation system according to an embodiment. FIG. 6 is a flowchart illustrating an example of the flow of cloud-side trajectory generation processing executed in the trajectory generation system according to an embodiment. FIG. 7 is a diagram illustrating an example of a screen display on a client terminal of the trajectory generation system according to an embodiment. FIG. 8 is a diagram illustrating an example of the configuration of metadata used in the trajectory generation processing according to an embodiment. FIG. 9 is a diagram illustrating another example of the functional configuration of the trajectory generation system according to an embodiment.

[0009] Hereinafter, with reference to the drawings, embodiments of an information processing method (trajectory generation method), an information processing device (trajectory generation device), an information processing system (trajectory generation system), a program, and a recording medium according to the present disclosure will be described.

[0010] In the description of the present disclosure, components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings may be given the same reference numerals, and descriptions thereof may be omitted as appropriate. Furthermore, even when the same or substantially the same parts are shown, the dimensions and proportions may be different depending on the drawing. Furthermore, for example, in order to ensure the visibility of the drawings, reference numerals may be given to only the main components in the description of each drawing, and reference numerals may not be given to components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings.

[0011] In the description of the present disclosure, components having the same or substantially the same functions may be distinguished by adding an alphanumeric character to the end of the reference symbol. Alternatively, when multiple components having the same or substantially the same functions are not distinguished, they may be collectively described by omitting the alphanumeric character at the end of the reference symbol.

[0012] In the following description, a case where a trajectory is generated by simultaneously performing position estimation and map creation using VSLAM, which uses images as three-dimensional information of the surrounding real space, is exemplified. Furthermore, a case where a moving image (video) including multiple frames is used as the image used as three-dimensional information of the surrounding real space is exemplified, but still images may be used for some or all of the images. Furthermore, the three-dimensional information of the surrounding real space is not limited to images; measurement information obtained by a distance sensor such as Lidar (Light Detection and Ranging) may also be used. In other words, the technology disclosed herein may be applied to a system that generates a trajectory using SLAM.

[0013] 1 is a diagram illustrating an example of the configuration of a trajectory generation system 1 according to an embodiment. As illustrated in FIG. 1, the trajectory generation system 1 includes a client terminal 2 (edge) and a cloud 3.

[0014] In the trajectory generation system 1, the client terminal 2 and the cloud 3 are communicatively connected via a network N (telecommunication line) such as the Internet. Note that communication between the client terminal 2 and the cloud 3 may be performed via a dedicated communication line (telecommunication line).

[0015] The client terminal 2 is, for example, a portable terminal (mobile terminal) such as a smartphone, a smart watch, a personal computer (PC), or a tablet PC. For example, the client terminal 2 is an example of a moving object that is held or worn by a user moving in the target area and moves in the target area. Alternatively, the client terminal 2 may be attached to or mounted on a moving object such as a forklift that moves in the target area. Note that in the present disclosure, a user moving with the client terminal 2 in the target area may also be referred to as a moving object.

[0016] The mobile object may be any type of vehicle, such as an electric vehicle (EV) driven by a motor, an automobile driven by an engine (internal combustion engine), or a hybrid vehicle driven by both an engine and a motor. The mobile object may be, for example, a passenger car, truck, motorcycle, or other automobile (vehicle), but may also be an electric bicycle, electric kick scooter, electric wheelchair, construction machine, agricultural machine, ship, train, airplane, or the like. The mobile object may be configured to be autonomously mobile, or may be configured to be mobile in response to direct or remote control by a driver.

[0017] The cloud 3 is a hardware resource configured to be available to the client terminal 2 via a network N. The cloud 3 is at least one server device constructed on the network N, such as the Internet. The cloud 3 may be constructed to further include other devices such as network storage (storage device).

[0018] Here, an outline of a trajectory generation method executed by cooperation between the client terminal 2 and the cloud 3 in the trajectory generation system 1 according to the embodiment will be described. Fig. 2 is a diagram for explaining the outline of the trajectory generation method according to the embodiment.

[0019] The trajectory generation system 1 according to the embodiment creates metadata related to trajectory generation. For example, the metadata includes information about a target area traveled while being photographed by the client terminal 2, and position information about the start and end points of the travel path. The information about the target area includes, for example, identification information for uniquely identifying the target area. The position information about the start and end points is, for example, information indicating positions in the coordinate system of an environmental map. Details of the metadata will be described later (see, for example, FIG. 8 ).

[0020] The trajectory generation system 1 also captures images using the client terminal 2 while moving within the target area, obtaining video data (moving image data) as three-dimensional information of the surrounding real space. The trajectory generation system 1 then performs VSLAM processing using the video data to create a map in the coordinate system of the client terminal 2.

[0021] Specifically, the trajectory generation system 1 extracts feature points, such as corners and edges, from each frame of video data. The trajectory generation system 1 alternately estimates the position information (hereinafter also referred to as the terminal position) of the client terminal 2 and the three-dimensional positions of the feature points, for example, so that geometric constraints regarding the positional relationships between the extracted feature points are satisfied between frames. The trajectory generation system 1 then creates a map, for example, a collection of the three-dimensional positions of the feature points relative to the terminal position in each frame. In other words, the trajectory generation system 1 generates a trajectory, which is a collection of the terminal positions in each frame, by creating the map. The map created by the trajectory generation method according to the present disclosure is, for example, a point cloud map, but other types of maps, such as a grid map, a geometric map, or a topological map, may also be used. The trajectory generation system 1 then converts the generated map (trajectory) in the coordinate system of the client terminal 2 into the coordinate system of the environmental map based on the position information of the start and end points in the metadata, and draws it.

[0022] Furthermore, in VSLAM processing, when the same position is observed after going around a loop (loop detection), the trajectory generation system 1 performs loop closing processing to correct the trajectory using the continuity of the loop as a constraint, and corrects the accumulated error of the trajectory. Conventionally, in an environment where a sufficient number of feature points cannot be obtained from three-dimensional information of the surrounding real space, there was a risk that the accumulated error of the trajectory could not be properly corrected, such as inappropriate loop detection.

[0023] In contrast, as shown in FIG. 2 , in a trajectory generation method according to an embodiment, a trajectory is corrected based on loop closing information stored as metadata, for example. Here, the loop closing information is an example of information related to the generation of a trajectory of a moving object in a target area. The information related to the generation of the trajectory is, for example, information related to the continuity of the trajectory based on a user operation. That is, the trajectory generation method according to an embodiment is configured to incorporate human knowledge regarding the same points that form a loop, i.e., the pair of the start point and end point (revisited point) of the loop. Specifically, in the trajectory generation method according to an embodiment, a loop closing process is performed with the continuity between the start point and end point (revisited point) of the loop as a constraint, and the trajectory is corrected. Details of the loop closing information will be described later (see, for example, FIGS. 7 and 8 ).

[0024] Therefore, according to the trajectory generation method of the embodiment, even in an environment where a sufficient number of feature points cannot be obtained from three-dimensional information of the surrounding real space, it is possible to perform appropriate loop detection and appropriately correct accumulated errors in the trajectory. Below, a trajectory generation system 1 that realizes this trajectory generation method will be described in more detail.

[0025] FIG. 3 is a diagram illustrating an example of a functional configuration of the trajectory generation system 1 according to the embodiment.

[0026] As shown in Figure 3, the client terminal 2 of the trajectory generation system 1 has a site information input unit 201, a site information transmission unit 202, a site drawing display unit 203, a drawing information receiving unit 204, a start point / end point input unit 205, a memory unit 206, a photographing unit 207, a photographing data transmission unit 208, a loop closure information input unit 209, and a trajectory viewing unit 210.

[0027] The site information input unit 201 acquires site information input in response to user operation. This site information is information about a target area for which trajectory generation (mapping) is performed by moving while photographing using the client terminal 2. As an example, the site information includes identification information for uniquely identifying the target area. For example, the site information input unit 201 stores the acquired site information in the storage unit 206 as information constituting metadata.

[0028] The site information transmission unit 202 transmits the site information created by the site information input unit 201 to the cloud 3 .

[0029] The site drawing display unit 203 displays the site drawing received from the cloud 3 by the drawing information receiving unit 204 on the display of the client terminal 2 (see FIG. 7).

[0030] The drawing information receiving unit 204 receives (downloads) from the cloud 3 drawings corresponding to the site information from among drawings showing at least one target area within at least one site that are pre-stored in the memory unit 305 of the cloud 3.

[0031] The start point / end point input unit 205 acquires position information of the start point and end point input in response to a user operation. This position information of the start point and end point is information indicating the positions in the coordinate system of the environmental map of the start point and end point of a movement path (route) when moving while taking pictures using the client terminal 2 and generating a trajectory (map). As an example, the position information of the start point and end point relates to a movement path that can form a loop-shaped path (circular path, closed path). As an example, the position information of the start point and end point relates to a movement path that passes through predetermined points (checkpoints) within the target area. For example, the start point / end point input unit 205 stores the acquired position information of the start point and end point in the storage unit 206 as information constituting metadata.

[0032] Here, a checkpoint is a point that a site worker or supervisor should check at a site (target area) such as a manufacturing site, a logistics warehouse, or a construction site. For example, a checkpoint is a point where inventory and materials are checked. For example, a checkpoint is a point where the safety and progress of a moving object's movement route, such as a route for transporting materials and equipment using a forklift, a production line in a home appliance manufacturing factory, or a loading and unloading of cargo within a logistics warehouse are checked. Note that checkpoints may be acquired from the cloud 3 together with the site map, for example, by being stored in advance in the storage unit 307 of the cloud 3 as data accompanying the site map. Alternatively, checkpoints may be acquired by being input or edited in response to a user operation on the client terminal 2.

[0033] The storage unit 206 stores programs, parameters, data being processed, and data resulting from processing related to each process executed by the client terminal 2. As an example, the storage unit 206 stores metadata (see FIG. 8 ) related to map creation (trajectory generation). As an example, the storage unit 206 stores video data related to map creation (trajectory generation).

[0034] The image capturing unit 207 includes a camera that captures images to acquire image information. As an example, this camera is an omnidirectional camera. The image capturing unit 207 captures images while the client terminal 2 is moving within the target area, and acquires video data (video data) as three-dimensional information of the surrounding real space. The image capturing unit 207 starts or stops capturing images in response to a user operation. For example, the image capturing unit 207 stores the acquired video data in the storage unit 206 together with the time of capture.

[0035] The imaging data transmission unit 208 transmits the video data captured by the imaging unit 207 to the cloud 3. Note that the video data may be transmitted, for example, at least one frame at a time, or the entire video data may be transmitted all at once. The imaging data transmission unit 208 also transmits the metadata stored in the storage unit 206 to the cloud 3 together with the video data or separately. Note that the metadata may be transmitted by another functional unit. Furthermore, the transmission of the metadata may be performed periodically or cyclically, or may be triggered by the creation or update of the metadata.

[0036] The loop closure information input unit 209 detects user operations related to the generation of a trajectory. This user operation related to the generation of a trajectory is, for example, a user operation related to the continuity of the trajectory. The loop closure information input unit 209 also acquires loop closure information input in response to the user operation. This loop closure information is information related to trajectory generation (map creation) performed by moving while capturing images using the client terminal 2, and is, for example, information related to the continuity of the trajectory of the client terminal 2 (user) in the target area for the trajectory generation. For example, the loop closure information input unit 209 stores the acquired loop closure information in the storage unit 206 as information constituting metadata.

[0037] For example, when the user reaches a checkpoint, the user inputs loop closure information via the client terminal 2. As an example, the loop closure information is acquired in response to a user operation specifying that the user has reached a checkpoint. As an example, the loop closure information is acquired in response to a user operation specifying two or more points included in the trajectory. The input of the loop closure information will be described later (see FIG. 7 ).

[0038] The trajectory viewing unit 210 receives (downloads) from the cloud 3 display information of the trajectory generated by the VSLAM unit 401 of the cloud 3, for example, in response to a user operation requesting viewing of a map (trajectory). Furthermore, based on the display information received from the cloud 3, the trajectory viewing unit 210 displays the trajectory on the display of the client terminal 2, for example, by superimposing the trajectory on a map of the site. As an example, the trajectory is displayed superimposed on the map in the same way as the checkpoints and loopable routes are displayed superimposed on the map in the example of FIG. 7 .

[0039] As shown in FIG. 3 , the cloud 3 of the trajectory generation system 1 includes a photographing data receiving unit 301 , a trajectory display unit 302 , a trajectory generating unit 303 , a loop closure determining unit 304 , and a storage unit 305 .

[0040] The image capturing data receiving unit 301 receives video data captured by the client terminal 2 .

[0041] The trajectory display unit 302 displays the trajectory generated by the VSLAM unit 401 on the display of the client terminal 2. As an example, the trajectory display unit 302 generates display information of the trajectory in response to a request to view a map (trajectory) from the client terminal 2. In addition, the trajectory display unit 302 transmits the generated display information of the trajectory to the client terminal 2.

[0042] The trajectory generation unit 303 and the loop closure determination unit 304 constitute a VSLAM unit 401 of the trajectory generation system 1. That is, in the trajectory generation system 1 according to this embodiment, map creation (trajectory generation) is performed by VSLAM processing on the cloud 3 side.

[0043] The trajectory generation unit 303 estimates the position of the client terminal 2 in the target area by using video data obtained by the imaging unit 207 of the client terminal 2. The trajectory generation unit 303 also estimates (acquires) position information for each of a plurality of points on the movement path of the client terminal 2, and generates a trajectory based on the estimated position information for each of the plurality of points.

[0044] Specifically, the trajectory generation unit 303 extracts feature points such as corners and edges from each frame of the video data received from the client terminal 2. The trajectory generation unit 303 also alternately estimates the terminal position and the three-dimensional positions of the feature points, for example, so that geometric constraints are satisfied regarding the positional relationships of the extracted feature points between frames. The trajectory generation system 1 then creates a map, for example, from a set of three-dimensional positions of the feature points relative to the terminal position in each frame. Note that the created map includes the set of terminal positions in each frame, which is the trajectory of the client terminal 2 moving while capturing images in the target area. The trajectory generation unit 303 also converts the generated map (trajectory) in the coordinate system of the client terminal 2 into the coordinate system of the environmental map based on the position information of the start point and end point in the metadata, and stores the converted map in the storage unit 305.

[0045] Furthermore, when a loop is detected by the loop closure determination unit 304, the trajectory generation unit 303 performs loop closing processing to correct the trajectory using the continuity of the loop as a constraint, and corrects the accumulated error of the trajectory. In other words, when there is a pair of points that are determined by the loop closure determination unit 304 to satisfy the determination condition or the relaxed determination condition, the trajectory generation unit 303 performs loop closing processing to close the trajectory with the pair of points (start point and end point of the loop) as the same point.

[0046] The loop closure determination unit 304 performs loop detection to detect identical points that form a loop, i.e., a pair of a start point and an end point (revisit point) of a loop. For example, the loop closure determination unit 304 performs loop closure determination to determine whether two or more points included in a trajectory satisfy a condition (hereinafter referred to as a determination condition) for being considered as the same point. As an example, the loop closure determination unit 304 periodically performs loop closure determination at a predetermined timing. If there is a pair of points that satisfy the determination condition, the loop closure determination unit 304 outputs the continuity of the loop with the pair of points as the start point and revisit point to the trajectory generation unit 303 as a constraint (constraint condition).

[0047] Furthermore, for example, when the metadata includes loop closure information, i.e., when a user operation related to the continuity of the trajectory in the target area is performed, the loop closure determination unit 304 relaxes the determination conditions, and then performs the loop closure determination as described above using the relaxed determination conditions.

[0048] Here, the loop closure information in this embodiment is frame information indicating the frame at which a user operation is detected. For example, when a user reaches a checkpoint, specifically, at the start point and revisited point of a loop included in the trajectory, the user performs a user operation to specify those points (to specify that the checkpoint has been reached). Then, the loop closure determination unit 304 acquires frame information at the time when the user operation was performed at each of the start point and revisited point of the loop.

[0049] Relaxing the judgment conditions means, for example, making it easier to determine that two locations are the same in the loop closure judgment, and is determined appropriately depending on the content of the judgment conditions. As an example, there may be a judgment condition that determines that two locations are the same when the number of matching feature points is greater than a predetermined first threshold. In this case, relaxing the judgment conditions means using a second threshold that is smaller than the first threshold so that two locations are determined to be the same when a smaller number of matching feature points are required.

[0050] The storage unit 305 stores programs, parameters, data being processed, and data on processing results related to each process executed in the cloud 3. As an example, the storage unit 305 stores information on a created map (trajectory). As an example, the storage unit 305 stores judgment conditions (e.g., thresholds) for loop closure judgment. Note that as the judgment conditions for loop closure judgment, information indicating the details of relaxation from the judgment conditions, such as the amount or rate of relaxation, may be stored together with the judgment conditions under normal circumstances (e.g., first threshold), or the relaxed judgment conditions (e.g., second threshold) themselves may be stored.

[0051] FIG. 4 is a diagram illustrating an example of the hardware configuration of an information processing device 8 that realizes each device of the trajectory generation system 1 according to the embodiment.

[0052] The information processing device 8 is a device that comprehensively controls the operations of each device that it implements. As shown in Fig. 4, the information processing device 8 has a processor 81, a main storage device 82, an auxiliary storage device 83, and a device I / F (interface) 84. The processor 81, the main storage device 82, the auxiliary storage device 83, and the device I / F 84 are interconnected by a bus or the like, and have a hardware configuration that utilizes a typical computer. Note that each component of the information processing device 8 may be implemented by a combination of two or more components.

[0053] The processor 81 includes, for example, a CPU (Central Processing Unit). The processor 81 loads a program stored in the auxiliary storage device 83 into the main storage device 82 and executes it, thereby comprehensively controlling the operation of the information processing device 8 and realizing each function of the information processing device 8. Note that some or all of the functions of the information processing device 8 may be realized by a dedicated hardware circuit.

[0054] Here, the processor 81 according to the embodiment is an example of at least one processor in the information processing device 8. As the at least one processor, at least one other processor may be used instead of or in addition to the CPU. As the other processor, various processors such as a CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or a dedicated arithmetic circuit realized by an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array) can be appropriately used.

[0055] As an example, in an information processing device 8 that realizes the client terminal 2, a processor 81 may realize each function of the client terminal 2, including some or all of the functional units exemplified in Fig. 3. As an example, in an information processing device 8 that realizes the cloud 3, a processor 81 may realize each function of the cloud 3, including some or all of the functional units exemplified in Fig. 3.

[0056] 3 illustrates only the functions necessary for explaining the main parts of this embodiment, but the functions of each device in the trajectory generation system 1 are not limited to these. Furthermore, in each device in the trajectory generation system 1 according to the embodiment, two or more components may be integrated. For example, in the cloud 3, the trajectory generation unit 303 and the loop closure determination unit 304 may be integrated and realized as a single function. Furthermore, part of the functional configuration and hardware configuration of each device in the trajectory generation system 1 according to the embodiment may be realized by an information processing device 8 external to each device.

[0057] The main storage device 82 is, for example, a RAM (Random Access Memory). The main storage device 82 temporarily stores data necessary for various processes performed by the processor 81. Here, the main storage device 82 according to the embodiment is an example of at least one memory in the information processing device 8.

[0058] The auxiliary storage device 83 is, for example, a read-only memory (ROM). The auxiliary storage device 83 stores programs and parameters that realize various processes performed by the processor 81, as well as various data used in the processes. Here, the auxiliary storage device 83 according to the embodiment is an example of at least one memory in the information processing device 8. As the at least one memory, at least one other memory may be used instead of or in addition to the ROM. As the other memory, various storage media and storage devices such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory may be used as appropriate.

[0059] As an example, in the information processing device 8 that realizes the client terminal 2, the main storage device 82 and the auxiliary storage device 83 may realize the storage unit 206. As an example, in the information processing device 8 that realizes the cloud 3, the main storage device 82 and the auxiliary storage device 83 may realize the storage unit 305.

[0060] The device I / F 84 is an interface for various input / output and / or communication in the information processing device 8. For example, the device I / F 84 may be configured to be connectable to an external output device that outputs audio, images, or video, or may include an output interface configured to function as such an output device. As output devices, various displays (display components) such as a liquid crystal display (LCD), an organic electroluminescence (EL) display, or a projector, as well as speakers, can be used as appropriate. For example, the device I / F 84 may be configured to be connectable to an external input device that acquires user operations, or may include an input interface configured to function as such an input device. As input devices, a keyboard, a mouse, a touch panel, a microphone, or the like can be used as appropriate. For example, the device I / F 84 may be configured to be connectable to an external communication device that communicates with the outside of the information processing device 8, or may include a communication interface configured to function as such a communication device. As the communication interface, a communication circuit for wired communication such as USB (Universal Serial Bus (registered trademark)) or Ethernet (registered trademark), or a communication circuit for wireless communication compatible with various standards such as 3G, LTE, 4G, 5G, 6G, Wi-Fi (registered trademark), Bluetooth (registered trademark), and infrared communication can be used as appropriate.

[0061] As an example, in the information processing device 8 that realizes the client terminal 2, the device I / F 84 may realize an input interface (site information input unit 201, start point / end point input unit 205, photographing unit 207, loop closure information input unit 209) that connects or realizes input devices. The device I / F 84 of the client terminal 2 may also realize an output interface (site drawing display unit 203, trajectory viewing unit 210) that connects or realizes output devices. This output interface is an example of a display that displays at least one of a site drawing display screen, a start point / end point position information input screen, a loop closure information input screen, and a map (trajectory) display screen. The device I / F 84 of the client terminal 2 may also realize a communication interface (site information transmission unit 202, drawing information receiving unit 204, photographed data transmission unit 208, trajectory viewing unit 210) that connects or realizes communication devices for communicating with external devices such as the cloud 3.

[0062] As an example, in the information processing device 8 that realizes the cloud 3, the device I / F 84 may realize a communication interface (photographing data receiving unit 301, trajectory display unit 302) for communicating with the outside, such as the client terminal 2.

[0063] Each device included in the trajectory generation system 1 may be realized by an information processing device 8 shared with other devices, or each device may be realized by a different information processing device 8. For example, when the client terminal 2 according to the present disclosure is implemented in a moving body such as a vehicle, the client terminal 2 may be configured integrally with a computer mounted on the moving body. For example, the client terminal 2 may be configured integrally with various on-board sensors such as sensors for Advanced Driver-Assistance Systems (ADAS) and drive recorders (cameras), or a computer that controls the operation of such sensors. Furthermore, for example, the client terminal 2 may be configured integrally with a car navigation device of the moving body that is configured to implement functions such as route guidance.

[0064] The computer (information processing device 8) mounted on these mobile bodies may be a computer such as an ECU (Electronic Control Unit) provided inside the mobile body (e.g., a vehicle), a DCU (Domain Control Unit) such as a CDC (Cockpit Domain Controller) integrating multiple ECUs, or an OBU (On Board Unit). Alternatively, the computer mounted on the mobile body may be an external computer installed in the passenger space (cabin) of the mobile body, for example, near the dashboard or in the trunk.

[0065] In addition, the computer (information processing device 8) that realizes each device included in the trajectory generation system 1 may be communicably connected to other computers in the moving body (e.g., vehicle) via an in-vehicle network including a CAN (Controller Area Network), Ethernet (registered trademark), USB (registered trademark), etc.

[0066] An example of the operation of the trajectory generation system 1 according to the embodiment will be described below with reference to the drawings. Note that the processing described below is an example, and it is possible to change the processing order, delete some processing, or add other processing.

[0067] 5 is a flowchart showing an example of the flow of a trajectory generation process on the edge (client terminal 2) side executed in the trajectory generation system 1 according to the embodiment. The flow in FIG. 5 is started, for example, when an application (program) related to trajectory generation is started in the client terminal 2.

[0068] The client terminal 2 accepts input of site information. The client terminal 2 acquires the input site information and transmits it to the cloud 3. The client terminal 2 also receives (downloads) a site map corresponding to the site information from the cloud 3 and displays the site map with predefined checkpoints superimposed (S101). The client terminal 2 also determines a route (travel path) that has at least a partial loop (S102). That is, the client terminal 2 determines a route in which some sections are looped or a route in which the entire section is a single loop. For example, the client terminal 2 determines a travel path (circular path, closed path) that includes a looped section and passes through predetermined points (checkpoints) within the target area. Note that this loopable route may be determined in response to a user operation specifying the order of checkpoints, etc., or may be determined automatically according to predetermined rules (such as the order of checkpoints and search conditions), or may be editable in response to a user operation after being automatically determined.

[0069] After the loopable route is determined, the client terminal 2 starts capturing images using the image capturing unit 207 in response to, for example, a user operation (S103). Then, when a moving object such as a user or a vehicle starts moving, the client terminal 2 starts moving along the loopable route set in the target area (S104).

[0070] The client terminal 2 also performs edge-side map creation processing while moving along a loopable route while taking photographs (S105). Specifically, the client terminal 2 detects user operations related to the continuity of the trajectory. For example, the user performs an operation to specify the loop start point or loop end point each time the user reaches a position where the loop closes (the start point or end point of the loop) on a loopable route. The client terminal 2 then acquires loop closure information when this user operation is performed.

[0071] Thereafter, when the movement along the loopable route is completed (S106) and the shooting is completed (S107), the client terminal 2 uploads the video data obtained by the shooting together with the metadata to the cloud 3 (S108). Then, the flow in FIG. 5 ends.

[0072] FIG. 6 is a flowchart showing an example of the flow of a trajectory generation process on the cloud 3 side executed in the trajectory generation system 1 according to the embodiment. The flow of FIG. 6 is initiated, for example, when an application (program) related to trajectory generation is launched on the client terminal 2 and communication with the cloud 3 is established, or when site information is received from the client terminal 2. Here, an example is shown in which "landmarks," which are feature points (markers) such as barcodes or two-dimensional codes, are placed in advance in the site (target area) on objects such as traffic cones (registered trademark), walls, pillars, and floors, and the "landmarks" extracted from the image are used as feature points to estimate location and create a map. In this case, depending on the number of "landmarks" placed or the shooting environment, there is a risk that a sufficient number of feature points may not be extracted. Furthermore, the number and placement of "landmarks" may differ from those of the previous measurement.

[0073] In response to a download request for a site map from the client terminal 2 (e.g., receiving site information), the cloud 3 transmits a map of the site (target area) to the client terminal 2 and displays it (S201). The cloud 3 also transmits information indicating predefined checkpoints for the site to the client terminal 2 and displays the information superimposed on the site map (S202). The cloud 3 also displays a loop close button on the client terminal 2 for inputting loop close information (S203).

[0074] When the VSLAM unit 401 of the cloud 3 receives the video data and metadata from the client terminal 2, it starts map creation (trajectory generation) (S204). The cloud 3 selects an arbitrary frame from multiple frames of the video data as a "key frame" (S205). The VSLAM unit 401 also extracts "landmarks" from each frame as feature points and registers their three-dimensional positions on a map (S206). Thereafter, the VSLAM unit 401 generates a trajectory by alternately repeating position estimation and map creation using the feature points as reference points.

[0075] Furthermore, the VSLAM unit 401 determines whether loop closing information has been input (S207).

[0076] If the loop closing information is input (S207: Yes), the VSLAM unit 401 relaxes the loop determination conditions (S208), and then performs a loop closing determination to determine whether the loop can be closed using the relaxed determination conditions (S209).

[0077] If it is determined that the loop cannot be closed under the relaxed criteria (S209: No), the VSLAM unit 401 determines whether the relaxed criteria have been relaxed to the minimum value of the relaxed criteria, which indicates the range within which the relaxed criteria can be relaxed (S210). This determination can also be expressed as a determination of whether the criteria have been relaxed to the limit allowed by the relaxed criteria.

[0078] If the current determination condition is not the minimum value of the relaxed condition (S210: No), the flow in Fig. 6 returns to the process of S208. On the other hand, if the current determination condition is the minimum value of the relaxed condition (S210: Yes), the flow in Fig. 6 ends.

[0079] If it is determined that the loop can be closed under the relaxed criteria (S209: Yes), the VSLAM unit 401 optimizes and recreates the map (trajectory) using the continuity of the loop, which has the pair of points that are considered to be the same under the relaxed criteria as its start and end points, as a constraint (constraint condition), i.e., corrects the trajectory through loop closing processing (S211).Then, the flow in FIG. 6 ends.

[0080] Note that the loop closure determination under relaxed determination conditions relates to, for example, a point estimated using a frame indicated by frame information (loop closure information) among a plurality of points included in the trajectory.

[0081] Furthermore, if loop closure information is not input (S207: No), the VSLAM unit 401 performs a loop closure determination to determine whether the loop can be closed using the unrelaxed criteria (S212). If it is determined that the loop cannot be closed using the unrelaxed criteria (S212: No), the flow in FIG. 6 ends. On the other hand, if it is determined that the loop can be closed using the unrelaxed criteria (S212: Yes), the VSLAM unit 401 optimizes and recreates the map (trajectory) using the continuity of the loop, with the pair of points considered to be the same as the start and end points, as a constraint (constraint condition), i.e., corrects the trajectory through loop closing processing (S211). Then, the flow in FIG. 6 ends.

[0082] Here, a screen display of the client terminal 2 in the trajectory generation process according to the embodiment will be described. FIG. 7 is a diagram illustrating an example of a screen display on the client terminal 2 of the trajectory generation system 1 according to the embodiment. FIG. 7 illustrates an example of a state in which information indicating a site map and information indicating checkpoints is received from the cloud 3, and predetermined checkpoints 503 are superimposed on the site map 501, along with loop close buttons 511 and 513 for inputting loop close information. As shown in FIG. 7 , the client terminal 2 may further superimpose a loopable route 505 on the site map 501. For example, a user may travel along the route 505 while taking photographs and passing through each of the predetermined checkpoints 503 while viewing this screen display. Then, when the user reaches a position 507 where the loop is to be closed, the user presses the loop close buttons 511 and 513 to input the loop close information.

[0083] For example, in the example of FIG. 7 , when the user travels along route 505 and reaches at least one of the second to fourth checkpoints 503, the user presses "Loop Start" loop close button 511 to specify the start point of the loop. Thereafter, the user reaches the same second to fourth checkpoints 503 as the fourteenth to sixteenth checkpoints 503, respectively. Therefore, when the user travels along route 505 and reaches at least one of the fourteenth to sixteenth checkpoints 503, the user presses "Loop End" loop close button 513 to specify the end point (revisit point) of the loop.

[0084] In the trajectory generation system 1 according to this embodiment, loop closing information corresponding to the user operation input in this manner is written to the metadata. FIG. 8 is a diagram illustrating an example of the configuration of metadata 601 used in the trajectory generation process according to this embodiment. As illustrated in FIG. 8 , the metadata 601 includes “identification information,” “loop closing information,” “start point information,” “end point information,” and “timestamp information.” The “identification information” is information for uniquely identifying the target area (site). The “loop closing information” is information regarding the continuity of the trajectory in the target area, and in this embodiment, is frame information indicating the frame at which a user operation on the loop closing button 511 or 513 is detected. The “start point information” is position information indicating the start point of the travel route along which the client terminal 2 moves while capturing images. For example, in the example of FIG. 7 , the “end point information” is position information indicating the first checkpoint when traveling along the loopable route 505. The “end point information” is position information indicating the end point of the travel route along which the client terminal 2 moves while capturing images. For example, in the example of FIG. 7 , the “end point information” is position information indicating the 22nd checkpoint when traveling along the loopable route 505. The "timestamp information" is information that indicates the time when the creation of the metadata 601 started, for example.

[0085] As described above, the trajectory generation system 1 according to this embodiment relaxes the criteria for determining that two or more points included in a trajectory are the same point based on user operations related to the continuity of the trajectory of a moving body in a target area (user operations related to the generation of a trajectory), and corrects the trajectory if there is a pair of points that meets the relaxed criteria.

[0086] Conventionally, for example, at manufacturing sites, logistics warehouses, and construction sites, omnidirectional cameras have been used to check inventory and materials, and to manage progress. For this reason, when labor shortages arise due to a decrease in the working population, a single supervisor must visit multiple sites. To address this issue, for example, an omnidirectional camera can be held or worn by a site worker or mounted on a mobile object such as a forklift. This allows for capturing images while the site worker is patrolling or the forklift is moving, and allows for map creation (trajectory generation), which may enable a supervisor to remotely check the site.

[0087] However, in environments where a sufficient number of feature points cannot be obtained from 3D information of the surrounding real space, there is a risk that the accumulated error of the trajectory cannot be properly corrected, such as inability to perform appropriate loop detection. For example, if it is not possible to always install markers in the same location at the site (target area) or if the distance from the camera to the marker is far, appropriate loop closing processing cannot be performed, resulting in a decrease in the accuracy of map creation (trajectory generation). Another problem is that it is time-consuming to match the positions of markers placed at the site with their positions on the drawing.

[0088] In this situation, as described above, the trajectory generation system 1 according to this embodiment can incorporate human knowledge as loop closing information into the loop closing process. Therefore, the trajectory generation system 1 according to this embodiment enables appropriate loop detection and appropriate correction of accumulated errors in the trajectory, even in an environment where a sufficient number of feature points cannot be obtained from three-dimensional information of the surrounding real space. Furthermore, the user can appropriately correct accumulated errors in the trajectory without having to change the route for the loop closing process or take the trouble of maintaining a common area in consecutive frames.

[0089] Other embodiments of the present disclosure will be described below with reference to the drawings. In the following description of each embodiment, differences will be mainly described, and descriptions of content that overlaps with the above-described content will be omitted as appropriate.

[0090] Second Embodiment In the trajectory generation system 1 according to the above-described embodiment, a camera (photographing unit 207) may be provided outside the client terminal 2.

[0091] The trajectory generation system 1 according to this embodiment further includes a camera (for example, an omnidirectional camera). This camera corresponds to the photographing unit 207 realized by the device I / F 84 of the client terminal 2 according to the first embodiment, for example. The camera may be connected to the input interface (device I / F 84) of the client terminal 2, or may be communicably connected to the communication interface (device I / F 84) of the client terminal 2. As an example, the client terminal 2 is communicably connected to the camera. The client terminal 2 and the camera may communicate via the network N, or may communicate directly without using the network N. This communication may be wired or wireless.

[0092] If the camera that captures and obtains the video data and the client terminal 2 that accepts user operations related to the continuity of the trajectory are separate terminals, the client terminal 2 may not be able to obtain the accurate number of frames of the video data. For this reason, the trajectory generation system 1 according to this embodiment uses, as the "loop close information," time information that indicates the "time" at which a user operation on the loop close buttons 511, 513 was detected.

[0093] Therefore, in the trajectory generation system 1 according to this embodiment, the criteria for determining whether two points are the same in the loop closing process are relaxed based on the time information at the time when a user operation related to the continuity of the trajectory (a user operation related to the generation of the trajectory) is detected. Specifically, when time information is input as loop closing information, a frame of the video data is identified based on the time information, and the relaxed criteria are applied to a point estimated using the frame indicated by the time information (loop closing information) among the multiple points included in the trajectory.

[0094] In this way, the trajectory generation system 1 according to this embodiment can appropriately correct accumulated errors in the trajectory even when video data acquired using an external camera is used.

[0095] Third Embodiment Note that the VSLAM processing executed on the cloud 3 side in the trajectory generation system 1 according to the above-described embodiments may be executed on the edge side. Here, differences from the trajectory generation system 1 according to the first embodiment will be mainly described, but the technology according to this embodiment can also be applied to the trajectory generation system 1 according to the second embodiment.

[0096] 9 is a diagram showing another example of the functional configuration of the trajectory generation system 1 according to the embodiment. In the trajectory generation system 1 according to the present embodiment, the client terminal 2 further includes a trajectory generation unit 211 and a loop closure determination unit 212 that constitute a VSLAM unit 401. The VSLAM unit 401 of the client terminal 2 according to the present embodiment corresponds to the VSLAM unit 401 of the cloud 3 according to the first embodiment. More specifically, the trajectory generation unit 211 and the loop closure determination unit 212 of the client terminal 2 according to the present embodiment are similar to the trajectory generation unit 303 and the loop closure determination unit 304 of the cloud 3 according to the first embodiment, respectively.

[0097] In the trajectory generation process according to each of the above-described embodiments, the map creation process (S105 in FIG. 5) is a process in which loop closure information is input and acquired. On the other hand, in the trajectory generation process according to this embodiment, the flow of FIG. 6 is executed as the map creation process (S105 in FIG. 5), and the trajectory is generated and corrected in the client terminal 2. Furthermore, in the process of S108 in FIG. 5, data of the generated and / or corrected map (trajectory) is transmitted (uploaded) to the cloud 3 instead of video data.

[0098] As described above, in the trajectory generation system 1 according to this embodiment, the client terminal 2 generates a trajectory for video data in real time. That is, in the trajectory generation system 1 according to this embodiment, VSLAM processing changes dynamically in response to input from the user. For this reason, the trajectory generation system 1 according to this embodiment can use, as loop closure information, location information at the time when a user operation related to the continuity of the trajectory (a user operation related to the generation of the trajectory) is detected. That is, in the trajectory generation system 1 according to this embodiment, it is possible to apply relaxed determination conditions regarding the location (point) of the client terminal 2 at the time when a user operation specifying that the user has reached a checkpoint is performed.

[0099] In the trajectory generation system 1 according to this embodiment, loop closing information is input to the client terminal 2, and the metadata 601 is held in the client terminal 2. Therefore, the loop closing information does not need to be written to the metadata 601, but may be written thereto.

[0100] In the trajectory generation system 1 according to each of the above-described embodiments, a user operation related to the continuity of a trajectory (a user operation related to the generation of a trajectory) may be performed to directly specify a set of points that are the same among multiple points included in the trajectory. For example, in the trajectory generation system 1 according to each of the above-described embodiments, each checkpoint superimposed on the site map may be displayed as selectable. Alternatively, each point forming the trajectory superimposed on the site map may be displayed as selectable. In other words, a user operation for inputting loop closure information may be configured to directly specify each checkpoint or each point forming the trajectory. In other words, when a user operation is performed to specify a set of points included in the trajectory, the VSLAM unit 401 according to each of the above-described embodiments may perform loop closing processing to close the trajectory by regarding the set of points as the same point.

[0101] In the trajectory generation system 1 according to each of the above-described embodiments, if the client terminal 2 is moved by a mobile object such as a vehicle, the user does not necessarily need to be present in the target area. For example, the trajectory generation system 1 according to each of the above-described embodiments may be implemented using a mobile object that can move within the target area by remotely operating the user. For example, the user may remotely move the mobile object along a route that closes the loop, and when the mobile object reaches the position where the loop closes, input loop closure information using an operating terminal (information processing device 8) that remotely operates the mobile object. Even with this configuration, accumulated errors in the trajectory can be appropriately corrected.

[0102] The trajectory generation system 1 according to each of the above-described embodiments may be configured to present the results of the above-described VSLAM processing using a machine learning model including pattern matching (loop closure results) to the user on the client terminal 2. For example, if the VSLAM processing detects a common object or otherwise observes the same position around a loop (loop detection), the trajectory generation system 1 according to each of the above-described embodiments can perform loop closure processing without inputting loop closure information. In such a case, the trajectory generation system 1 according to each of the above-described embodiments may present information indicating that a loop has been detected to the user.

[0103] As an example, the trajectory generation system 1 according to each of the above-described embodiments may perform a loop closure determination in the same manner as in step S212 in the map creation (trajectory generation) processing of S204 to S206 in Fig. 6. Furthermore, if the trajectory generation system 1 according to each of the above-described embodiments determines in the loop closure determination that the loop can be closed under unrelaxed determination conditions (S212: Yes), it may present information indicating that a loop has been detected to the user.

[0104] As an example, the result of VSLAM processing (loop closure result), i.e., information indicating that a loop has been detected, may be information indicating which checkpoints superimposed on the site map were deemed to be the same point in VSLAM processing, or it may be information indicating the detected loop.

[0105] As an example, the results of the VSLAM processing (loop closure results) may be presented by highlighting the display of checkpoints 503 at points that are considered to be the same point, by making them different from the display of other checkpoints 503 through the presence or absence of a frame, the thickness of the frame, the color of the frame, the display color, blinking, etc.

[0106] As an example, the results of the VSLAM processing (loop closure results) may be presented in a manner similar to the display of the loopable route 505, by displaying a route corresponding to the detected loop, or by highlighting the display differently from the display of the route 505 by using the presence or absence of a line, the thickness of the line, the color of the line, the display color, blinking, etc.

[0107] As an example, the results of the VSLAM process (loop closing results) may be presented by displaying a message indicating that the points are considered to be the same point.

[0108] As an example, the results of the VSLAM process (loop closure results) may be presented by displaying a message indicating that a loop has been detected.

[0109] As an example, the results of the VSLAM processing (loop closure results) are presented by, for example, a screen display, but may also be presented by audio.

[0110] This allows the user to receive the results of the VSLAM processing and, if they agree, perform the loop end operation. In other words, with the above configuration, if a loop is detected, assistance can be provided, such as prompting the user to perform the loop end operation. Therefore, with the above configuration, it is possible to appropriately correct the accumulated error in the trajectory.

[0111] In the trajectory generation system 1 according to each of the above-described embodiments, the point at which the user performs the loop end operation by pressing the loop close button 511 and the point at which the user performs the loop end operation by pressing the loop close button 513 are points that the user has determined to be the "same" point. Therefore, at least two images at the point at which the user performed the loop end operation can be considered to be images that the user has determined to be the "same" point, regardless of whether a loop was detected in the VSLAM process or whether the results of the VSLAM process (loop closure results) have been presented.

[0112] Therefore, the machine learning model used in the VSLAM processing in the trajectory generation system 1 according to each of the above-described embodiments may have its parameters updated (tuned) using two or more images of the point where the user performed the loop end operation as training data.

[0113] The tuning of the machine learning model may be performed in the trajectory generation system 1 according to each of the above-described embodiments. Alternatively, the trajectory generation system 1 according to each of the above-described embodiments may be configured to output training data for tuning the machine learning model, and tuning of the machine learning model may be performed externally.

[0114] This makes it possible to further improve the accuracy of VSLAM processing.

[0115] In the above-described embodiments, the loop close buttons 511 and 513 are displayed, and the "loop close information" is acquired in response to a user operation on the loop close buttons 511 and 513. However, this is not limiting. The UI for inputting the "loop close information" may be a voice or a gesture in addition to or instead of a screen tap on the loop close buttons 511 and 513.

[0116] As an example, the trajectory generation system 1 according to each of the above-described embodiments may acquire user utterances such as "loop start" and "loop end" as "loop closure information" when the trajectory generation system 1 acquires the user utterances such as "loop start" and "loop end" through the microphone of the client terminal 2.

[0117] As an example, the trajectory generation system 1 according to each of the above-described embodiments may acquire a predetermined movement of the client terminal 2, such as a "shake," as "loop closure information" when the movement is acquired via a sensor such as an acceleration sensor of the client terminal 2.

[0118] As an example, the trajectory generation system 1 according to each of the above-described embodiments may acquire predetermined user movements as “loop closure information” when the movements are acquired via a sensor such as an acceleration sensor or camera of the client terminal 2.

[0119] In each of the above-described embodiments, "is it A?" refers to at least one of "is A" and "is not A." In other words, in each of the above-described embodiments, the determination of "is A" may be realized by determining "is A," or by determining "is not A," or by determining both of these.

[0120] The programs executed by each device of the trajectory generation system 1 according to each of the above-described embodiments may be provided by being recorded in an installable or executable file format on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0121] The programs executed by each device of the trajectory generation system 1 according to each of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The programs executed by each device of the trajectory generation system 1 according to each of the above-described embodiments may be provided or distributed via a network such as the Internet.

[0122] Furthermore, the programs executed by the devices of the trajectory generation system 1 according to the above-described embodiments may be configured to be provided by being pre-installed in a ROM or the like.

[0123] According to at least one of the embodiments described above, it is possible to appropriately correct the accumulated error of the trajectory even in an environment where a sufficient number of feature points cannot be obtained.

[0124] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents.

[0125] (Additional Notes) The above embodiments disclose the following technologies. (1) A trajectory generation method executed by at least one processor in an information processing device including at least one processor, the trajectory generation method detecting a user operation related to generation of a trajectory of a moving object in a target area, relaxing a condition for regarding two or more points included in the trajectory as the same point based on the user operation, and correcting the trajectory if there is a pair of points that satisfy the relaxed condition. (2) The trajectory generation method according to (1) above, displaying a map of the target area including at least one preset checkpoint, the user operation being an operation for specifying that the moving object has reached the checkpoint. (3) The trajectory generation method according to (1) or (2) above, the user operation being an operation for specifying two or more points included in the trajectory. (4) The trajectory generation method according to any one of (1) to (3), wherein the method periodically determines whether two or more points included in the trajectory are the same point under the condition for regarding them as the same point, and corrects the trajectory if there is a pair of points that satisfy the condition for regarding them as the same point. (5) The trajectory generation method according to any one of (1) to (4), wherein, if there is a pair of points that satisfy the relaxed condition, the method corrects the trajectory by performing loop closing to close the trajectory by regarding the pair of points as the same point. (6) The trajectory generation method according to any one of (1) to (5), wherein the relaxation of the condition for regarding them as the same point is based on frame information of an image acquired by a camera mounted on the moving object at a time when the user operation is detected. (7) The trajectory generation method according to any one of (1) to (6), wherein the relaxation of the condition for regarding them as the same point is based on time information at a time when the user operation is detected. (8) The trajectory generation method according to any one of (1) to (7), wherein the relaxation of the condition for determining the same point is based on position information at a time point when the user operation is detected.(9) The trajectory generation method according to any one of (1) to (8), wherein, when the user operation is performed to specify a set of points included in the trajectory, loop closing is performed to close the trajectory as a route with the set of points as the same point. (10) The trajectory generation method according to any one of (1) to (9), wherein position information of the moving body is estimated. (11) The trajectory generation method according to any one of (1) to (10), wherein position information of the moving body is estimated using an image acquired by a camera mounted on the moving body. (12) The trajectory generation method according to any one of (1) to (11), wherein position information of each of a plurality of points on the movement route of the moving body is acquired, the trajectory is generated based on the position information of each of the plurality of points, and the trajectory is displayed. (13) A trajectory generation system comprising: a camera that captures images to acquire image information, at least one memory that stores the image information together with position information at the time of capturing the images, at least one processor that generates a trajectory of a moving object in a target area based on the position information, an input interface that detects a user operation related to the generation of the trajectory, and an output interface that displays the trajectory, wherein the at least one processor relaxes a condition for regarding two or more points included in the trajectory as the same point based on the user operation, and corrects the trajectory when there is a pair of points that satisfy the relaxed condition. (14) The trajectory generation system according to (13) above, wherein the at least one memory, the input interface, and the output interface are mounted on a mobile terminal that is communicatively connected to the camera, the moving object is the camera and the mobile terminal that are moved together with a user, and the at least one processor is mounted on an information processing device that is communicatively connected to the mobile terminal. (15) The trajectory generation system according to (13), wherein the camera, the at least one memory, the input interface, and the output interface are mounted on a mobile terminal, which is the mobile body that is moved with a user, and the at least one processor is mounted on an information processing device that is communicatively connected to the mobile terminal.(16) A trajectory generation system comprising: at least one processor; and at least one memory, wherein the at least one processor executes a program stored in the at least one memory to execute the trajectory generation method according to any one of (1) to (12) above. (17) A program that causes a computer to execute the trajectory generation method according to any one of (1) to (12) above, or a computer-readable non-transitory recording medium on which the program is stored.

[0126] REFERENCE SIGNS LIST 1 Trajectory generation system 2 Client terminal 201 Site information input unit 202 Site information transmission unit 203 Site drawing display unit 204 Drawing information reception unit 205 Start point / end point input unit 206 Memory unit 207 Photography unit 208 Photography data transmission unit 209 Loop closure information input unit 210 Trajectory viewing unit 211 Trajectory generation unit 212 Loop closure determination unit 3 Cloud 301 Photography data reception unit 302 Trajectory display unit 303 Trajectory generation unit 304 Loop closure determination unit 305 Memory unit 401 VSLAM unit 501 Site drawing 503 Checkpoint 505 Route 507 Loop closure position 511, 513 Loop closure button 601 Metadata 8 Information processing device 81 Processor 82 Main memory device 83 Auxiliary memory device 84 Device I / F N Network

Claims

1. A trajectory generation method executed by at least one processor in an information processing device having the at least one processor, comprising: detecting a user operation related to the generation of a trajectory of a moving object in a target area; relaxing a condition for regarding two or more points included in the trajectory as the same point based on the user operation; and correcting the trajectory if there is a pair of points that satisfies the relaxed condition.

2. The trajectory generation method according to claim 1, wherein a map of the target area including at least one pre-set checkpoint is displayed, and the user operation is an operation specifying that the moving object has reached the checkpoint.

3. The trajectory generation method according to claim 1, wherein the user operation is an operation of specifying two or more points to be included in the trajectory.

4. A trajectory generation method according to claim 1, which periodically determines whether two or more points included in the trajectory are the same point based on the conditions for determining that they are the same point, and corrects the trajectory if there is a pair of points that meets the conditions for determining that they are the same point.

5. The trajectory generation method according to claim 1, wherein if there is a pair of points that satisfies the relaxed condition, the trajectory is corrected by performing loop closing to treat the pair of points as the same point and close the trajectory as a closed path.

6. The trajectory generation method according to claim 1, wherein the relaxation of the conditions for determining the same point is based on frame information at the time when the user operation is detected in an image obtained by a camera mounted on the moving object.

7. The trajectory generation method according to claim 1, wherein the relaxation of the conditions for determining the same location is based on time information at the time the user operation is detected.

8. The trajectory generation method according to claim 1, wherein the relaxation of the conditions for determining the same location is based on the position information at the time when the user operation is detected.

9. The trajectory generation method according to claim 1, wherein, when the user operation is performed to specify a set of points included in the trajectory, loop closing is performed to close the trajectory by treating the set of points as the same point.

10. The trajectory generation method according to claim 1, further comprising estimating position information of the moving object.

11. The trajectory generation method according to claim 1, wherein position information of the moving body is estimated using an image obtained by a camera mounted on the moving body.

12. The trajectory generation method according to claim 1, further comprising: acquiring position information of each of a plurality of points on the movement path of the moving object; generating the trajectory based on the position information of each of the plurality of points; and displaying the trajectory.

13. A trajectory generation system comprising: a camera that takes pictures and acquires image information; at least one memory that stores the image information together with position information at the time of shooting; at least one processor that generates a trajectory of a moving object in a target area based on the position information; an input interface that detects user operations related to the generation of the trajectory; and an output interface that displays the trajectory, wherein the at least one processor relaxes the conditions for regarding two or more points included in the trajectory as the same point based on the user operation, and corrects the trajectory if there is a pair of points that satisfy the relaxed conditions.

14. The trajectory generation system described in claim 13, wherein the at least one memory, the input interface, and the output interface are mounted on a mobile terminal communicatively connected to the camera, the moving object is the camera and the mobile terminal that are moved together with a user, and the at least one processor is mounted on an information processing device communicatively connected to the mobile terminal.

15. The trajectory generation system described in claim 13, wherein the camera, the at least one memory, the input interface, and the output interface are mounted on a mobile terminal, which is the mobile body moved with the user, and the at least one processor is mounted on an information processing device communicatively connected to the mobile terminal.

Images