Information processing apparatus, information processing method, and program

Abstract

To more accurately perform route searching for an object to move in a space.SOLUTION: An information processing apparatus comprises: a storage unit that stores object information related to a moving object, and space information related to a space in which the object moves; an input unit that receives input of movement condition information indicating a condition related to the movement of the object; and an information processing unit that generates route related information including information related to the movement of the object in the space. The space information includes at least space three-dimensional information of the space. The movement condition information includes at least information on a starting point and an end point through which the object moves. The information processing unit generates the route related information on the basis of the object information, the space information, and the movement condition information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, mainly at construction sites, technologies for acquiring and using three-dimensional information of a three-dimensional space, which is a real space, using a laser scanner, LiDAR (Light Detection And Ranging), etc. have been spreading.

[0003] Patent Document 1 discloses a technique for generating three-dimensional reference data in a structure based on three-dimensional model data indicating the arrangement of components in the structure where a moving object moves and three-dimensional sensing data indicating the result of sensing the inside of the structure by a sensor, and generating a moving route for the moving object based on this three-dimensional reference data.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the prior art, since it only uses the arrangement of components in the structure and the three-dimensional sensing data obtained by sensing the inside of the structure for generating the moving route, the degree of danger when the moving object collides with a component is not considered, and it has been required to perform more accurate route search.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and a program capable of performing route search for an object to move in a space with higher accuracy.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, the present invention includes a storage unit that stores object information regarding a moving object and space information regarding the space in which the object moves, an input unit to which movement condition information indicating conditions regarding the movement of the object is input, and a generation unit that generates route-related information including information regarding the movement of the object in the space. The space information includes at least three-dimensional space information of the space, the movement condition information includes at least information on a starting point and an ending point where the object moves, and the generation unit generates the route-related information based on the object information, the space information, and the movement condition information.

Advantages of the Invention

[0007] According to the present invention, there is an effect that route search for an object to move in a space can be performed with higher accuracy.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

DETAILED DESCRIPTION OF THE INVENTION

[0009] With reference to the accompanying drawings, embodiments of an information processing apparatus, an information processing method, and a program will be described in detail below.

[0010] (First Embodiment) FIG. 1 is a block diagram showing an example of an information processing apparatus according to the first embodiment. This information processing apparatus is a device that generates route information indicating the route along which an object should move and information regarding risks present along the route, using object information, space information, etc., which will be described later, and presents it to the user.

[0011] As shown in FIG. 1, the information processing apparatus 1 includes a storage unit 10, a UI (User Interface) unit 11, and an information processing unit 12. The information processing unit 12 controls the storage unit 10 and the UI unit 11.

[0012] As shown in FIG. 1, the information processing unit 12 includes, as an example, a CPU (Central Processing Unit) 12A, a ROM (Read Only Memory) 12B, a RAM (Random Access Memory) 12C, and a network interface 12D. The CPU 12A controls the operation of the entire information processing apparatus 1. The ROM 12B stores programs used for driving the CPU 12A. The RAM 12C is used as a work area for the CPU 12A. The network interface 12D is an interface for data communication using a communication network such as the Internet and a gateway (repeater).

[0013] The information processing unit 12 reads out object information and space information, which will be described later, from the storage unit 10. Also, movement condition information, which will be described later, is read from the input unit 11B into the information processing unit 12. The information processing unit 12 generates route-related information based on this object information, space information, and movement condition information.

[0014] The memory unit 10 stores various types of information. The various types of information include object information regarding an object moving within a space and space information of the space in which the object moves. The movement of the object includes the movement of the object due to a carrying-in operation or the like performed by a person, the movement of the object by an operation of a remote control or the like, the movement of an object equipped with a sensor or a camera that autonomously travels, and the like.

[0015] The various types of information stored in the memory unit 10 may be stored in the memory unit 10 in advance, or may be input and stored via the UI unit 11.

[0016] The object information includes information such as the size, weight, shape, hardness, and material of the object. These information may be three-dimensional information of the object (point cloud information, 3D model information, etc.), information estimated based on the three-dimensional information and luminance image (RGB image, etc.) of the object (an example of the estimated information of the object), or information input by the user (an example of the input information of the object). The input information may be input along with the three-dimensional information of the object. The object information may be configured to include information indicating that the size of the object increases or decreases during the route. For example, by performing a predetermined process (assembly, folding, division, combination with another object, etc.) on the object, information indicating that the size and weight of the object increase or the size and weight decrease can be included.

[0017] The space information includes three-dimensional information of the space (point cloud information, 3D model information, etc.). The space information may include information such as the size, weight, shape, hardness, and material of the components of the space. These information regarding the components of the space may be information estimated based on the three-dimensional information and luminance image (RGB image, etc.) of the space (an example of the estimated information of the space), or information input by the user (an example of the input information of the space). The input information may be input along with the three-dimensional information of the space.

[0018] Information such as size, weight, shape, hardness, and material may be information expressed in text such as "large, small", "heavy, light", "hard, soft", "round, square", "metal, plastic, wood", etc., or may be information of continuous quantities such as volume, weight, shape feature quantities (radius of a circle, length of each side of a cube), hardness, etc. For text indicating qualitative states such as "large, small", "heavy, light", "hard, soft", etc., it is preferable that a stepwise finite numerical range is associated therewith.

[0019] Three-dimensional information is information representing a three-dimensional space that is a real space (depth information and preferably luminance information), point cloud data, a 3D model, etc. The three-dimensional information may be obtained by a normal optical camera, an omnidirectional camera, a ToF (Time-of-Flight) type camera, a stereo camera, a laser scanner, LiDAR (Light Detection And Ranging), a system using photogrammetry, etc. The ToF method is a method of irradiating an object to be measured with infrared rays and obtaining the distance from the time until the reflected light returns. A stereo camera is a camera that obtains depth information as distance information using the distance between two cameras and the parallax information of the images obtained by each of the two cameras.

[0020] The file format of the three-dimensional information is not limited. The three-dimensional information is represented by, for example, shape data representing the three-dimensional shape of a three-dimensional object included in a real space. The three-dimensional information is, for example, a file in a point cloud format representing a three-dimensional space by discrete points, a file in a polygon mesh format representing a three-dimensional space by vertices and faces, etc. A file in a point cloud format may be referred to as a depth map, a distance image, etc.

[0021] Examples of files in a point cloud format include files represented by extensions such as ".xyz", ".e57", ".ply", etc. Examples of files in a polygon mesh format include files represented by extensions such as ".obj", ".fbx", ".stl", etc.

[0022] The UI unit 11 includes a display unit 11A and an input unit 11B. The display unit 11A is a display for displaying various types of information. The input unit 11B receives operation instructions from the user. The input unit 11B is, for example, a keyboard, a pointing device, a mouse, etc. The display unit 11A and the input unit 11B may be an integrally configured touch panel.

[0023] Route-related information generated by the information processing unit 12 is displayed on the display unit 11A. The route-related information includes route information indicating the route of an object moving in space, risk level information indicating the risk level on the route, etc. The display unit 11A may display caution information including explanatory text for conveying the risk level to the user.

[0024] The route-related information may be configured to include caution information. When the route-related information includes caution information, the display unit 11A displays the caution information together when displaying the route-related information. The caution information may be displayed in association with the route information or the risk level information, or may be displayed independently of the route information and the risk level information (for example, in a display area different from the display area for the route information and the risk level information).

[0025] The route information indicates the route connecting the starting point and the ending point of an object moving in space. The route information may indicate the three-dimensional shortest route that travels from the starting point to the ending point without colliding with the components of the space, etc., or may indicate a three-dimensional route that is not the shortest but has a low risk of accidentally colliding. Also, the route information may be a three-dimensional route passing through a specified point (such as an elevator or a work space), or may include a plurality of route information generated according to the work content during the route.

[0026] The risk level information indicates the risk level corresponding to each point on the route. The risk level information may include the coordinates of each point and the corresponding risk level. The risk level is a measure indicating the degree of risk based on the object information and the space information, and may be a measure represented by text information such as "dangerous" or "safe", or may be a continuous quantity (such as numerical information from 0 to 100 with 100 representing danger and 0 representing safety).

[0027] The risk level indicates the likelihood of problems (such as the collapse of components, damage or failure of objects, etc.) occurring when an object contacts components or the like on the path, or the degree to which some countermeasures (such as dirt removal or preventive measures through maintenance) are necessary. For example, consider a case where the object information indicates that the object is heavy and hard, and the spatial information of the components on the path through which the object passes indicates that they are light and hard. Since the object is heavier, there is a possibility that the components may easily fall. In this case, the risk level may be expressed as a level such as "dangerous" or a numerical value such as 80.

[0028] Caution information is information for conveying to the user the risks and countermeasures existing at each point along the path. For example, it includes text information (explanation text) such as "There is a risk of hitting the wall", "The side of the passage is easily damaged", "It is better to reinforce with a sheet", "Be careful not to trip due to a low step", "The floor is uneven", "A forklift is required due to a high step", "A cart with a stopper is required due to a continuous slope".

[0029] Caution information is generated based on risk level information, but it may also be generated using object information and spatial information. Caution information may be generated included in the path-related information, or may be generated without being included in the path-related information.

[0030] Movement condition information is input to the input unit 11B. The movement condition information is information such as the start point and end point information of an object moving in space, and information on points (elevator entrances on different floors, work spaces where a predetermined process (assembly, folding, disassembly, combination with another object, etc.) is to be performed on the object, etc.) that the object should pass through on the path.

[0031] The information on the start point, end point, and points to be passed through may be coordinate data or information indicating locations in the space ("second floor entrance", "first floor exit", "third floor elevator", etc.).

[0032] Subsequently, the generation process of path-related information in the information processing unit 12 will be described.

[0033] FIG. 2 is a functional block diagram showing an example of the generation process of route-related information. The information processing unit 12 in the present embodiment includes a route information generation unit 20, a risk level information generation unit 21, a route-related information generation unit 22, and a control unit 23.

[0034] The route information generation unit 20 generates route information based on the object information and space information read from the storage unit 10 and the movement condition information input to the input unit 11B.

[0035] The risk level information generation unit 21 generates risk level information based on the object information, space information, and the route information generated by the route information generation unit 20.

[0036] The route-related information generation unit 22 generates route-related information from the route information and the risk level information.

[0037] The control unit 23 controls the route information generation unit 20, the risk level information generation unit 21, and the route-related information generation unit 22, and performs input and output of data, generation of route information, risk level information, and route-related information, etc.

[0038] FIG. 3 is a flowchart showing the generation procedure of route-related information.

[0039] First, the control unit 23 inputs movement condition information from the input unit 11B to the information processing unit 12 (S31), reads object information from the storage unit 10 (S32), and reads space information from the storage unit 10 (S33). Note that the object information and the space information may be input from the input unit 11B without passing through the storage unit 10.

[0040] Next, the route information generation unit 20 generates route information based on the object information, space information, and movement condition information (S34).

[0041] Next, the risk level information generation unit 21 generates risk level information based on the object information, space information, and route information (S35).

[0042] Next, the route-related information generation unit 22 generates route-related information from the generated route information and the risk level information, and the control unit 23 displays the route-related information on the display unit 11A (S36).

[0043] Next, a specific method for generating the route information and the risk level information will be described.

[0044] The route information is generated as a three-dimensional optimal route by performing a search for the shortest route based on the object information, the space information, the start point information, and the end point information, such that the object does not collide with the components within the space. The route information may be generated based on the object information, the space information, the start point information, the end point information, and the risk level information described later. The risk level information can be used in a re-search (re-exploration) if there are passing points with a high risk level on the route generated in the first route search. Even if there are no passing points with a high risk level, the risk level information can be used in the route search when performing a re-search, for example, according to a user's instruction.

[0045] For the route search, an AI for route search that extends a solution algorithm for the optimal route problem in the deep learning version to three dimensions is applied. The AI for route search is machine-learned with a plurality of sets consisting of object information, space information, start point information, and end point information as inputs and the optimal route information (correct data) corresponding to each set as outputs. When performing a route search using the risk level information, the AI for route search is machine-learned with a plurality of sets consisting of object information, space information, start point information, end point information, and risk level information as inputs and the optimal route information (correct data) corresponding to each set as outputs.

[0046] For the route search, a rule-based method may be used instead of the AI. Specifically, the point cloud included in the space information is voxelized to form a "barrier", and a route search algorithm that extends a solution algorithm for the optimal route problem, such as Dijkstra's algorithm, A* algorithm, Bellman-Ford algorithm, etc., to three dimensions is applied.

[0047] The risk information is generated by calculating or inferring the risk corresponding to each point on the path based on the object information, the space information, and the path information (information on the path search result).

[0048] Figure 4 is a flowchart showing the generation procedure of the risk information.

[0049] The risk information generation unit 21 first extracts the points existing on the path from the point cloud based on the path information and the coordinates of each point of the point cloud of the space information (S41).

[0050] Next, based on the space information at each of the extracted points and the object information of the object passing through each point, the risk at each point is derived (S42).

[0051] For the derivation of the risk, an AI for risk derivation equipped with a solution algorithm for the input-output problem of the deep learning version is applied. The AI for risk derivation is machine-learned with the object information and the space information at each point of the space point cloud as inputs and the risk information (correct data) corresponding to each point as the output. The space point cloud used for machine learning may be input all at once, or may be input serially for each region obtained by dividing the space into regions. When inputting for each region, the correct data shall be the risk data when an object comes into contact with the components of each region, etc. The risk may be derived as a continuous quantity based on a predetermined calculation formula, or a predetermined range with a high risk among the derived continuous quantities may be derived as "dangerous".

[0052] Next, the risk information is generated as data associating the coordinates of each of the extracted points with the risk at each point (S43).

[0053] FIG. 5 shows an example of the generated risk information. In FIG. 5, the route number is a number for distinguishing each route when a plurality of routes are generated. The coordinates are the three-dimensional coordinates of points on the route. For example, the x-axis and y-axis are defined on a horizontal plane, and the z-axis is defined perpendicular to the horizontal plane for representation. In FIG. 5, the risk is expressed as a continuous quantity between 0 (indicating safety) and 100 (indicating danger). Regarding the case indicated by the dashed line in FIG. 5 (the case where the risk is 40 at the coordinates (1, 0, h)), it will be described in the explanation of FIG. 6 below.

[0054] FIG. 6 shows an example of displaying the route information and risk information generated by the information processing unit 12 according to the present embodiment on the display unit 11A. In this example, on the plane of the first floor of the building (the x-y plane including the x-axis and y-axis), the route from the starting point to the ending point where the object moves is displayed as route (1). The vertical axis (z-axis) is perpendicular to the x-y plane although not shown in the figure. On the first floor, there are structures A to D such as rooms and shelves that the object cannot pass through, a work space E where the object can pass through, and an elevator (EV). Also, it is assumed that the unit of the scale on each axis is a meter.

[0055] In FIG. 6, the arrow connecting the starting point and the ending point (route (1)) indicates the route information. The text "risk = numerical value" and the arrow extending from it represent the risk information.

[0056] Route (1) indicates the shortest route that proceeds along the passage in the right direction from the starting point, changes the direction downward (makes a right turn) at the dead end, and reaches the ending point. Since the width of the first straight line of route (1) is sufficiently wider than the object, the risk is shown as 20 (solid line arrow).

[0057] In the section labeled A1 with the constituent A, a material that is vulnerable to damage is used above the wall surface (at a location with a height h from the floor). Therefore, when the height of the moving object is higher than h and the upper part of the object is harder than the material of the wall surface, the information within the dashed frame in Figure 5 (risk level = 40 at the coordinates (1, 0, h)) is generated, and the risk level at the corresponding location in Figure 6 is shown as 40 (dashed arrow). On the other hand, when the height of the object is lower than h, the information indicated by the dashed line in Figure 5 is not generated, and there is no display of the risk level at the corresponding location in Figure 6 either.

[0058] In the next straight line of Route (1), the passage width becomes narrow immediately after a right turn, so the risk level is shown as 80 (solid arrow).

[0059] When the movement condition information is configured to include the points through which the object should pass during the route, the user can specify waypoints such as, for example, "workspace", "elevator", etc. In that case, the route generation unit generates route information passing through these points. The route information generation unit 20 may generate a route passing through the waypoint and a route not passing through the waypoint even when the waypoint is specified. This is because the route not passing through the waypoint may have a lower risk level or there may be a space suitable for work along the way.

[0060] When the object information includes information indicating that the size of the object increases or decreases, the user can include information such as that the size of the object can be decreased at a location where the passage is narrow, for example. In that case, the risk level information generation unit 21 generates risk level information with the risk level at the second straight-line route of Route (1) being 20.

[0061] Figure 7 shows another example of displaying the route information and the risk level information on the display unit 11A. In this example, a part of the passage along the route is displayed three-dimensionally. There is a large step in the passage along the route, and if the object proceeds as it is, there is a risk of falling and being damaged, so the risk level of the step part is shown as 80.

[0062] According to this embodiment, since the danger level information in the movement route is generated using the object information and the space information, it becomes possible to perform a more accurate route search, such as ensuring the safety on the route before moving the object and considering another route.

[0063] Each part included in the information processing apparatus 1 is realized by, for example, one or more processors and one or more memories. Each of the above parts may be realized by software, that is, by causing a processor such as a CPU (Central Processing Unit) to execute a program while reading data from a memory. For the memory, ROM (Read Only Memory), RAM (Random Access Memory), etc. can be used. Each of the above parts may be realized by a processor or a memory such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above parts may be realized by using software and hardware in combination. When using a plurality of processors, each processor may realize one of each part or two or more of each part. Further, the software for realizing each of the above parts may be stored in a terminal on the user side and the above program may be executed on the terminal, or may be stored in a server connected to a network such as the Internet and the above program may be executed on the server.

[0064] (Second Embodiment) FIG. 8 is a functional block diagram showing the generation process of route-related information in the second embodiment. The information processing unit 12 in this embodiment includes a route information generation unit 80, a danger level information generation unit 81, a route-related information generation unit 82, and a control unit 83. The operations of each part are the same as those of the corresponding parts in the first embodiment described above, but the difference from the first embodiment is that the danger level information can be fed back from the danger level information generation unit 81 to the route information generation unit 80, and the control unit 83 determines whether re-search is necessary.

[0065] FIG. 9 is a flowchart showing the generation processing procedure of route-related information in the second embodiment. The procedures from S91 to S95 after the start of the flow are the same as those in the first embodiment, but it is different from the first embodiment in that a procedure for determining the necessity of re-search (S96) is added.

[0066] In the second embodiment, after the risk information is generated in the same procedure as in the first embodiment, the control unit 83 determines whether re-search of the route is necessary (S96). This determination of necessity is made based on the risk information. For example, when there is a point (risk point) where the risk level exceeds (or is equal to or higher than) a preset threshold value, it is determined that re-search is necessary. Even if there is only one risk point, it may be determined to be necessary, or when the number of risk points exceeds (or is equal to or more than) a predetermined number, it may be determined to be necessary.

[0067] The determination of necessity in S96 may be made without using the risk information. For example, an upper limit on the number of times of re-search may be determined in advance, and if the number of times is less than the upper limit, it is determined to be no, and if the number of times reaches the upper limit, it is determined to be necessary. In this case as well, the risk information can be used in the re-search in S94.

[0068] When it is determined that re-search is necessary, the flow returns to S94, and the route information generation unit 80 generates route information again (S94). In the re-search of the route again, route information is generated with reference to the risk information fed back from the risk information generation unit 81 in addition to the object information, space information, and movement condition information.

[0069] Next, the risk information generation unit 81 generates risk information based on the object information, space information, and the route information after re-search (S95).

[0070] The procedures from S94 to S96 are repeated until it is determined in S96 that re-search is not necessary.

[0071] When it is determined that re-exploration is not necessary, the flow proceeds to S97, where the route-related information generation unit 82 generates route-related information from the generated route information and the risk level information, and displays it on the display unit 11A (S97).

[0072] FIG. 10 shows an example of the risk level information (information including the risk level associated with route (2)) generated after re-exploring the route. This is the risk level information generated in the first route exploration, which includes points with a high risk level as shown in FIG. 5, and it is determined that re-exploration is required. This is the risk level information generated after re-exploring the route. As shown in FIG. 10, for route (2), the risk level is a low value (10) at all points.

[0073] FIG. 11 shows an example of displaying the route information and the risk level information generated by the information processing unit 12 according to the second embodiment on the display unit 11A. The route (1) and the risk level associated with it are the results of the first route exploration, which are the same as the route (1) and the risk level described in FIG. 6. The route (2) and the risk level associated with it are the results of re-exploring the route (re-exploration).

[0074] Route (2) indicates that it proceeds along the passage downward from the starting point, turns right (makes a left turn) at the first corner, turns downward (makes a right turn) in front of the structure C, turns right (makes a left turn) within the work space E, turns upward (makes a left turn) at the dead end, and reaches the end point. Route (2) is longer than route (1), but since the passage width is sufficiently wider than the object, the risk level is displayed as 10.

[0075] When displaying the route-related information after re-exploration, only route (2) and the risk level associated with it may be displayed on the display unit 11A, or route (1) and the risk level associated with it may be displayed together on the display unit 11A.

[0076] Thus, according to the second embodiment, since the route information is generated using the risk level information in addition to the object information and the space information, it becomes possible to safely move the object by moving along a route from which the risk level has been excluded in advance.

[0077] (Third Embodiment) FIG. 12 is a functional block diagram showing the generation process of route-related information in the third embodiment. The information processing unit 12 in this embodiment includes a route information generation unit 120, a risk level information generation unit 121, a route-related information generation unit 122, and a control unit 123. The operations of each unit are the same as those of the corresponding units in the first embodiment described above, but the risk level information can be fed back from the risk level information generation unit 121 to the route information generation unit 120, the control unit 123 determines whether re-search is necessary, and a user instruction is input to the information processing unit 12, which are different from the first embodiment.

[0078] FIG. 13 is a flowchart showing the generation process procedure of route-related information in the third embodiment. The procedures from S131 to S136 after the start of the flow are the same as those of each procedure in the first embodiment, but it is different from the first embodiment in that a re-search necessity determination procedure (S137) is added.

[0079] In the third embodiment, after the route-related information is displayed on the display unit 11A in the same procedure as in the first embodiment, the control unit 123 determines whether re-search of the route is necessary (S137). This necessity determination is made based on an instruction from the user. For example, the information processing device 1 may display the route-related information on the display unit 11A and display a message such as "Do you want to re-search?" and accept the input of a user instruction to the input unit 11B. At this time, on the display unit 11A, an icon image or the like indicating the location of a point (risk point) where the risk level exceeds (or is equal to or higher than) a preset threshold value may be superimposed on the route-related information and displayed. Also, text indicating the location of the risk point or text indicating the number of risk points may be added to the above message.

[0080] When it is determined in S137 that re-search is required based on a user instruction, the flow returns to S134, and the route information generation unit 120 generates route information again (S134). In the re-route search, route information is generated with reference to the risk level information fed back from the risk level information generation unit 121 in addition to the object information, the space information, and the movement condition information.

[0081] Next, the risk level information generation unit 121 generates risk level information based on the object information, the space information, and the route information after the re-search (S135). The control unit 123 displays the re-created route-related information on the display unit 11A (S136), and determines whether re-search of the route is necessary based on a user instruction (S137).

[0082] The procedure from S134 to S137 is repeated until it is determined in S137 that re-search is not required.

[0083] When it is determined in S137 that re-search is not required based on a user instruction, the flow ends.

[0084] An example of the risk level information generated in the third embodiment is the same as that shown in FIG. 10, and the risk level is a low value (10) at all points on the route by the re-route search. Also, an example of displaying the route information and the risk level information generated in the third embodiment on the display unit 11A is the same as that shown in FIG. 11.

[0085] As described above, according to the third embodiment, when it is determined that re-search is required based on a user instruction, route information is generated using risk level information in addition to object information and space information. Therefore, a user with high transportation skills can use the shortest route without re-search, and a user with low transportation skills can safely move an object by moving along the route after re-search in which the risk level has been eliminated in advance.

[0086] (Fourth Embodiment) FIG. 14 is a functional block diagram showing a process for generating route-related information and caution information in the fourth embodiment. The information processing unit 12 in this embodiment includes a route information generation unit 140, a risk level information generation unit 141, a caution information generation unit 142, a route-related information generation unit 143, and a control unit 144. The operations of each unit except the caution information generation unit 142 are the same as those of the corresponding units in the first embodiment described above. However, the route information and the risk level information are output from the risk level information generation unit 141 to the caution information generation unit 142, the caution information is input to the route-related information generation unit 143, and the caution information is output from the route-related information generation unit 143, which are different from the first embodiment.

[0087] FIG. 15 is a flowchart showing a procedure for generating route-related information and caution information. From S151 to S155 after the start of the flow, it is the same as each procedure in the first embodiment.

[0088] After the risk level information is generated in S155, the caution information generation unit 142 generates caution information based on the object information, space information, route information, and risk level information (S156).

[0089] Next, the route-related information generation unit 143 generates route-related information from the generated route information and risk level information, and the control unit 144 displays the route-related information on the display unit 11A (S157).

[0090] Next, the control unit 144 generates caution information based on the object information, space information, route information, and risk level information, and displays it on the display unit 11A (S158).

[0091] Next, a specific method for generating caution information will be described.

[0092] The caution information is generated as explanatory text corresponding to each point on the route based on the object information, space information, route information, and risk level information.

[0093] FIG. 16 is a flowchart showing a procedure for generating caution information.

[0094] First, based on the route information and the coordinates of each point in the point cloud of spatial information, points existing on the route in the point cloud are extracted (S161).

[0095] Next, an explanatory text is derived based on the spatial information at each of the extracted points, the object information of the object passing through each point, and the degree of danger at each point (S162).

[0096] For deriving the explanatory text, an AI for deriving explanatory text equipped with a solution algorithm for input-output problems of deep learning is applied. The AI for generating explanatory text is machine-learned with the object information, the spatial information at each point in the point cloud of the space, and the risk information corresponding to each point as input and the explanatory data (correct answer data) as output. The point cloud of the space used for machine learning may be input all at once, or may be input serially for each region obtained by dividing the space into regions. When inputting for each region, the correct answer data is text data or the like that explains the points to note when an object passes through each region.

[0097] For deriving the explanatory text, a deep learning architecture capable of executing a generation AI task that fuses a three-dimensional space data analysis AI model such as 3D-LLM (Large Language Mode) and a large-scale language AI model may be applied. For example, 3D-VQA (Visual Question Answering), which returns a relatively short answer to a question sentence regarding the three-dimensional scene indicated by the point cloud of three-dimensional information, can be applied. Also, 3D-Captioning or the like that generates and outputs a simple explanatory text for a text sentence prompting an explanation of the three-dimensional scene can be applied. In recent years, AI models that have machine-learned various datasets created and accumulated so far and a large amount of information on the Internet have become available. A pre-trained AI model may be used as it is, but it is also possible to improve the performance (fine-tuning) using a training dataset in which the point cloud and the text sentence are paired.

[0098] Next, caution information is generated as data in which the coordinates of each of the extracted points are associated with the explanatory text of each point (S163).

[0099] FIG. 17 shows an example of the generated caution information. The columns of the route number and coordinates are the same as those of each item in FIG. 5. In the column of the explanatory text, the content to be noted at each point on the route (1) when the object moves is generated in a text that can be understood by the user. Note that the case of the coordinates (1, 0, h) is the same as that described in the explanations of FIGS. 5 and 6, and is an example in which explanatory text prompting attention is generated when the height of the object is h or more.

[0100] FIG. 18 shows an example of displaying the generated route information, risk level information, and caution information on the display unit 11A. The display of the route (1) and the associated risk level is the same as that in FIG. 6.

[0101] The caution information is displayed in a speech bubble including explanatory text. Since the first straight line of the route (1) has a passage width that is sufficiently wider than the object and a straight passage continues, explanatory text to that effect is displayed as the caution information.

[0102] In the section indicated as A1 with the component A, when the height of the moving object is higher than h and the upper part of the object is harder than the material of A1, explanatory text prompting attention to this location is displayed (dashed speech bubble). On the other hand, when the height of the object is lower than h, the corresponding caution information is not generated and the explanatory text at this location is not displayed either.

[0103] In the next straight line of the route (1), since the passage width immediately after the right turn becomes narrow, explanatory text conveying danger is displayed (solid speech bubble).

[0104] When the object information includes information indicating that the size of the object increases or decreases, the user can include information such as that the size of the object can be decreased at a point where the passage is narrow, for example. In that case, the caution information generation unit 142 may derive explanatory text such as "If the size of the object is decreased, the risk level will decrease to 20" for the second straight line route of the route (1).

[0105] FIG. 19 shows another example in which route information, risk level information, and caution information are displayed on the display unit 11A. In this example, a part of the passage on the route is displayed three-dimensionally. Since there is a large step in the passage on the route and there is a risk that the object will fall and be damaged if it proceeds as it is, the risk level of the step portion is shown as 80. Also, since the object is heavy, it is described in the explanatory text that a forklift or the like is required when lowering the object.

[0106] As described above, according to the fourth embodiment, caution information including text explaining the risk level and the like of each point on the route is generated based on the object information, space information, route information, and risk level information. Therefore, it becomes possible to ensure safety on the route before moving the object and to consider another route.

[0107] (Fifth Embodiment) FIG. 20 is a functional block diagram showing the generation process of route-related information and caution information in the fifth embodiment. The information processing unit 12 in the present embodiment includes a route information generation unit 200, a risk level information generation unit 201, a caution information generation unit 202, a route-related information generation unit 203, and a control unit 204. The operations of each unit are the same as those of the corresponding units in the fourth embodiment described above, but the difference from the fourth embodiment is that the risk level information can be fed back from the risk level information generation unit 201 to the route information generation unit 200, and the control unit 204 determines whether re-search is necessary.

[0108] FIG. 21 is a flowchart showing the generation process procedure of route-related information and caution information in the fifth embodiment. The procedures from S211 to S215 after the start of the flow are the same as those of each procedure in the fourth embodiment, but the difference from the fourth embodiment is that a procedure for determining whether re-search is necessary (S216) is added.

[0109] In the fifth embodiment, after the risk level information is generated in the same procedure as in the fourth embodiment, the control unit 204 determines whether re-search of the route is necessary (S216). This procedure for determining necessity is the same as the procedure of S96 in the second embodiment.

[0110] If it is determined that re-search is required, the flow returns to S214, and the route information generation unit 200 generates route information again (S214). In the re-route search, in addition to the object information, the space information, and the movement condition information, the route information is generated with reference to the risk information fed back from the risk information generation unit 201.

[0111] Next, the risk information generation unit 201 generates risk information based on the object information, the space information, and the route information after re-search (S215).

[0112] The procedures from S214 to S216 are repeated until it is determined in S216 that re-search is not required.

[0113] If it is determined that re-search is not required, the flow proceeds to S217, and the caution information generation unit 202 generates caution information (S217). The procedures from S218 to S219 are the same as the procedures from S157 to S148 in the fourth embodiment.

[0114] An example of the risk information generated in the fifth embodiment is the same as that shown in FIG. 10, and the risk is a low value (10) at all points on the route by the re-route search.

[0115] FIG. 22 shows an example of displaying the route information and the risk information generated by the information processing unit 12 according to the fifth embodiment on the display unit 11A. The content of FIG. 21 is the same as that of FIG. 18 except for the caution information about the route (2). Since the passage width of the route (2) is sufficiently wider than the object and the risk is 10, an explanatory text indicating that there is no problem is displayed as the caution information (the speech bubble of the solid line indicating the route (2)).

[0116] When displaying the route-related information after re-search, only the route (2) and the associated risk and caution information may be displayed on the display unit 11A, or the route (1) and the associated risk and caution information may be displayed together on the display unit 11A.

[0117] According to the fifth embodiment as described above, in addition to object information and space information, risk level information is used to generate route information with the risk level excluded in advance, and caution information including text explaining the risk level and the like of each point on the route is generated. Therefore, by checking the explanatory text about the route, the object can be moved with confidence.

[0118] (Sixth Embodiment) FIG. 23 is a functional block diagram showing the generation process of route-related information and caution information in the sixth embodiment. The information processing unit 12 in the present embodiment includes a route information generation unit 230, a risk level information generation unit 231, a caution information generation unit 232, a route-related information generation unit 233, and a control unit 234. The operations of each unit are the same as those of the corresponding units in the fourth embodiment described above, but the risk level information can be fed back from the risk level information generation unit 231 to the route information generation unit 230, the control unit 234 determines whether re-search is necessary, and a user instruction is input to the information processing unit 12, which are different from the fourth embodiment.

[0119] FIG. 24 is a flowchart showing the generation process procedure of route-related information and caution information in the sixth embodiment. The procedures from S241 to S248 after the start of the flow are the same as those of each procedure in the fourth embodiment, but it is different from the fourth embodiment in that a procedure for determining whether re-search is necessary (S249) is added.

[0120] In the sixth embodiment, after the route-related information and the caution information are displayed on the display unit 11A in the same procedure as in the fourth embodiment, the control unit 234 determines whether re-searching of the route is necessary (S249). This determination of necessity is made based on an instruction from the user. For example, the information processing apparatus 1 may display the route-related information and the caution information on the display unit 11A and also display a message such as "Do you want to re-search?" and accept an input of a user instruction to the input unit 11B. At this time, on the display unit 11A, an icon image or the like indicating the location of a point (hazard point) whose risk level exceeds (or is equal to or higher than) a threshold value set in advance may be superimposed on the route-related information and displayed. Also, the caution information at the hazard point may be displayed in a color that draws attention, such as red, or may be displayed blinking.

[0121] Based on the user instruction, if it is determined in S249 that re-searching is necessary, the flow returns to S244, and the route information generation unit 230 generates route information again (S244). In the re-search of the route, in addition to the object information, the space information, and the movement condition information, the risk level information fed back from the risk level information generation unit 231 is also referred to for generating the route information. An example of the risk level information generated in the sixth embodiment is the same as that shown in FIG. 10, and by the re-search of the route, the risk level becomes a low value (for example, 10) at all points on the route.

[0122] Next, the risk level information generation unit 231 generates risk level information based on the object information, the space information, and the route information after re-search (S245), and the caution information generation unit 232 generates caution information based on the object information, the space information, the route information after re-search, and the risk level information (S246). The control unit 234 displays the route-related information and the caution information created again on the display unit 11A (S247, S248), and determines whether re-searching of the route is necessary based on the user's instruction (S249).

[0123] The procedures from S244 to S249 are repeated until it is determined in S249 that re-searching is not necessary.

[0124] When it is determined in S249 that re-search is not required based on a user instruction, the flow ends.

[0125] An example of the risk information generated in the sixth embodiment is the same as that shown in FIG. 10, and by re-searching the route, the risk level is a low value (10) at all points on the route. Also, an example of displaying the route information, risk information, and caution information generated in the sixth embodiment on the display unit 11A is the same as that shown in FIG. 21.

[0126] As described above, according to the sixth embodiment, when it is determined that re-search is required based on a user instruction, route information and caution information are generated using risk information in addition to object information and space information. Therefore, a user with high transportation skills can use the shortest route without re-searching, and a user with low transportation skills can move the object safely and with peace of mind by referring to the route and caution information after re-searching with the risks eliminated in advance.

[0127] The program executed by the information processing apparatus 1 according to each of the above-described embodiments may be recorded on a computer-readable recording medium such as a CD-ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (Compact Disc Recordable), or a DVD (Digital Versatile Disc) in an installable or executable file format, and provided as a computer program product.

[0128] Furthermore, the program executed by the information processing apparatus 1 according to each embodiment may be stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Also, the program executed by the information processing apparatus 1 according to each embodiment may be configured to be provided or distributed via a network such as the Internet.

[0129] Note that the above-described embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. This novel embodiment and modification can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. This embodiment and modification are included in the scope and gist of the invention and are included in the invention described in the claims and the equivalent scope thereof.

[0130] Note that machine learning is a technology for enabling a computer to acquire learning capabilities like those of humans. It refers to a technology in which a computer autonomously generates an algorithm necessary for judgments such as data identification from pre-loaded learning data and applies this to new data for prediction. The learning method for machine learning may be any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a learning method that combines these learning methods, and the learning method for machine learning is not limited.

[0131] Aspects of the present invention are as follows, for example. <1> An information processing apparatus comprising: a storage unit that stores object information regarding a moving object and space information regarding the space in which the object moves; an input unit to which movement condition information indicating conditions regarding the movement of the object is input; and an information processing unit that generates route-related information including information regarding the movement of the object in the space, wherein the space information includes at least three-dimensional space information of the space, the movement condition information includes at least information on a starting point and an ending point where the object moves, and the information processing unit generates the route-related information based on the object information, the space information, and the movement condition information.

[0132] <2> The information processing apparatus according to <1>, wherein the object information includes any one of three-dimensional object information of the object, estimated information of the object estimated based on the three-dimensional object information, and input information of the object input from the outside.

[0133] <3> The spatial information includes any one of the three-dimensional spatial information of the space, the estimated information of the space estimated based on the three-dimensional spatial information, and the input information of the space input from the outside. The information processing apparatus according to <1>.

[0134] <4> The route-related information includes route information indicating a route from the start point to the end point and risk level information indicating the risk level on the route. The information processing apparatus according to any one of <1> to <3>.

[0135] <5> The information processing unit regenerates the risk level information based on the object information, the spatial information, and the movement condition information. The information processing apparatus according to <4>.

[0136] <6> The information processing unit generates caution information based on the object information, the spatial information, the movement condition information, and the risk level information. The information processing apparatus according to <4> or <5>, characterized in that.

[0137] <7> The information processing unit determines whether to regenerate the route information based on the generated risk level information. The information processing apparatus according to any one of <4> to <6>.

[0138] <8> A reading step of reading object information regarding an object to move and spatial information regarding a space in which the object moves; an input step of inputting movement condition information indicating conditions regarding the movement of the object; and an information processing step of generating route-related information including information regarding the movement of the object in the space. The spatial information includes at least the three-dimensional spatial information of the space, the movement condition information includes at least information on a start point and an end point where the object moves, and the generation step generates the route-related information based on the object information, the spatial information, and the movement condition information. An information processing method.

[0139] <9> The computer is made to function as a storage means for storing object information regarding a moving object and space information regarding the space in which the object moves, an input means for inputting movement condition information indicating conditions regarding the movement of the object, and an information processing means for generating route-related information including information regarding the movement of the object in the space. The space information includes at least three-dimensional space information of the space, the movement condition information includes at least information on the starting point and the ending point where the object moves, and the generating means generates the route-related information based on the object information, the space information, and the movement condition information. This is a program characterized by the above.

Explanation of Signs

[0140] 1 Information processing device 10 Storage unit 11 UI unit 12 Information processing unit 20 Route information generation unit 21 Danger level information generation unit 22 Route-related information generation unit 23 Control unit

Prior Art Documents

Patent Documents

[0141]

Patent Document 1

Claims

1. A storage unit that stores object information regarding a moving object and space information regarding the space in which the object moves; An input unit to which movement condition information indicating conditions regarding the movement of the object is input; An information processing unit that generates route-related information including information regarding the movement of the object in the space, and; The space information includes at least three-dimensional space information of the space, The movement condition information includes at least information on a starting point and an ending point where the object moves, The information processing unit generates the route-related information based on the object information, the space information, and the movement condition information. An information processing device.

2. The object information according to claim 1, including any one of three-dimensional object information of the object, estimated information of the object estimated based on the three-dimensional object information or a luminance image, and input information of the object input from the outside.

3. The information processing device according to claim 1, wherein the space information further includes any one of estimated information of the space estimated based on the three-dimensional space information or a luminance image, and input information of the space input from the outside.

4. The route-related information according to any one of claims 1 to 3, including route information indicating a route from the starting point to the ending point and risk level information indicating a risk level on the route.

5. The information processing device according to claim 4, wherein the information processing unit regenerates the risk level information based on the object information, the space information, and the movement condition information.

6. The information processing device according to claim 4, wherein the information processing unit generates caution information based on the object information, the space information, the movement condition information, and the risk level information.

7. The information processing unit according to claim 4 determines whether to regenerate the route information based on the generated risk degree information.

8. A reading step of reading object information regarding a moving object and space information regarding the space in which the object moves; An input step of inputting movement condition information indicating conditions regarding the movement of the object; An information processing step of generating route-related information including information regarding the movement of the object in the space, the information processing method comprising: The space information includes at least three-dimensional space information of the space; The movement condition information includes at least information on a starting point and an ending point where the object moves; The information processing step generates the route-related information based on the object information, the space information, and the movement condition information.

9. A computer, A storage means for storing object information regarding a moving object and space information regarding the space in which the object moves; An input means for inputting movement condition information indicating conditions regarding the movement of the object; An information processing means for generating route-related information including information regarding the movement of the object in the space, functioning as: The space information includes at least three-dimensional space information of the space; The movement condition information includes at least information on a starting point and an ending point where the object moves; The information processing means generates the route-related information based on the object information, the space information, and the movement condition information.

Images