Information processing device, information processing method, and program

Abstract

A path evaluation device 100 according to the present disclosure comprises: a calculation unit 121 that calculates, for each partition image obtained by partitioning an image into a plurality of regions, condition information representing a condition of a road surface shown in the partition image on the basis of the partition image; and an evaluation unit 122 that evaluates a path formed in the image by connecting prescribed partition images, said evaluation being made on the basis of the conditions of the road surface shown in the partition images included in the path. For example, by calculating the road surface condition information from images using a model generated by machine learning, evaluating paths, and providing such paths to a user, it is possible to assist with path decision-making by the user.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Patent Literature 1 describes a method for automatically generating a movement path for a mobile object such as a robot from an input image. Specifically, Patent Literature 1 identifies free space areas and obstacle areas from the image, calculates the cost of movement, and searches for a movement path.

[0003] Japanese Patent Application Laid-Open No. 2005-92820

[0004] However, the technology described in the above-mentioned Patent Document 1 searches for a travel route by taking only obstacles into consideration, and does not take other circumstances into consideration. As a result, it is unclear whether the searched route is appropriate, and it is difficult to search for an appropriate route.

[0005] Therefore, an object of the present disclosure is to solve the above-mentioned problem that it is difficult to search for a more appropriate route from an image.

[0006] An information processing device according to one aspect of the present disclosure includes: a calculation unit that calculates, for each divided image obtained by dividing an image into a plurality of regions, situation information representing the situation of the road surface reflected in the divided image based on the divided image; and an evaluation unit that evaluates a route formed on the image by connecting predetermined divided images based on the situation information of the road surface reflected in the divided images included in the route. Also, an information processing method according to one aspect of the present disclosure includes: calculating, for each divided image obtained by dividing an image into a plurality of regions, situation information representing the situation of the road surface reflected in the divided image based on the divided image; and evaluating a route formed on the image by connecting predetermined divided images based on the situation information of the road surface reflected in the divided images included in the route. Furthermore, a program that is one form of the present disclosure is configured to cause a computer to execute the following process: for each divided image obtained by dividing an image into multiple regions, calculate situation information that represents the condition of the road surface reflected in the divided image based on the divided image; and evaluate a route formed on the image by connecting predetermined divided images based on the situation information of the road surface reflected in the divided images included in the route.

[0007] With the above-described configuration, the present disclosure can search for a more appropriate route from an image.

[0008] FIG. 1 is a diagram showing a usage status of the present disclosure. FIG. 2 is a block diagram showing a configuration of a first information processing device according to the present disclosure. FIG. 3 is a diagram showing a processing state by a first information processing device according to the present disclosure. FIG. 4 is a diagram showing a processing state by a first information processing device according to the present disclosure. FIG. 5 is a diagram showing a processing state by a first information processing device according to the present disclosure. FIG. 6 is a diagram showing a processing state by a first information processing device according to the present disclosure. FIG. 7 is a flowchart showing a processing operation of a first information processing device according to the present disclosure. FIG. 8 is a block diagram showing a configuration of a third information processing device according to the present disclosure. FIG. 9 is a diagram showing a processing state by a third information processing device according to the present disclosure. FIG. 10 is a block diagram showing a hardware configuration of a fifth path evaluation device according to the present disclosure. FIG. 11 is a block diagram showing a configuration of a fifth path evaluation device according to the present disclosure.

[0009] First Embodiment A first embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any of the embodiments.

[0010] [Configuration] The information processing device 10 in this embodiment functions as a route evaluation device that evaluates a route that a moving object will take based on an image. As an example, in this embodiment, the moving object M is a robot that carries a package B at a predetermined location P, such as a warehouse. As shown in FIG. 1 , when the moving object M moves from its current location to a destination E, the information processing device 10 acquires an image of the location P in the warehouse captured by an image capture device C and evaluates a route from the image to the destination E. However, the moving object M is not limited to a robot that carries a package B, and may also be a person or a vehicle operated or ridden by a person. In other words, the information processing device 10 can be applied to evaluating routes that any moving object may take, and can also be applied to routes related to travel for any purpose.

[0011] The information processing device 10 is composed of one or more information processing devices each including a calculation device and a storage device. As shown in FIG. 2 , the information processing device 10 includes a travel area detection unit 11, a road surface condition calculation unit 12, a route evaluation unit 13, an output unit 14, and a control unit 15. The functions of the travel area detection unit 11, the road surface condition calculation unit 12, the route evaluation unit 13, the output unit 14, and the control unit 15 can be realized by the calculation device executing a program for realizing each function stored in the storage device. The information processing device 10 also includes an image storage unit 16 and a route information storage unit 17. The image storage unit 16 and the route information storage unit 17 are each configured using a storage device. Each component will be described in detail below.

[0012] The image storage unit 16 acquires an image of a predetermined location P within the warehouse where the moving object M may move, photographed by the photographing device C, and stores the image in the image storage unit 16. At this time, the photographing device C is, for example, a stereo camera (RGB sensor), an RGB-D sensor, or a LiDAR (Light Detection and Ranging) that can photograph an image including distance information to an object. The photographing device C is installed above the predetermined location P so as to photograph an image of the predetermined location P viewed from above. Therefore, the image shows the road surface on which the wheels and other legs of the moving object M touch the ground.

[0013] It should be noted that a plurality of image capturing devices C may be installed. In this case, a plurality of image capturing devices C of the same type using the same image capturing method may be installed, or one or more of each of a plurality of types of image capturing devices C may be installed. The image capturing device C may be installed anywhere, for example, on a moving body M. As an example, if the moving body M is a dolly, the image capturing device C may be attached to a handle provided on the dolly facing slightly downward, thereby capturing images of the road surface in the same manner as described above. In this case, only one image capturing device C may be installed on the moving body M, and by using a technique such as Structure from Motion (SfM), the road surface at the above-mentioned predetermined location P can be sequentially converted into a three-dimensional image (point cloud) from images captured at a plurality of times.

[0014] Furthermore, the images stored in the image storage unit 16 are updated at regular time intervals or at any timing with new images captured by the image capture device C. As a result, the images can become images that represent the latest or most recent state at the specified location P. Note that the images stored in the image storage unit 16 and that are to be processed as described below are not necessarily limited to images that include distance information, and may be any images.

[0015] As shown in FIG. 3 (3-1), the passage area detection unit 11 reads out an image G stored in the image storage unit 16 and uses the image G to detect a passage area in which a moving object M can pass at a location P corresponding to the image G. Specifically, the passage area detection unit 11 detects areas in the image that form substantially the same plane, and detects the area as a passage area. As an example, the passage area detection unit 11 extracts a subset of points in the captured image as individual objects, and extracts objects such as horizontal and vertical planes based on distance information in the image. Then, the passage area detection unit 11 detects a horizontal plane having a predetermined width from the image, excluding objects that form step portions including vertical planes, as a passage area. As an example, the passage area detection unit 11 detects a passage area Q, as shown by the shaded area in FIG. 3 (3-2), from the image shown in FIG. 3 (3-1). The passage area detection unit 11 may simply detect the widest area of ​​approximately the same horizontal plane as the passage area based on the distance image included in the image, or may detect it using other methods. Also, the passage area detection unit 11 does not need to be provided, and the following processing may be performed using the entire image as the passage area.

[0016] The road surface condition calculation unit 12 (calculation unit) calculates condition information that represents the condition of the road surface in the travel area of ​​the image G. In this embodiment, the road surface condition information is calculated as undulation condition information that represents the undulation condition of the road surface, i.e., the amount of undulation of the road surface. Specifically, as shown in FIG. 4 , the road surface condition calculation unit 12 divides the image G into a plurality of regions and calculates the amount of undulation of the road surface for each divided image g that is each divided region. Note that while FIG. 4 shows the image G, a moving object M located at a predetermined location P captured in the image G and a destination E are illustrated corresponding to the image G.

[0017] The road surface condition calculation unit 12 calculates the amount of undulation relative to the aforementioned travel area surface based on the distance information included in the segmented image g. Specifically, the road surface condition calculation unit 12 sets the travel area surface to a height of 0, calculates the difference between the height of each of the highest and lowest points in the segmented image g and the travel area surface, and calculates the greater absolute value of the difference as the amount of undulation x_i for each segmented image g(i). Therefore, if a point higher than the travel area surface is included, the amount of undulation is positive, and if a point lower than the travel area surface is included, the amount of undulation is negative, which can be recognized as a convex obstacle or a concave obstacle, respectively. In this case, for example, if the road surface shown in the segmented image g has deep grooves or a large gradient, the difference in elevation will be large, and the absolute value of the amount of undulation will be calculated to be large. On the other hand, if the road surface shown in the segmented image g is flat, the absolute value of the amount of undulation will be calculated to be small. Therefore, in this embodiment, the greater the actual amount of undulation, the greater the absolute value of the amount of undulation. In this embodiment, the value of the amount of undulation itself is expressed as a positive or negative value, but when referring to the magnitude of the amount of undulation, it represents the magnitude of the undulation, that is, the height or depth relative to the surface of the passage area, and is expressed as a value obtained by adding an absolute value to the value of the amount of undulation. Note that the road surface condition calculation unit 12 may predefine a reference elevation difference diff_base at which the amount of undulation is 1.0, and may use a value (normalized value) obtained by dividing the elevation difference diff calculated for each divided image g by the reference elevation difference diff_base as the amount of undulation x_i of each divided image g(i). The road surface condition calculation unit 12 then associates the amount of undulation calculated for each divided image g with the corresponding divided image g and stores it in the image storage unit 16.

[0018] The road surface condition calculation unit 12 may calculate the above-mentioned amount of undulation using information other than the distance information included in the above-mentioned divided image g. For example, the road surface condition calculation unit 12 may acquire topographical information of a predetermined location P in the warehouse corresponding to image G consisting of the above-mentioned divided images g, and calculate the amount of undulation using the topographical information and the distance information included in the divided image g. Here, the topographical information of the predetermined location is assumed to be a three-dimensional model including information on the surrounding environment, such as the placement of objects and the positions of walls and pillars. By using such information, for example, objects for which distance information could not be measured in image G can be extracted from the topographical image and associated with the divided image g, and the amount of undulation in the divided image g can be calculated, including the objects.

[0019] FIG. 5 shows the state when the above-described undulation amount is calculated for each divided image g. FIG. 5 illustrates multiple divided area images a corresponding to multiple divided images g, and an area image A, which is the entire divided area image a, can correspond to image G. Each divided area image a is filled with a predetermined color, and the color, i.e., the gradation value, represents the undulation amount. That is, each divided area image a is displayed with a gradation value corresponding to the undulation amount calculated for the corresponding divided image g. In the example of FIG. 5, the darker the gradation value, the greater the undulation amount. This indicates that most of image G has a small undulation amount, but there are areas with a large undulation amount in places. Note that FIG. 5 illustrates area image A corresponding to image G, and a moving object M and a destination E located at a predetermined location P captured in image G are illustrated corresponding to area image A.

[0020] The road surface condition calculation unit 12 may also calculate the undulation direction of the road surface as the above-mentioned road surface undulation condition. Specifically, for each divided image g, the road surface condition calculation unit 12 calculates the slope direction from the lowest point to the highest point in the divided image g as the undulation direction based on the distance information included in the divided image g. Then, the road surface condition calculation unit 12 associates the undulation direction calculated for each divided image g with the corresponding divided image g and stores it in the image storage unit 16. Note that the road surface condition calculation unit 12 may calculate the above-mentioned undulation direction using information other than the distance information included in the above-mentioned divided image g. For example, the road surface condition calculation unit 12 may calculate the undulation direction using topographical information of the above-mentioned predetermined location P and the distance information included in the divided image g. By using such information, for example, objects for which distance information could not be measured in image G can be extracted from the topographical image and associated with the divided image g, and the undulation direction in the divided image g can be calculated including the object.

[0021] Here, the road surface condition calculation unit 12 may calculate road surface conditions such as the amount and direction of undulation of the road surface from the divided image g using a model previously constructed by machine learning. In this case, the model is generated by machine learning learning data in which the divided image g is associated with the amount and direction of undulation. Then, by inputting the divided image g to the model, the road surface condition calculation unit 12 can obtain the amount and direction of undulation corresponding to the divided image g as output. In this case, the image G constituting the divided image g does not necessarily need to include distance information.

[0022] The road surface condition calculation unit 12 is not limited to calculating information representing the road surface condition, such as the amount and direction of undulation of the road surface, using the method described above, and may calculate the information using any method. As an example, the road surface condition calculation unit 12 may calculate the normal to each point on the road surface from the distance information included in the divided image g, and calculate the amount and direction of undulation from the variance of the normal.

[0023] The route evaluation unit 13 (evaluation unit) generates one or more routes on the image G along which the moving object M can travel, and evaluates each route by calculating an evaluation value for each route. At this time, the route evaluation unit 13 calculates the evaluation value for the route based on the amount of undulation in the divided image g included in the route. As an example, the route evaluation unit 13 calculates a "score," which is an evaluation value that represents the ease of movement of the moving object M along the route, using the following formula 1. Here, the magnitude of the amount of undulation, i.e., the absolute value of the amount of undulation, is a value greater than or equal to 0, and the greater the actual undulation, the larger the value.

[0024] Here, the evaluation value "score" is assumed to be a value in the range of 0≦score≦1, and according to the above formula 1, the greater the average undulation of the route, the smaller the value (approaching 0), indicating that travel is more difficult, and the smaller the average undulation of the route, the larger the value (approaching 1), indicating that travel is easier. Then, the route evaluation unit 13 associates the calculated evaluation value with the position information of the divided images g located on the route for each route and stores them in the route information storage unit 17.

[0025] The route evaluation unit 13 may calculate the route evaluation value using a method different from that described above. For example, the route evaluation unit 13 may calculate the evaluation value based on the maximum amount of undulation on the route. In this case, if the maximum amount of undulation on the route exceeds a threshold set for the moving body M, the route evaluation unit 13 may calculate a smaller evaluation value or may calculate the route as being unevaluable. This may be because the amount of undulation is an undulation that the moving body M cannot overcome in the first place, or an undulation that may cause inconvenience such as the possibility of the luggage B falling. Taking such circumstances into consideration, the route evaluation unit 13 may calculate a smaller evaluation value or may calculate the route as being unevaluable, as described above.

[0026] Furthermore, the route evaluation unit 13 may calculate the evaluation value of the route based on the undulation direction in the divided images g included in the route, in addition to the amount of undulation described above. For example, if there is a divided image g in which the undulation increases in the traveling direction of the route, the route evaluation unit 13 calculates a lower evaluation value score for the route. On the other hand, if there is a divided area g in which the undulation decreases in the traveling direction of the route, the route evaluation unit 13 calculates a higher evaluation value score for the route. Note that the route evaluation unit 13 may calculate the evaluation value score for the route based only on the undulation direction in the divided images g included in the route.

[0027] The route evaluation unit 13 then selects the shortest route and settable routes as candidates for routes that the moving object M can travel on the image G, and calculates an evaluation value for each route. At this time, the route evaluation unit 13 may determine the route with the best evaluation value, or may determine several routes with the highest evaluation values ​​and store them in the route information storage unit 17. At this time, the route with the best evaluation value can be determined using, for example, the Dijkstra algorithm.

[0028] Here, the road surface condition calculation unit 12 may calculate the undulation conditions, such as the amount and direction of undulation, of the road surface for a new divided image g' obtained by changing the size of the divided area of ​​the divided image g in image G, in the same manner as described above. For example, FIG. 6 (6-1) shows a divided image g corresponding to a part of image G, and FIG. 6 (6-2) shows a new divided image g' obtained by changing the size of the divided image g. FIG. 6 (6-1') shows a gradation value corresponding to the amount of undulation in each divided area image a of the area image A corresponding to FIG. 6 (6-1), and FIG. 6 (6-2') shows a gradation value corresponding to the amount of undulation in each divided area image a' of the area image A corresponding to FIG. 6 (6-2). Here, in the divided image g in FIG. 6 (6-1), a side view of the road surface corresponding to the divided image g is shown above, and two deep cracks, as indicated by the symbol f, exist on the road surface corresponding to the divided image g. In this case, as shown in Figure 6 (6-1'), the divided area image a located between two cracks f is calculated to have a small amount of undulation, but as shown in Figure 6 (6-2'), in the new divided area image a' corresponding to the new divided area image g' in which the divided area is expanded, the two cracks f are located within one divided area image a', and the amount of undulation can be calculated to be large.

[0029] Then, in response to the change in the divided area as described above, the route evaluation unit 13 calculates an evaluation value for the route formed on the new divided image g'. That is, for the route formed by connecting the new divided images g', the route evaluation unit 13 calculates an evaluation value based on the undulation conditions, such as the amount of undulation and the direction of undulation, calculated from the new divided image g'. In this way, when the size of the divided images g, g' is changed, evaluation values ​​for the routes formed before and after the change can be obtained. For example, when there are moving bodies M of different sizes (e.g., the size of legs such as wheels) and it is desired to consider a route that takes into account the safety of movement, the divided images g, g' may be changed to various sizes as described above, and various routes may be evaluated in each case.

[0030] The output unit 14 (display unit) displays information obtained through the above-described process on an information processing terminal of a user, such as an administrator managing the movement of the moving object M. For example, as shown in FIG. 7 , the output unit 14 displays the route for which the evaluation value has been calculated superimposed on an area image A corresponding to an image G. That is, the output unit 14 displays the route as a line superimposed on each divided area image a corresponding to each divided image g included in the route. As an example, in FIG. 7 , the output unit 14 displays the route R1 having the highest evaluation value, i.e., the route evaluated as the easiest to travel, as a solid line, and displays the route R2 having a lower evaluation value but the shortest distance as a dotted line. Note that the output unit 14 may display the top several routes or routes selected based on other criteria. Furthermore, the output unit 14 may display information based on the evaluation results together with the route. For example, as shown in FIG. 7 , the output unit 14 may display the evaluation value score together with the route, or may display the evaluation ranking or information describing the characteristics of the route.

[0031] Furthermore, the output unit 14 may display the calculated amount of undulation for the corresponding divided image g on the divided area image a in addition to displaying the above-mentioned route. In the example of FIG. 7 , the output unit 14 displays the divided area image a by filling it with a color with a gray value corresponding to the amount of undulation. In this case, a darker gray value indicates a larger amount of undulation. Furthermore, when the output unit 14 calculates the undulation direction for each divided image g as described above, the output unit 14 may display the undulation direction on the divided area image a corresponding to the divided image g so that the undulation direction can be seen. Here, FIG. 8 (8-1) shows an example of the divided area image a. The amount of undulation may be displayed on the divided area image a using gray values, and the undulation direction may be displayed using an arrow. In this case, the arrow indicates that the undulation increases in the direction of the arrow. Furthermore, the output unit 14 may display the undulation direction on the divided area image a using gray values. For example, the gray values ​​are displayed so that they become darker in the direction from low to high undulation areas. Figures 8 (8-2) to (8-4) each show an example of the undulation direction displayed using gray values, with the lower diagrams showing a schematic representation of the road surface condition. Figure 8 (8-2) shows a case where the amount of undulation increases from right to left, Figure 8 (8-3) shows a case where the road surface is convex in the center, and Figure 8 (8-4) shows a case where the road surface is concave in the center. However, the output unit 14 may display the undulation direction in any display format. Furthermore, the output unit 14 may display other road surface conditions different from the above-mentioned amount of undulation and undulation direction.

[0032] The output unit 14 may display the area image A made up of the divided area images a as an image G made up of the divided images g, as described above. In other words, the output unit 14 may display to the user an image G of a predetermined location P in the warehouse that was actually photographed, and may also superimpose on the image G the route, the amount of undulation, the direction of undulation, and the like.

[0033] The control unit 15 outputs a control command to control the movement of the moving object M so that the moving object M moves along the route evaluated as described above within a predetermined location P in the warehouse corresponding to the image G. For example, the control unit 15 controls the moving object M to move along the route R1 with the best evaluation value, or controls the moving object M to move along the selected route in response to a route selection by the user who displayed the route. As a result, the moving object M moves to the destination E in accordance with the control command. Note that the control unit 15 does not necessarily have to be provided, and the moving object M may be operated by a user or the like.

[0034] [Operation] Next, the operation of the above-mentioned information processing device 10 will be described. First, the information processing device 10 detects a passage area from an image G of a predetermined location P in a warehouse where a moving object M can move, which is captured by the imaging device C (step S1 in FIG. 9). For example, the information processing device 10 detects, as the passage area Q, a horizontal plane shown by the shaded area in FIG. 3 (3-2), which is obtained by extracting and removing steps and the like from the image G as shown in FIG. 3 (3-1).

[0035] Next, the information processing device 10 divides the image G into a plurality of regions as shown in Fig. 4 (step S2 in Fig. 9 ), and calculates the amount of undulation of the road surface shown in the image for each divided region, that is, divided image g (step S3 in Fig. 9 ). For example, based on the distance information included in the divided image g, the information processing device 10 calculates the amount of undulation as the greater absolute value of the difference between the highest point and the lowest point in the divided image g relative to the height of the reference travel area surface, as described above. Alternatively, the information processing device 10 may calculate the amount of undulation of the road surface from the divided image g using a model previously constructed by machine learning.

[0036] At this time, the information processing device 10 may calculate, for each divided image g, the undulation direction, which is the slope direction from the low point to the high point within the divided image g, based on the distance information contained in the divided image g, and may also calculate other road surface conditions.

[0037] The information processing device 10 then searches for one or more routes on the image G along which the moving object M can travel, and calculates and evaluates each route (step S4 in FIG. 9 ). The information processing device 10 evaluates the route so that the greater the amount of undulation in the divided image g included in the route, the smaller the evaluation value "score" representing the ease of travel along the route, and the smaller the amount of undulation in the divided image g included in the route, the larger the evaluation value "score". The information processing device 10 may also calculate the evaluation value of the route based on the direction of undulation in the divided image g included in the route. Furthermore, the information processing device 10 may calculate the undulation conditions, such as the amount and direction of undulation of the road surface, for a new divided image g' obtained by changing the size of the divided region of the divided image g described above in image G, in the same manner as described above, and calculate the evaluation value of the route formed on the new divided image g' (steps S3 and S4 in FIG. 9 ).

[0038] The information processing device 10 then outputs the evaluated route to an information processing terminal of a user, such as an administrator managing the movement of the mobile object M, to display the evaluated route (step S5 in FIG. 9 ). For example, as shown in FIG. 7 , the information processing device 10 displays the routes R1 and R2 for which the evaluation values ​​have been calculated, superimposed on an area image A corresponding to image G. At this time, the information processing device 10 may display not only the route R1 with the highest evaluation value, but also other routes (e.g., the route R2 with the shortest distance) and may also display the evaluation value score. Furthermore, the information processing device 10 may display the amount of undulation and the direction of undulation on the divided area image a corresponding to the divided area image a in addition to displaying the route. For example, as shown in FIG. 7 , the divided area image a may be displayed with a color corresponding to the amount of undulation, and the direction of undulation may be displayed with an arrow or a gray value.

[0039] Furthermore, the information processing device 10 controls the movement operation of the moving object M so that the moving object M moves along the evaluated route within a predetermined location P in the warehouse corresponding to the image G (step S6 in FIG. 9). Note that the movement of the moving object M may be controlled by a user or the like.

[0040] As described above, the information processing device 10 of this embodiment calculates the amount and direction of undulation of the road surface shown in the image, and evaluates the route according to the undulation. In other words, the ease of movement is evaluated according to the undulations present on the route. Therefore, it is possible to search for a route suitable for movement according to the moving body M, such as a robot.

[0041] <Embodiment 2> Embodiment 2 of the present disclosure will be described. The information processing device 10 in this embodiment has a configuration substantially similar to that of the information processing device described in the above-described embodiment. In addition, the information processing device 10 in this embodiment has a configuration that evaluates a route by taking into account road surface conditions such as the amount and direction of undulation of the road surface as well as road surface conditions such as the slipperiness that indicates how slippery the road surface is. Below, configurations that differ from the above-described embodiment will be mainly described in detail.

[0042] In this embodiment, the road surface condition calculation unit 12 (calculation unit) uses the segmented images g of image G in the same manner as described above to calculate, for each segmented image g, slipperiness information representing the slipperiness of the road surface. For example, the road surface condition calculation unit 12 extracts feature amounts of the segmented images g, i.e., feature amounts due to the texture of the road surface reflected in the segmented images g, and calculates a slipperiness value representing the slipperiness of the road surface from the feature amounts. Specifically, the road surface condition calculation unit 12 uses a model previously constructed by machine learning to input the segmented images g into the model and obtains an output value of the slipperiness of the road surface. In this case, the model is generated by machine learning learning data in which the segmented images g are associated with slipperiness values. As a result, for example, if the road surface reflected in the segmented image g has a marble-like texture, the slipperiness value representing the slipperiness is calculated to be high, and if the road surface reflected in the segmented image g has an asphalt-like texture, the slipperiness value representing the slipperiness is calculated to be low. As another example, if the road surface reflected in the segmented image g has a wet texture, the slipperiness value representing the slipperiness is calculated to be high. The value of the sliding property is set, for example, between 0 and 1, with a higher value indicating greater slipperiness. The road surface condition calculation unit 12 is not limited to calculating the sliding property of the road surface using the above-described method, and may calculate the property using any method.

[0043] The route evaluation unit 13 (evaluation unit) in this embodiment then calculates an evaluation value "score" for the route by taking into account the amount of undulation and the direction of undulation as well as the sliding property calculated as described above. As an example, the route evaluation unit 13 calculates "score," which is an evaluation value that represents the ease of movement of the moving object M along the route, using the following formula 2. Note that while formula 2 uses only the amount of undulation as the undulation condition of the road surface, the evaluation value "score" may also be calculated by using the direction of undulation as described above.

[0044] According to the above formula 2, the evaluation value "score" becomes smaller (approaching 0) as the average smoothness of the route increases, indicating that movement is more difficult, and the evaluation value "score" becomes larger (approaching 1) as the average smoothness of the route decreases, indicating that movement is easier. Then, the route evaluation unit 13 stores the calculated evaluation value together with the position information of the divided images g located on the route in the route information storage unit 17, in association with each other, for each route.

[0045] The route evaluation unit 13 may calculate the route evaluation value using a method different from that described above. For example, the route evaluation unit 13 may calculate the evaluation value based on the maximum slidability on the route. In this case, if the maximum slidability on the route exceeds a threshold value set according to the moving body M, the route evaluation unit 13 may calculate a smaller evaluation value or calculate the route as being impossible to evaluate. This is because the slidability may be so slippery that the moving body M will slip and be unable to move. Taking this into consideration, the route evaluation unit 13 may calculate a smaller evaluation value or calculate the route as being impossible to evaluate, as described above.

[0046] As described in embodiment 1, the road surface condition calculation unit 12 and the route evaluation unit 13 may also calculate the road surface frictionality in the same manner as described above for a new divided image g' obtained by changing the size of the divided area of ​​the divided image g in image G, and use this frictionality to calculate the route evaluation value.

[0047] Then, the output unit 14 displays the evaluated route and the evaluation value together with the route, similar to the first embodiment. Furthermore, the control unit 15 may output a control command to control the movement of the moving object M so that the moving object M moves along the evaluated route within a predetermined location P in the warehouse corresponding to the image G, similar to the first embodiment.

[0048] As described above, the information processing device 10 of this embodiment calculates the road surface roughness, which indicates the slipperiness of the road surface, in addition to the undulation conditions, such as the amount and direction of undulation, of the road surface shown in the image, and evaluates the route based on these conditions. In other words, the ease of movement is evaluated based on the undulations and slipperiness conditions present on the route. Therefore, it is possible to search for a route suitable for movement based on the moving body M, such as a robot.

[0049] Third Embodiment A third embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any of the embodiments.

[0050] 10, the information processing device 10 of this embodiment has substantially the same configuration as the information processing device described in the above-mentioned embodiments, and also has a mobile object information storage unit 18 configured with a storage device. The information processing device 10 of this embodiment has a function of calculating an evaluation value of a route in accordance with the characteristics of the mobile object M. The following mainly describes in detail the configuration that differs from the other embodiments.

[0051] In this embodiment, the mobile object M may be, for example, a person, a cart or stroller operated by a person, a wheelchair or smart mobility vehicle carried by a person, or multiple automated guided vehicles with different wheel diameters. In this case, the information processing device 10 acquires mobile object information that indicates the characteristics of the mobile object M that may move along a route and stores it in the mobile object information storage unit 18. For example, the mobile object information is input to the information processing device 10 from the terminal of a user who wishes to search for a route. Examples include a case where a user wishes to search for a route when heading to a certain destination, or a case where a user who manages the transportation of cargo in a warehouse wishes to search for a route for transportation using a specific automated guided vehicle.

[0052] The mobile object information is information that represents the characteristics of the mobile object M, and includes the type of the mobile object. The type of mobile object is information that represents the method of movement of the mobile object, such as walking (bipedal walking), cart / stroller (bipedal walking and wheeled movement), wheelchair / smart mobility (wheeled movement), and automated guided vehicle (wheeled movement). The mobile object information may also include information that represents the characteristics of each type of mobile object. For example, if the mobile object is a person and the method of movement is walking, the mobile object information may include information that represents the person's physical characteristics, such as the person's gender, age, and medical history. If the method of movement is wheeled movement, the mobile object information may also include information about the wheels of the mobile object, such as the size (diameter and thickness) of the wheels. If the method of movement is wheeled movement, the mobile object information may also include information about the presence or absence of a transported object and its weight. However, the above-described mobile object information is merely an example, and is not limited to the above information, and may include any information that represents the characteristics of the mobile object.

[0053] The route evaluation unit 13 (evaluation unit) in this embodiment calculates an evaluation value for each route based on the undulations and sliding properties of the segmented images g on the route, as described in other embodiments, and also based on the above-mentioned moving body information. For example, if the moving body information indicates walking (bipedal walking), the route evaluation unit 13 thins out the segmented images g included in the route and calculates the evaluation value for the route using the undulations and other parameters of the remaining segmented images g. This is because, when a person walks bipedally, their feet touch the ground at predetermined distances depending on their stride, and they may cross a number of segmented images g corresponding to their stride. As an example, the route evaluation unit 13 thins out every other segmented image g on the route, calculates the average undulations of all the remaining segmented images g on the route, and calculates the evaluation value "score" from this average undulation. The amount of thinning of the segmented images g on the route is set according to the physical characteristics of the person included in the moving body information. For example, the information processing device 10 stores stride information corresponding to the physical characteristics of a person in advance, sets the stride based on the gender, age, etc. included in the moving body information, and sets the thinning amount corresponding to the stride. For this reason, as an example, the thinning amount is set smaller for women than for men, the older the person is, and further, the smaller the thinning amount is set if the person has a specific type of medical history.

[0054] The route evaluation unit 13 may also change the method of calculating the route evaluation value depending on, for example, the characteristics included in the mobile object information. For example, if the mobile object information includes a specific medical history (e.g., a knee injury, a cardiopulmonary disorder, etc.) as a physical characteristic of a person, the evaluation value score may be calculated by multiplying the evaluation value score by a coefficient α (0≦α≦1) corresponding to the specific medical history, or by multiplying the undulations of each divided image g by a coefficient β (1≦β) corresponding to the specific medical history. In this way, the route evaluation value is calculated to be low depending on the person's characteristics, such as their medical history, and the route is evaluated taking into account the risks involved in travel. As another example, if the mobile object information includes information such as the size of wheels, the presence or absence of a transported object, and its weight, the evaluation value or the undulations may be calculated by multiplying a coefficient (weight) depending on the size, presence or absence of a transported object, and its weight. For example, the evaluation value may be calculated by assigning a greater weight to the undulations when the wheels are smaller, or when the transported object is present or is heavy. However, the above-described method of calculating the evaluation value by the route evaluation unit 13 is one example, and the evaluation value may be calculated using another calculation method according to the mobile object information.

[0055] As described in the first embodiment, the calculation of the evaluation value by the route evaluation unit 13 taking into consideration the moving body information is also applied to a new divided image g' obtained by changing the size of the divided area of ​​the divided image g in the image G. That is, the route evaluation unit 13 may also calculate an evaluation value for a route set on the new divided image g' whose size has been changed, taking into consideration the moving body information in the same manner as described above.

[0056] Then, the output unit 14 displays the evaluated route, as in the first embodiment. For example, when a user requests a route to travel to a destination by walking, the route for which the evaluation value has been calculated is superimposed on an area image A corresponding to image G and displayed on the terminal T of the user U, as shown in Fig. 11. As an example, in Fig. 11, the output unit 14 displays a route R1 having the best evaluation value, that is, evaluated as the easiest route to travel, with a solid line, and displays a route R2 having a lower evaluation value but the shortest distance with a dotted line, and further displays the evaluation value "score" together with the route.

[0057] As described above, the information processing device 10 in this embodiment evaluates a route based on the characteristics of the moving object in addition to the undulation conditions, such as the amount and direction of undulation of the road surface shown in the image. Therefore, it is possible to search for a route suitable for travel depending on the moving object, such as a person or a robot.

[0058] Fourth Embodiment A fourth embodiment of the present disclosure will be described. An information processing device 10 in this embodiment is configured to be used in the healthcare field to provide a menu optimized for a user. Here, as an example, a case will be described in which a user requests a search for a travel route for exercise, such as a jogging route or a walking route.

[0059] First, the information processing device 10 acquires user information (person information) representing the characteristics of the user from the user who is a moving object. For example, the acquired user information includes information representing the user's exercise-related characteristics, such as the user's gender, age, health condition, and exercise preferences. The health condition is information representing the user's health condition, such as sleep time and fatigue level. The exercise preferences are information representing the user's desired exercise, such as exercise intensity (e.g., hard running, gentle jogging, walking, etc.), exercise distance and time, and exercise goals (e.g., hill training, flat areas, avoiding crowded places, etc.).

[0060] The information processing device 10 also acquires information on the user's current location and exercise start location, and acquires images of locations where a route can be set based on the location information. For example, the information processing device 10 acquires and stores images such as database images, satellite images, and security camera images provided on the Internet.

[0061] Then, as described above, the information processing device 10 calculates conditions such as the amount of undulations on the road surface from the image, and calculates an evaluation value for a route that is a candidate for exercise such as jogging by the user. Here, while in the other embodiments described above the evaluation value for a route represents ease of movement, in this embodiment, since the route is one for movement involving exercise such as walking or running, a value representing exercise load is calculated from the evaluation value. Therefore, in this embodiment, the higher the ease of movement, which is the evaluation value described above, the lower the exercise load value calculated, and the lower the ease of movement, the higher the exercise load value calculated. In other words, in this embodiment, the greater the amount of undulations on the route, the higher the exercise load of the route calculated, and the smaller the amount of undulations on the road, the lower the exercise load of the route calculated.

[0062] The information processing device 10 sets multiple routes and calculates the exercise load for each route. Then, the information processing device 10 selects a route optimized for the user based on user information that represents the user's exercise characteristics. For example, if the user desires hard running as their exercise intensity or hill training as their exercise goal, the information processing device 10 selects a travel route with a high exercise load. On the other hand, if the user desires a flat location as their exercise goal, the information processing device 10 selects a travel route with a low exercise load. The information processing device 10 may also select a travel route taking into account the user's health condition, age, and other characteristics. For example, if the user's health condition is high in fatigue or they are older, the information processing device 10 may select a travel route with a low exercise load that does not require them to overexert themselves.

[0063] The information processing device 10 then outputs a route optimized for the user based on the user information as described above to the user's information processing terminal for display. This allows the user to obtain exercise information, which is a walking or running route that is suitable for the user. The information processing device 10 may also output multiple routes along with information on the exercise load for each route to the user's information processing terminal for display. The user can then select their own travel route by referring to the multiple routes and the exercise load values. In this way, the information processing device 10 in this embodiment can support the user's decision-making.

[0064] Fifth Embodiment Next, a fifth embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any of the embodiments. In this embodiment, an outline of the configuration of the information processing device described in the above-described embodiments is shown.

[0065] First, the hardware configuration of the path evaluation device 100 will be described with reference to Fig. 12. The path evaluation device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, for example: - CPU (Central Processing Unit) 101 (arithmetic device) - ROM (Read Only Memory) 102 (storage device) - RAM (Random Access Memory) 103 (storage device) - Programs 104 loaded into the RAM 103 - Storage device 105 storing the programs 104 - Drive device 106 for reading and writing data from and to a storage medium 110 external to the information processing device - Communication interface 107 for connecting to a communication network 111 external to the information processing device - Input / output interface 108 for inputting and outputting data - Bus 109 for connecting the various components

[0066] 12 shows an example of the hardware configuration of the information processing device that is the path evaluation device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with a part of the above-described configuration, such as not including the drive device 106. Furthermore, instead of the above-described CPU, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point Number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof.

[0067] The path evaluation device 100 can be equipped with the calculation unit 121 and evaluation unit 122 shown in FIG. 13 by having the CPU 101 acquire and execute the program group 104. The program group 104 is stored in advance in the storage device 105 or the ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read out the program and supply it to the CPU 101. However, the calculation unit 121 and evaluation unit 122 described above may be constructed using dedicated electronic circuits for realizing such means.

[0068] The calculation unit 121 calculates, for each divided image obtained by dividing an image into a plurality of regions, situation information representing the situation of the road surface reflected in the divided image based on the divided image. The evaluation unit 122 evaluates a route formed on the image by connecting a predetermined number of the divided images based on the situation information of the road surface reflected in the divided images included in the route.

[0069] With the above-described configuration, the present disclosure calculates condition information of the road surface shown in the input image and evaluates the route based on the condition information. This makes it possible to evaluate the route taking into account conditions such as the undulations and friction of the road surface, and to search for an appropriate route.

[0070] In addition, at least one of the functions of the calculation unit 121 and the evaluation unit 122 described above may be executed by an information processing device installed and connected anywhere on the network, that is, they may be executed by so-called cloud computing.

[0071] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-RWs, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program can also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0072] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each of the above-described embodiments can be combined with other embodiments as appropriate.

[0073] <Supplementary Notes> Some or all of the above embodiments can also be described as in the following supplementary notes. Below, an outline of the configurations of a route evaluation device, a route evaluation method, and a program according to the present disclosure will be described. However, the present disclosure is not limited to the following configurations. (Supplementary Note 1) A route evaluation device comprising: a calculation unit that calculates, for each divided image obtained by dividing an image into a plurality of regions, situation information that represents the situation of a road surface reflected in the divided image, based on the divided image; and an evaluation unit that evaluates a route formed on the image by connecting predetermined divided images, based on the situation information of the road surface reflected in the divided images included in the route. (Supplementary Note 2) The route evaluation device according to Supplementary Note 1, wherein the calculation unit calculates, as the undulation method, undulation situation information that represents the undulation situation of the road surface reflected in the divided image, based on the divided images, and the evaluation unit evaluates the route based on the undulation situation information of the road surface reflected in the divided images. (Supplementary Note 3) The route evaluation device according to Supplementary Note 2, wherein the calculation unit calculates an amount of undulation of the road surface shown in the segmented image as the undulation information. (Supplementary Note 4) The route evaluation device according to Supplementary Note 2, wherein the calculation unit calculates an undulation direction of the road surface shown in the segmented image as the undulation information. (Supplementary Note 5) The route evaluation device according to Supplementary Note 3, wherein the evaluation unit evaluates the route including the segmented image so that the greater the amount of undulation calculated for the segmented image, the lower the evaluation of the route. (Supplementary Note 6) The route evaluation device according to Supplementary Note 2, wherein the calculation unit calculates sliding property information indicating sliding property that indicates how slippery the road surface shown in the segmented image is, based on the segmented image, and the evaluation unit evaluates the route based on the undulation and the sliding property of the road surface shown in the segmented image. (Supplementary Note 7) The route evaluation device according to Supplementary Note 6, wherein the evaluation unit evaluates the route including the segmented image so that the higher the value of the sliding property information calculated for the segmented image, the lower the evaluation of the route including the segmented image.(Supplementary Note 8) The route evaluation device according to Supplementary Note 1, wherein the calculation unit calculates, for each of the divided images, the situation information of the road surface shown in the divided image based on the divided image and information different from the image associated with the location corresponding to the divided image. (Supplementary Note 9) The route evaluation device according to Supplementary Note 1, wherein the calculation unit calculates, for each of the new divided images obtained by changing the size of the area, the situation information of the road surface shown in the new divided image based on the new divided image, and the evaluation unit evaluates the route formed on the image by connecting predetermined new divided images based on the situation information of the road surface shown in the new divided image included in the route. (Supplementary Note 10) The route evaluation device according to Supplementary Note 1, wherein the evaluation unit evaluates the route based on moving object information indicating characteristics of moving objects moving on the route. (Supplementary Note 11) The route evaluation device according to Supplementary Note 10, wherein the evaluation unit evaluates the route based on the moving object information indicating a type of the moving object. (Supplementary Note 12) The route evaluation device according to Supplementary Note 10, wherein the evaluation unit evaluates the route based on the moving body information representing physical characteristics of a person, which is the moving body. (Supplementary Note 13) The route evaluation device according to Supplementary Note 1, comprising a display unit that displays an area image made up of a plurality of divided area images corresponding to a plurality of the divided images, and displays the route in association with the divided area image corresponding to the divided image included in the route on the area image. (Supplementary Note 14) The route evaluation device according to Supplementary Note 13, wherein the display unit displays information based on an evaluation result of the route. (Supplementary Note 15) The route evaluation device according to Supplementary Note 13, wherein the display unit displays, on the divided area image, information corresponding to the situation information calculated for the divided image corresponding to the divided area image. (Supplementary Note 16) The route evaluation device according to Supplementary Note 1, comprising a control unit that controls a movement operation of the moving body so that the moving body moves according to the route at a location corresponding to the image.(Supplementary Note 17) The route evaluation device according to Supplementary Note 1, comprising an output unit that outputs the route optimized for the person based on person information that represents characteristics related to the person's movement and load information that represents the load imposed when moving on the route based on an evaluation result of the route. (Supplementary Note 18) The route evaluation device according to Supplementary Note 1, wherein the calculation unit acquires the situation information output from a model generated by machine learning learning data in which the prepared segmented images are associated with the situation information by inputting the segmented images into the model. (Supplementary Note 19) A route evaluation method that calculates, for each segmented image obtained by dividing an image into a plurality of regions, situation information that represents the situation of a road surface reflected in the segmented image, and evaluates a route formed on the image by connecting predetermined segmented images, based on the situation information of the road surface reflected in the segmented images included in the route. (Supplementary Note 20) A computer-readable storage medium storing a program that causes a computer to execute the following processes: for each divided image obtained by dividing an image into a plurality of regions, calculate situation information that represents the condition of the road surface reflected in the divided image based on the divided image; and evaluate a route formed on the image by connecting predetermined divided images based on the condition of the road surface reflected in the divided images included in the route.

[0074] REFERENCE SIGNS LIST 10 Information processing device 11 Passage area detection unit 12 Road surface condition calculation unit 13 Route evaluation unit 14 Output unit 15 Control unit 16 Image storage unit 17 Route information storage unit 18 Mobile object information storage unit C Camera M Mobile object A Area image a, a' Divided area image G Image g, g' Divided image 100 Route evaluation device 101 CPU 102 ROM 103 RAM 104 Program group 105 Storage device 106 Drive device 107 Communication interface 108 Input / output interface 109 Bus 110 Storage medium 111 Communication network 121 Calculation unit 122 Evaluation unit

Claims

1. A calculation unit calculates situational information representing the road surface conditions shown in each divided image obtained by dividing an image into multiple regions, based on that divided image. An evaluation unit that evaluates a path formed on an image by concatenating predetermined divided images, based on the condition information of the road surface reflected in the divided images included in the path, A route evaluation device equipped with the following features.

2. A route evaluation device according to claim 1, The calculation unit calculates the undulation information representing the undulation of the road surface as seen in the divided image, based on the divided image, as the condition information. The evaluation unit evaluates the route based on the undulation information of the road surface shown in the segmented image. Route evaluation device.

3. A route evaluation device according to claim 2, The calculation unit calculates the amount of undulation of the road surface shown in the segmented image as the undulation condition information. Route evaluation device.

4. A route evaluation device according to claim 2, The calculation unit calculates the direction of the undulations of the road surface shown in the segmented image as the undulation condition information. Route evaluation device.

5. A route evaluation device according to claim 3, The evaluation unit evaluates the path including the divided image such that the larger the amount of relief calculated for the divided image, the lower the evaluation of the path. Route evaluation device.

6. A route evaluation device according to claim 2, The calculation unit calculates sliding properties information representing the slipperiness of the road surface shown in the divided image based on the divided image, The evaluation unit evaluates the route based on the undulation information and sliding properties information of the road surface shown in the segmented image. Route evaluation device.

7. A route evaluation device according to claim 6, The evaluation unit evaluates the path including the divided image such that the higher the value of the sliding properties information calculated for the divided image, the lower the evaluation of the path including the divided image. Route evaluation device.

8. A route evaluation device according to claim 1, The calculation unit calculates the road surface condition information shown in each of the divided images based on the divided image and information different from the image associated with the location corresponding to the divided image. Route evaluation device.

9. The information processing device is For each segmented image obtained by dividing the image into multiple regions, situational information representing the road surface conditions shown in that segmented image is calculated based on that segmented image. The path formed on the image by concatenating predetermined segmented images is evaluated based on the road surface condition information reflected in the segmented images included in the path. Route evaluation method.

10. For each segmented image obtained by dividing the image into multiple regions, situational information representing the road surface conditions shown in that segmented image is calculated based on that segmented image. The path formed on the image by concatenating predetermined segmented images is evaluated based on the road surface condition information reflected in the segmented images included in the path. A program that instructs a computer to perform a process.