Information processing device, information processing method, and program product

Abstract

This technology relates to an information processing device, method, and program capable of developing a route plan that avoids crowds. It uses crowd information to generate a cost map indicating the risk of traversing an area. This technology can be applied, for example, to controlling unmanned aerial vehicles (UAVs) such as UAV traffic management (UTM).

Description

Technical Field

[0001] The present technology relates to an information processing device, an information processing method, and a program, and, for example, to an information processing device, an information processing method, and a program that can plan a route that avoids crowds. Background Art

[0002] For example, Patent Document 1 proposes an information providing device that narrows down object information to be distributed to a mobile body.

[0003] The information providing device in Patent Document 1 acquires object information from external recognition results of an external recognition device installed on a mobile body or in an environment in which the mobile body is operating, and generates and stores an object map by associating the object information with map information. Furthermore, the information providing device searches for objects along a route based on the route information acquired from the mobile body, calculates the predicted time at which the mobile body will reach each searched object, and then determines the probability of the object's presence at that predicted time. The device then distributes object information to the mobile body only for objects with a probability of presence equal to or greater than a predetermined value.

[0004] Furthermore, the information providing device of Patent Document 1 can acquire, as object information, future events from a schedule of events such as loading and unloading of a dump truck as a mobile object.

[0005] Citation List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2018-185596 Summary of the Invention

[0008] Problems to be solved by the present invention

[0009] As a path planning for a moving object, it is safe to formulate a path planning that avoids crowds if possible.

[0010] The present technology has been made in view of such circumstances, and its purpose is to make it possible to formulate a path planning that avoids crowds.

[0011] Solution to the problem

[0012] The information processing device or program of the present technology is an information processing device including a cost map generation unit that generates a cost map indicating the risk of passing through an area by using crowd information, or a program for causing a computer to function as such an information processing device.

[0013] The information processing method of the present technology is an information processing method including generating a cost map indicating the risk of passing through an area by using crowd information.

[0014] In the information processing device, information processing method, and program of the present technology, a cost map indicating the risk of passing through an area is generated using crowd information.

[0015] Note that the information processing device may be an independent device or an internal block constituting one device.

[0016] Furthermore, the program can be provided by recording it on a recording medium or transmitting it via a transmission medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a diagram illustrating an outline of a UTM as an information processing device to which the present technology is applied.

[0018] Figure 2 is a diagram presenting an example of information used to generate a costmap.

[0019] Figure 3 is a block diagram presenting a configuration example of the UTM 10 .

[0020] Figure 4 is a flowchart illustrating an example of the processing of the UTM 10 .

[0021] Figure 5 : is a flowchart illustrating an example of a process of generating an integrated map (final cost map) in step S12 .

[0022] Figure 6 is a diagram illustrating an example of generating an image individual map in the individual map generating unit 32 .

[0023] Figure 7 is a diagram illustrating an example of registering costs in an image-individual map.

[0024] Figure 8 is a diagram for explaining the limitation of resetting the cost of the event area during an event.

[0025] Figure 9 is a diagram illustrating an example of generating a position information individual map in the individual map generating unit 33 .

[0026] Figure 10 is a diagram illustrating an example of generating a weather information individual map in the individual map generating unit 34 .

[0027] Figure 11 is a diagram illustrating an example of generating a final map in the integration unit 35 .

[0028] Figure 12 is a flowchart presenting an example of processing performed by the UTM 10 on an image individual map.

[0029] Figure 13is a flowchart presenting an example of processing performed by the UTM 10 on the weather information individual map.

[0030] Figure 14 is a flowchart presenting an example of processing performed by the UTM 10 on the position information individual map.

[0031] Figure 15 is a block diagram presenting a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0032] <Example of UTM to which the prior art is applied>

[0033] Figure 1 It is a diagram illustrating an outline of a UTM as an information processing device to which the present technology is applied.

[0034] exist Figure 1 In the present invention, UAV traffic management (UTM) 10 generates a cost map in which the cost indicating the risk of passing through an area is registered by using crowd information about a crowd. Then, using the cost map, UTM 10 generates a path plan for a mobile object (e.g., drone 11, which is an unmanned aerial vehicle (UAV)) and transmits the path plan to drone 11, thereby controlling drone 11.

[0035] Note that in this embodiment, the drone 11 as a UAV is used as a mobile body for which a path plan is to be generated, but a path plan can be generated for a mobile body moving, for example, underwater or on land (surface), a mobile body moving in outer space, etc.

[0036] The UTM 10 can generate a cost map using information from a single pattern or multiple patterns. The single pattern or at least one of the multiple patterns includes information from which crowd information can be generated. In this embodiment, the information from which crowd information can be generated includes, for example, images captured by the drone 11 (described later), images captured by the surveillance camera 13, location information of mobile terminals, and event information.

[0037] Note that the information from which crowd information can be generated is not limited to images, position information of mobile terminals, and event information.

[0038] exist Figure 1 In the present invention, the UTM 10 acquires an image, location information of a mobile terminal, weather information about weather, and event information about an event where people gather, as information of various types of patterns.

[0039] For example, the UTM 10 acquires images from one or more drones 11 flying at various locations controlled by the UTM 10. The drone 11 is equipped with, for example, a camera 11A, and transmits images captured by the camera 11A to the UTM 10. In this way, the UTM 10 receives images captured by the drone 11 (equipped with the camera 11A) and transmitted from the drone 11.

[0040] The UTM 10 acquires images captured by, for example, monitoring cameras installed in various places, particularly, for example, monitoring cameras 13 installed near a path in a path plan generated for the drone 11 controlled by the UTM 10 .

[0041] The UTM 10 acquires location information of a mobile terminal from a smartphone, for example. As the location information of the smartphone, for example, global positioning system (GPS) information acquired by a GPS function equipped on the smartphone in a format used in Google Maps by Google Inc. may be used.

[0042] For example, if the drone 11 is equipped with an anemometer (wind speed sensor), the UTM 10 acquires the wind speed measured by the anemometer as weather information. Specifically, if the drone 11 is equipped with an anemometer, the drone 11 transmits the wind speed measured by the anemometer to the UTM 10 as weather information. In this manner, the UTM 10 receives weather information transmitted from the drone 11. Note that weather information can be acquired not only from the drone 11 but also from sensors installed in various locations that detect physical quantities related to weather, such as anemometers.

[0043] The UTM 10 obtains event information, for example, from a web server. Specifically, the UTM 10 receives event information from the web server regarding events that have actually occurred or will occur near a route, for example, in a route plan generated for the drone 11 controlled by the UTM 10. The event information includes the location (place of occurrence) and the date and time of the event.

[0044] The UTM 10 acquires not only images and weather information from the drone 11 but also position information of the current position of the drone 11. Furthermore, the UTM 10 acquires not only images from the monitoring camera 13 but also position information of the location where the monitoring camera 13 is installed.

[0045] The UTM 10 generates crowd information using the information acquired by the UTM 10, and sets a cost indicating the risk of passing through the area for an area using the crowd information. Then, the UTM 10 generates a cost map in which the cost of the area is registered.

[0046] exist Figure 1 , generates a cost map where the cost is set for the area on the bird's-eye view map. Additionally, in Figure 1 In the middle, three levels of cost are used: low, medium, and high. The higher the cost, the higher the risk of passing through the area for which the cost is set. Note that in addition to the three levels of cost, two levels of low and high, or four or more levels of cost can also be used as the cost.

[0047] For example, the cost map is a map in which a cost indicating the risk of passing through each area is registered for an area of ​​a predetermined range.

[0048] In the cost map, for example, the entire earth can be adopted as an area for which costs are registered (hereinafter also referred to as a registration area), and when the flight range of the drone 11 is set in advance, the area of ​​the flight range can be adopted. In addition, an area set by the administrator of the UTM 10 or the like can be adopted as the registration area.

[0049] In the cost map, for example, for divided areas in which a registration area is divided into areas of a predetermined size, a cost indicating a risk of passing through each divided area is registered.

[0050] Note that in the cost map, for example, for a registration area in which a high object such as a mountain having an elevation equal to or greater than a predetermined value or a building having a height equal to or greater than a predetermined height exists, it is possible to register in advance that it is impossible to pass through the registration area (impassable area).

[0051] The UTM 10 generates or updates the cost map periodically or irregularly. Furthermore, the UTM 10 generates or regenerates a new path plan for the UAV 11 using the cost map as needed, and transmits the path plan to the UAV 11.

[0052] Since the cost map is generated using crowd information, the UTM 10 can develop a route plan that avoids crowds.

[0053] Furthermore, since the cost map is generated periodically or irregularly, by using the latest cost map to generate path planning, the drone 11 can fly flexibly according to the latest situation (ideally in real time).

[0054] Figure 2 is a view presenting an example of information used to generate a costmap.

[0055] Figure 2 Information, content or form, and application of the information for generating a cost map are presented.

[0056] Images captured by the drone 11 and the surveillance camera 13 (hereinafter also referred to as captured images) can be used to generate a cost map. As the form (format) of the captured images, any image form can be adopted. The captured images can be used to detect a crowd and set the cost of the area where the captured images appear.

[0057] The position information of the drone 11 and the surveillance camera 13 can be used to generate a cost map. As the form of the position information of the drone 11 and the surveillance camera 13, coordinates representing the position, such as latitude, longitude, etc., can be adopted. The position information of the drone 11 and the surveillance camera 13 can be used to register (designate the position of the area where its set cost is) the cost set using the captured images captured by the drone 11 and the surveillance camera 13 into the cost map.

[0058] The weather information measured (observed) by the drone 11 can be used to generate a cost map. As the weather information, for example, the measured value of a wind speed meter (wind speed sensor) equipped on the drone 11 can be adopted. The measured value of the wind speed meter as the weather information can be used to estimate the risk of the drone 11 flying at the point where the measured value (wind speed) is measured, and set the cost corresponding to the risk.

[0059] The position information of a smartphone as a mobile terminal can be used to generate a cost map. As the position information of the smartphone, GPS information obtained through the GPS function equipped on the smartphone can be adopted. The position information of the smartphone can be used to detect a crowd in an area not captured by the camera 11A equipped on the drone 11 and the surveillance camera 13, etc., and set the cost of the area where the crowd exists.

[0060] The event information can be used to generate a cost map. As the event information, information including the location (coordinates) of the event and the date and time of the event can be adopted. For example, the event information can be used to determine (estimate) what type of crowd is detected from the captured images captured by the drone 11 or the cause of the crowd, such as whether it is transient or due to an all-day event.

[0061] <Configuration example of UTM 10>

[0062] Figure 3 is a presentation Figure 1 block diagram of the configuration example of UTM 10 in

[0063] In Figure 3 UTM 10 has a cost map generation unit 21 and a path planning unit 22.

[0064] The cost map generation unit 21 uses information of one or more types of patterns to generate a cost map and supplies the cost map to the path planning unit 22.

[0065] The information used by the cost map generation unit 21 to generate the cost map includes information from which crowd information can be generated.

[0066] The cost map generation unit 21 generates crowd information by using at least information from which crowd information can be generated among information of one or more types of patterns, and generates a cost map by using the crowd information.

[0067] The cost map generating unit 21 has an information receiving unit 31 , respective map generating units 32 , 33 , and 34 , and an integrating unit 35 .

[0068] By receiving the imaging image, weather information, current position information of the drone 11 , and the like from the drone 11 , the information receiving unit 31 acquires them.

[0069] Furthermore, by receiving the imaged image, the position information of the monitoring camera 13 , and the like from the monitoring camera 13 , the information receiving unit 31 acquires them.

[0070] Furthermore, by receiving position information of a plurality of smartphones 51 from smartphones 51 carried by a plurality of people, the information receiving unit 31 acquires them.

[0071] Furthermore, by accessing the web server 52 and searching for event information, the information receiving unit 31 acquires it.

[0072] Here, since the information acquired by the information receiving unit 31 is used to generate a cost map, the information is also referred to as map information hereinafter.

[0073] The information receiving unit 31 supplies necessary information for a map to necessary blocks in the respective map generating units 32 to 34 .

[0074] For example, the information receiving unit 31 supplies the individual map generating unit 32 with the imaged image, position information of the imaged image (position information of the drone 11 or the surveillance camera 13 that has imaged the imaged image), and event information.

[0075] Furthermore, the information receiving unit 31 supplies the position information of the plurality of smartphones 51 to the individual map generating unit 33 .

[0076] Furthermore, the information receiving unit 31 supplies the weather information to the individual map generating unit 34 .

[0077] By performing image processing on the imaged image from the information receiving unit 31, the individual map generating unit 32 generates an image individual map which is an individual cost map of the image (imaged image) as information of the first mode and supplies it to the integrating unit 35.

[0078] When generating an individual map of an image, the individual map generation unit 32 uses, as needed, the position information of the imaged image and the event information from the information receiving unit 31.

[0079] By performing position information processing on the position information of the smart phone 51 from the information receiving unit 31, the individual map generation unit 33 generates an individual map of position information as the individual cost map of (the smart phone 51) and supplies it to the integration unit 35 as information in the second mode.

[0080] By performing weather information processing on the weather information from the information receiving unit 31, the individual map generation unit 34 generates an individual map of weather information as the individual cost map of weather information and supplies it to the integration unit 35 as information in the third mode.

[0081] The integration unit 35 integrates the image individual map from the individual map generation unit 32, the position information individual map from the individual map generation unit 33, and the weather information individual map from the individual map generation unit 34, and generates an integrated map as the final cost map and supplies it to the path planning unit 22.

[0082] The path planning unit 22 uses the integrated map from the integration unit 35 to generate a path plan and transmits it to the drone 11.

[0083] <Processing of UTM 10>

[0084] Figure 4 Is illustrated Figure 3 is a flowchart of an example of the processing of UTM 10.

[0085] In step S <subscript>11, in UTM 10, the information receiving unit 31 of the cost map generation unit 21 waits for a certain period of time (time) to pass and receives the information for the map, and the process advances to step S <subscript>12.

[0086] In step S <subscript>12, using the information for the map, the cost map generation unit 21 generates an integrated map as the final cost map and supplies it to the path planning unit 22, and the process advances to step S <subscript>13.

[0087] In step S <subscript>13, the path planning unit 22 determines whether it is necessary to generate or regenerate a path plan.

[0088] If it is determined in step S <subscript>13 that there is no need to generate and regenerate a path plan, the process returns to step S <subscript>11, and similar processing is repeated thereafter.

[0089] Furthermore, in the event that it is determined in step S13 that it is necessary to generate or regenerate the path plan, the process proceeds to step S14.

[0090] In step S14 , the path planning unit 22 generates a path plan using the latest cost map from the cost map generating unit 21 , and the process proceeds to step S15 .

[0091] For example, when the three levels of low, medium, and high costs are adopted as the costs, the path planning unit 22 only allows the path to pass through the low-cost area on the cost map and searches for one or more paths to the destination.

[0092] In a case where it is possible to reach the destination by a path that allows passing through only areas with low costs and the path length is equal to or less than an allowable length as a pre-permitted length (for example, a predetermined number of straight-line distances from the current location to the destination, etc.), the path planning unit 22 selects the path with the smallest length from among the searched paths as the generated result of the path planning.

[0093] In a case where it is impossible to reach the destination with a path that allows passing through only areas with low costs, or in a case where the path length exceeds the allowable length, in addition to the cost areas on the cost map, the path planning unit 22 also allows passing through areas with intermediate costs and searches for one or more paths to the destination.

[0094] If it is possible to reach the destination using a path that allows passage through both low-cost areas and medium-cost areas, the path planning unit 22 selects the path with the shortest length for passing through the medium-cost areas from among the searched paths as the generated result of the path planning. Alternatively, the path planning unit 22 selects the path with the shortest length for passing through the medium-cost areas and the shortest total length as the generated result of the path planning. For example, the path planning unit 22 selects the path with the shortest length for passing through the medium-cost areas and the weighted added value of the path length as the generated result of the path planning.

[0095] If it is impossible to reach the destination using only a path that allows passage through low-cost areas and medium-cost areas, the path planning unit 22 abandons path planning. Alternatively, the path planning unit 22 searches for one or more paths to the destination that allow passage through low-cost areas, medium-cost areas, and high-cost areas on the cost map. The path planning unit 22 then selects the path with the shortest length for passing through high-cost areas from among the searched paths as the generated path planning result.

[0096] In step S15, the path planning unit 22 transmits the path plan generated in the previous step S14 to the drone 11. Then, the process returns from step S15 to step S11, and similar processes are repeated thereafter.

[0097] Figure 5 It is an icon Figure 4 Flowchart of an example of a process of generating an integrated map (final cost map) in step S12 of FIG.

[0098] In step S21, the information receiving unit 31 of the cost map generating unit 21 will Figure 4 Necessary information among the information for maps received in step S11 is supplied to necessary blocks in the individual map generating units 32 to 34. The individual map generating units 32 to 34 respectively generate image individual maps, position information individual maps, and weather information individual maps by using the information for maps from the information receiving unit 31 and supply them to the integrating unit 35, and the process proceeds from step S21 to step S22.

[0099] In step S22, the integration unit 35 integrates the image individual map from the individual map generation unit 32, the position information individual map from the individual map generation unit 33, and the weather information individual map from the individual map generation unit 34, and generates an integrated map as the final cost map. The integration unit 35 then supplies the integrated map to the path planning unit 22, and the process ends.

[0100] <Generation of Image Individual Map>

[0101] Figure 6 is a diagram illustrating an example of generating an image individual map in the individual map generating unit 32 .

[0102] Using the imaged images imaged by the drone 11 and the surveillance camera 13 , the individual map generation unit 32 detects a crowd of people appearing in the imaged images, and generates crowd information including (information indicating) an area where the crowd of people is present.

[0103] As a method of detecting a crowd using an imaged image, for example, it is possible to adopt a method of detecting people from an imaged image and detecting a gathering of people having a density equal to or greater than a predetermined density as a crowd based on, for example, the degree to which people gather.

[0104] In addition, as a method of detecting a crowd using an imaged image, it is possible to adopt a method of detecting a crowd by performing learning of a neural network using, for example, an image in which a crowd appears and an image in which no crowd appears and a label indicating the presence or absence of a crowd as learning data, and giving the imaged image as an input to the learned neural network.

[0105] When a crowd is detected from the imaged image, that is, when a crowd appears in the imaged image, the individual map generation unit 32 detects an area where the crowd exists as a crowd area where the crowd exists and generates crowd information indicating the crowd area.

[0106] Furthermore, using the crowd information, the individual map generation unit 32 sets a cost indicating the risk of the drone 11 passing through the crowd area indicated by the crowd information.

[0107] For example, the individual map generation unit 32 sets the cost of the crowd area by thresholding the area according to the area of ​​the crowd area in the real world (not the area of ​​the crowd area in the imaged image). Note that the cost can be set according to the area of ​​the crowd area in the real world or according to the area of ​​the crowd area in the imaged image.

[0108] For example, three levels of risk, high, medium, and low, from high to low, are adopted as costs, and a first area threshold and a second area threshold greater than the first area threshold are adopted as two area thresholds.

[0109] If the area of ​​the crowd region (in the real world) is equal to or less than the first area threshold, the cost is set to low. If the area of ​​the crowd region is greater than the first area threshold and equal to or less than the second area threshold, the cost is set to medium. If the area of ​​the crowd region is greater than the second area threshold, the cost is set to high.

[0110] The cost of the area other than the crowd area in the area appearing in the image for imaging (hereinafter also referred to as the imaging area) is set to low, where the risk is the lowest, or to undetermined. Hereinafter, the description of the setting of the cost of the area other than the crowd area will be omitted.

[0111] Note that, for example, the cost can be set based on the (average) density of people in the crowd area, etc. For example, the cost can be set to a value with a higher risk as the density of people increases. In addition, for example, the cost can also be set based on the area of ​​the crowd area and the density of people in the crowd area.

[0112] The individual map generating unit 32 sets a cost for a crowd area and then generates an image individual map by registering the cost into a cost map.

[0113] Figure 7 is a diagram illustrating an example of registering costs in an image-individual map.

[0114] Using the images imaged by the drone 11 and the surveillance camera 13, the individual map generation unit 32 sets the cost, such as Figure 6 As described in.

[0115] Furthermore, the individual map generation unit 32 detects imaging areas (in the real world) that appear in the imaged images of the drone 11 and the surveillance camera 13 .

[0116] Here, in addition to obtaining imaged images, etc. from the drone 11 and the surveillance camera 13, the information receiving unit 31 also obtains the location information (for example, latitude, longitude), altitude (elevation) information and camera information (viewing angle, resolution, imaging direction, etc.) of the drone 11 and the surveillance camera 13.

[0117] The individual map generation unit 32 detects (the position and range of) an imaging area appearing in an imaged image using the position information, altitude information, and camera information of the drone 11 and the monitoring camera 13 .

[0118] Then, the individual map generation unit 32 registers, for each divided area, in the registration area of ​​the image individual map, a cost set using an image regarding imaging of an area corresponding to the imaging area.

[0119] Here, among the costs registered in the image individual map, the cost set using the imaged image imaged by the drone 11 is maintained until the area appearing in the imaged image is imaged again by the drone 11 and is not updated (however, except for the case where the area appearing in the imaged image is imaged by the surveillance camera 13, etc.).

[0120] Therefore, for example, if the drone 11 images the predetermined area while an event in which people gather (such as a fireworks display, festival, or sporting event) is occurring in the predetermined area, thereby setting the cost to high and registering the predetermined area in the image individual map, then unless the drone 11 flies near the predetermined area and images the predetermined area again after the event is over and no crowd is present in the predetermined area, the cost is still registered as high in the image individual map.

[0121] Therefore, regarding the cost set using an image of an image of a mobile object such as the drone 11 in the individual image map, it is possible to reset the cost to an initial value when a certain period of time has passed since the last cost was registered. As the initial value of the cost, for example, a value indicating the lowest risk or a value indicating that the cost is undetermined may be adopted.

[0122] As described above, by performing the reset process, it is possible to prevent the cost from remaining registered as high in the image individual map even when the event ends and a crowd no longer exists.

[0123] By the way, in the image individual map, in the case where the cost of the image set using the imaging of the drone 11 is reset to the initial value at a timing after a certain time has passed from the registration of the latest cost, the cost of the image set using the imaging of the drone 11 is reset even if the event continues and the crowd still exists after a certain time has passed from the latest cost registration.

[0124] However, the cost of resetting the area where the crowd exists while the event continues and the crowd exists is not expected.

[0125] Therefore, the individual map generation unit 32 can limit the resetting of the cost of the event area during the event registered in the image individual map by using the event information.

[0126] Figure 8 is a diagram for explaining the limitation of resetting the cost of the event area during an event.

[0127] For example, the individual map generation unit 32 uses event information to specify the event area where the event is occurring. Furthermore, in the image individual map generated immediately before the event, the individual map generation unit 32 resets the costs for only non-event areas other than the event area during the event, and for areas where a specific period of time has passed since the most recent cost registration.

[0128] Therefore, it is possible to prevent the cost of the event area during an event from being reset after a certain period of time has passed since registration into the image individual map, that is, to restrict the resetting of the cost of the event area during an event.

[0129] Note that the individual map generation unit 32 can use event information to reset the cost for the event area. For example, using the event information, the individual map generation unit 32 can specify the end date and time of the event that occurred in the event area. The individual map generation unit 32 can then reset the cost for the event area in the image individual map, where the event has ended, when the end date and time have passed.

[0130] The process of generating or updating the image individual map in the individual map generating unit 32 is as follows, for example.

[0131] The individual map generation unit 32 detects a crowd using the latest imaging image, and generates crowd information indicating a crowd area using the detection result of the crowd.

[0132] Using the crowd information, the individual map generating unit 32 sets a cost for each divided area with respect to the imaged area appearing in the most recently imaged image.

[0133] Using the event information, in the image individual map generated immediately before, the individual map generation unit 32 performs a reset process of resetting the costs of the area where a certain period has passed since the latest cost was registered to the initial value only for the costs registered in the non-event area.

[0134] By performing the reset process only for the costs registered in the non-event area, it is possible to prevent the costs registered for the area where a crowd is likely to be present during the event from being reset.

[0135] The individual map generation unit 32 generates a latest image individual map (updated image individual map) by updating the cost of an area for which the latest cost is set using the latest image among the costs registered in the image after the reset process to the latest cost set using the latest image.

[0136] That is, the individual map generation unit 32 overwrites the latest cost set using the latest image imaged with the latest cost set using the latest image imaged registered in the image individual map after the reset process.

[0137] Note that using event information, the individual map generation unit 32 can generate an individual event information map, which is an individual cost map for the event information. For example, using event information, the individual map generation unit 32 can designate the event area as a crowd area and set a cost for the event area based on the area of ​​the event area. The individual map generation unit 32 can then register the cost of the event area with the cost map and generate an individual event information map.

[0138] Event information individual map can be combined with image individual map, location information individual map and weather information individual map. Figure 1 It is used to generate the integrated map in the integration unit 35.

[0139] <Generation of Location Information Individual Map>

[0140] Figure 9 3 is a diagram illustrating a generation example of a position information individual map in the individual map generation unit 33 .

[0141] Using the position information of the smartphone 51 , the individual map generation unit 33 detects a crowd and generates crowd information.

[0142] For example, taking the position indicated by the position information of the smartphone 51 as the position where a person exists, the individual map generation unit 33 detects a gathering of people having a density equal to or greater than a predetermined density as a crowd.

[0143] In the case where a crowd is detected based on the position information from the smartphone 51 , the individual map generation unit 33 detects a crowd area where the crowd exists, and generates crowd information indicating the crowd area.

[0144] Furthermore, using the crowd information, the individual map generation unit 33 sets a cost indicating the risk of the drone 11 passing through the crowd area indicated by the crowd information. The individual map generation unit 33 can set the cost similarly to the individual map generation unit 32.

[0145] The individual map generating unit 33 sets the cost of the crowd area and then generates a position information individual map by registering the cost into a cost map.

[0146] The individual map generation unit 33 can detect people in an area that is not imaged (cannot be imaged) by the drone 11 or the surveillance camera 13, for example, set a cost indicating the risk of passing through the area, and generate a location information individual map as a cost map in which the cost is registered.

[0147] The process of generating or updating the position information individual map in the individual map generating unit 33 is as follows, for example.

[0148] The individual map generating unit 33 detects a crowd using the latest position information of the smartphone 51 , and generates crowd information using the detection result of the crowd.

[0149] Using the crowd information generated using the latest position information, the individual map generating unit 33 sets a cost for each divided area with respect to the registered area of ​​the position information individual map.

[0150] The individual map generation unit 33 resets the cost of the position information individual map generated immediately before, and overwrites the cost set using the latest position information and registers it in the position information individual map, thereby generating the latest position information individual map (updated position information individual map).

[0151] <Generation of individual weather information maps>

[0152] Figure 10 is a diagram illustrating an example of generating a weather information individual map in the individual map generating unit 34 .

[0153] Using weather information measured by the drone 11 , the individual map generation unit 34 sets a cost indicating the risk of the drone 11 passing through an area including the position of the drone 11 measuring the weather information (e.g., an imaging area appearing in an image imaged by the drone 11 ).

[0154] For example, the individual map generating unit 34 sets the cost for the imaging area according to the wind speed indicated by the weather information through threshold processing of the wind speed.

[0155] For example, three levels of risk, high, medium, and low, from high to low, are used as costs, and a first wind speed threshold and a second wind speed threshold greater than the first wind speed threshold are used as the two threshold wind speeds.

[0156] If the wind speed indicated by the weather information is equal to or less than a first wind speed threshold, the cost is set to low. If the wind speed indicated by the weather information is greater than the first wind speed threshold and equal to or less than a second wind speed threshold, the cost is set to medium. If the wind speed indicated by the weather information is greater than the second wind speed threshold, the cost is set to high.

[0157] Note that when setting the cost using weather information, in addition to wind speed, you can also use rainfall, weather (sunny, rainy, snowy, etc.) to set the cost.

[0158] The individual map generating unit 34 sets a cost to the imaging area and then generates a weather information individual map by registering the cost into a cost map.

[0159] For the weather information individual map, it is possible to perform a reset process similar to that performed for the image individual map.

[0160] The process of generating or updating the weather information individual map in the individual map generating unit 34 is as follows, for example.

[0161] Using the latest weather information, the individual map generating unit 34 sets a cost (for each divided area) with respect to the registered area of ​​the weather information individual map.

[0162] The individual map generating unit 34 resets the cost of the weather information individual map generated immediately before and overwrites and registers the cost set using the latest weather information into the weather information individual map, thereby generating the latest weather information individual map (updated weather information individual map).

[0163] Alternatively, the individual map generating unit 34 overwrites the latest cost set using the latest weather information with the cost of the divided area for which the latest cost is set using the latest weather information in the weather information individual map generated immediately before and registers it therein, thereby generating the latest weather information individual map.

[0164] As described above, by generating a weather information individual map using weather information, it is possible to generate a route plan that takes the influence of weather into consideration.

[0165] <Final Map Generation>

[0166] Figure 11 is a diagram illustrating an example of generating a final map in the integration unit 35 .

[0167] The integration unit 35 integrates the individual image map, the individual location information map, and the individual weather information map to generate an integrated map, which is the final cost map.

[0168] For each divided area that divides the registration area of the integrated map, the integration unit 35 integrates the individual image map (the cost registered therein), the individual location information map, and the individual weather information map by selecting the highest cost (the cost indicating a higher risk) from the costs registered in the individual image map, the individual location information map, and the individual weather information map.

[0169] That is, for each divided area that divides the registration area of the integrated map, the integration unit 35 selects the highest cost from the costs registered in the individual image map, the individual location information map, and the individual weather information map and registers it in the integrated map.

[0170] As described above, by integrating the individual image map of an image used for imaging, for example, the individual location information map, the individual weather information map, the location information, and the weather information of the smart phone 51 into multiple pieces of pattern information respectively, it is possible to obtain an integrated map in which, for a wide area that not only includes the imaging area that appears in the imaged image captured by the drone 11 and the surveillance camera 13, but also includes areas other than the imaging area, the integrated map registering significant costs with high accuracy is obtained. Therefore, it is possible to generate a path plan that makes flight safer.

[0171] <Another example of the processing of UTM 10>

[0172] Figures 12 to 14 is a diagram Figure 3 of a flowchart of another example of the processing of UTM 10.

[0173] Here, the imaging area that appears in the imaged image transmitted from the drone 11 and the surveillance camera 13 is an area near the positions of the drone 11 and the surveillance camera 13 when imaging the imaged image, and is a part of the registration area of the cost map of the individual image map. Therefore, if the overall individual image map is generated again every time the imaged image is transmitted from the drone 11 and the surveillance camera 13, it takes time and effort.

[0174] Therefore, regarding the individual image map, in the registration area of the individual image map updated or generated immediately before, it is possible to update only the cost of the area where the value has changed from the cost obtained using the existing imaged image among the areas that appear in the latest imaged image transmitted from the drone 11 and the surveillance camera 13 (for example, the area where the cost obtained using the latest imaged image has changed).

[0175] The updating of the image individual map (cost) may be performed each time an imaged image is transmitted from the drone 11 or the surveillance camera 13, or may be performed by storing the imaged images transmitted from the drone 11 and the surveillance camera 13 and using the imaged images stored at a fixed period (periodically) (imaged images acquired within a specific period). Alternatively, the updating of the image individual map may be performed by storing the imaged images transmitted from the drone 11 and the surveillance camera 13 and irregularly using the imaged images stored since the image individual map was updated immediately before. Figure 1 It is performed based on the imaged images stored up to now (imaged images acquired within a specific period of time).

[0176] The weather information individual map generated or updated using the weather information transmitted from the drone 11 can also be generated or updated similarly to the image individual map.

[0177] With the position information individual map generated or updated using the position information transmitted from the smartphones 51 , position information can be simultaneously acquired from the smartphones 51 existing in the registration area of ​​the position information individual map.

[0178] Therefore, by updating not only the cost of the area in which the cost has changed but also the cost of the entire registration area of ​​the position information individual map (such as the image individual map and the weather information individual map), the updating of the position information individual map can be efficiently performed.

[0179] Figure 12 is a flowchart presenting an example of processing performed by the UTM 10 on an image individual map.

[0180] In step S31, in the UTM 10, the information receiving unit 31 of the cost map generation unit 21 waits for the imaged images transmitted from the drone 11 and the surveillance camera 13, asynchronously receives the imaged images, and supplies the imaged images to the individual map generation unit 32, and the processing proceeds to step S32.

[0181] In step S32 , the individual map generating unit 32 stores the imaged image from the information receiving unit 31 into a built-in memory not shown, and the process proceeds to step S33 .

[0182] In step S33 , the individual map generation unit 32 determines whether the current time is the update timing for updating the image individual map.

[0183] In the case where it is determined in step S33 that the current time is not the update timing, for example, in the case where a predetermined period has not elapsed since the last update of the image individual map, the process returns to step S31 and similar processing is repeated thereafter.

[0184] Furthermore, in the case where it is determined in step S33 that the current time is the update timing, for example, in the case where a predetermined period of time has elapsed since the last update of the image individual map, the process proceeds to step S34 .

[0185] In step S34 , using the imaged image stored in the memory until now after the previous update of the image individual map, the individual map generation unit 32 updates the image individual map and supplies it to the integration unit 35 , and the process proceeds to step S35 .

[0186] In step S35, the integration unit 35 integrates the latest image individual map supplied from the individual map generation unit 32, the latest position information individual map supplied from the individual map generation unit 33, and the latest weather information supplied from the individual map generation unit 34, and generates an integrated map as the final cost map. The integration unit 35 then supplies the integrated map to the path planning unit 22, and the process proceeds from step S35 to step S36.

[0187] In step S36 , the path planning unit 22 determines whether it is necessary to generate or regenerate a path plan.

[0188] In the case where it is determined in step S36 that the path plan does not need to be generated and regenerated, the process returns to step S31 and similar processes are repeated thereafter.

[0189] Furthermore, in the event that it is determined in step S33 that it is necessary to generate or regenerate the path plan, the process proceeds to step S37 .

[0190] In step S37 , the path planning unit 22 generates a path plan using the cost map as the latest final map from the integration unit 35 , and the process proceeds to step S38 .

[0191] In step S38, the path planning unit 22 transmits the path plan generated in the immediately preceding step S37 to the drone 11. Then, the process returns from step S38 to step S31, and similar processes are repeated thereafter.

[0192] Figure 13 is a flowchart presenting an example of processing performed by the UTM 10 on the weather information individual map.

[0193] In step S41, in the UTM 10, the information receiving unit 31 of the cost map generating unit 21 waits for weather information to be transmitted from the drone 11, asynchronously receives the weather information, and supplies the weather information to the individual map generating unit 34, and the process proceeds to step S42.

[0194] In step S42 , the individual map generating unit 34 stores the weather information from the information receiving unit 31 into a built-in memory not shown, and the process proceeds to step S43 .

[0195] In step S43 , the individual map generation unit 34 determines whether the current time is the update timing for updating the weather information individual map.

[0196] In the case where it is determined in step S43 that the current time is not the update timing, for example, in the case where a predetermined period has not elapsed since the weather information individual map was last updated, the process returns to step S41 and similar processing is repeated thereafter.

[0197] Furthermore, in the case where it is determined in step S43 that the current time is the update timing, for example, in the case where a predetermined period has elapsed since the weather information individual map was last updated, the process proceeds to step S44 .

[0198] In step S44 , using the weather information stored in the memory after the weather information individual map was last updated, the individual map generation unit 34 updates the weather information individual map and supplies it to the integration unit 35 , and the process proceeds to step S45 .

[0199] In steps S45 to S48, respectively, Figure 12 The processing is similar to the processing in steps S35 to S38 in .

[0200] Figure 14 is a flowchart presenting an example of processing performed by the UTM 10 on the position information individual map.

[0201] In step S51, in the UTM 10, the information receiving unit 31 of the cost map generating unit 21 periodically or irregularly requests location information from each smartphone 51 existing in the registration area of ​​the location information individual map. By receiving the location information transmitted from the smartphone 51 in response to the request for location information, the information receiving unit 31 acquires the location information of each smartphone 51 existing in the registration area of ​​the location information individual map at a time.

[0202] For example, acquisition of the position information of each smartphone 51 existing in the registration area of ​​the position information individual map as described above can be performed using an application programming interface (API) that acquires the position information of the smartphone 51 at a time, such as Google Maps of Google Inc.

[0203] The information receiving unit 31 supplies the individual map generating unit 33 with the position information of each smartphone 51 existing in the registration area of ​​the position information individual map, and the process proceeds from step S51 to step S52 .

[0204] In step S52 , the individual map generation unit 33 updates the position information individual map using the position information acquired in the immediately preceding step S51 and supplies it to the integration unit 35 , and the process proceeds to step S53 .

[0205] In steps S53 to S56, respectively, Figure 12 The processing is similar to the processing in steps S35 to S38 in .

[0206] The UTM 10 to which the present technology is applied has been described above, and the functions of the UTM 10 may be equipped on the drone 11 , for example.

[0207] <Description of Computer to Which the Present Technology is Applied>

[0208] Next, the above-described series of processing can be executed by hardware, or can be executed by software. In the case of executing the series of processing by software, a program constituting the software is installed in a general-purpose computer or the like.

[0209] Figure 15 1 is a block diagram showing a configuration example of an embodiment of a computer in which a program for executing the above-described series of processes is installed. Figure 15 The computer is used as the hardware configuration of UTM 10.

[0210] The program can be recorded in advance in the hard disk 905 or the ROM 903 which is a recording medium built into the computer.

[0211] Alternatively, the program may be stored (recorded) in a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 may be provided as so-called packaged software. Here, examples of the removable recording medium 911 include, for example, a flexible disk, a compact disk read-only memory (CD-ROM), a magneto-optical disk (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like.

[0212] Note that the program can be installed in the computer from the removable recording medium 911 as described above, or can be downloaded to the computer via a communication network or a broadcast network and installed in the built-in disk 905. That is, the program can be wirelessly transmitted to the computer from, for example, a download site via an artificial satellite used for digital satellite broadcasting, or can be transmitted to the computer by wire via a network such as a local area network (LAN) or the Internet.

[0213] The computer incorporates a central processing unit (CPU) 902 , and an input / output interface 910 is connected to the CPU 902 via a bus 901 .

[0214] When a command is input by a user operating the input unit 907 or the like via the input / output interface 910, the CPU 902 executes a program stored in a read-only memory (ROM) 903 according to the command. Alternatively, the CPU 902 loads a program stored in the hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0215] Thus, the CPU 902 executes the processing according to the above flowchart or the processing executed by the configuration of the above block diagram. Then, the CPU 902 outputs the processing result from, for example, the output unit 906, transmits the processing result from the communication unit 908, or records the processing result in the hard disk 905 via the input / output interface 910 as needed.

[0216] Note that the input unit 907 includes a keyboard, a mouse, a microphone, etc. Furthermore, the output unit 906 includes a liquid crystal display (LCD), a speaker, and the like.

[0217] Here, in this description, the processing executed by the computer according to the program is not necessarily executed in time series in the order described in the flowchart. That is, the processing executed by the computer according to the program includes processing executed in parallel or individually (for example, parallel processing or object processing).

[0218] In addition, the program may be processed by one computer (processor) or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

[0219] In this description, a system refers to a collection of multiple configuration elements (devices, modules (components), etc.), and it does not matter whether all the configuration elements are located in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0220] Note that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made within the scope not departing from the spirit of the present technology.

[0221] For example, the present technology may have a configuration of cloud computing in which one function is shared and collaboratively processed by a plurality of devices via a network.

[0222] Furthermore, each step explained in the above flowchart may be performed by one device or performed by a plurality of devices in a shared manner.

[0223] Furthermore, in the case where one step includes a plurality of processes, the plurality of processes included in one step may be executed by one device or may be executed by a plurality of devices in a shared manner.

[0224] Furthermore, the effects described in this description are merely examples and are not limited thereto, and other effects may exist.

[0225] Note that the present technology can have the following configurations.

[0226] <1>

[0227] An information processing device, comprising:

[0228] A cost map generating unit generates a cost map indicating a risk of passing through an area by using the crowd information.

[0229] <2>

[0230] according to <1> The information processing device, wherein

[0231] The cost map generation unit uses images to detect crowds and generates crowd information.

[0232] <3>

[0233] according to <2> The information processing device, wherein

[0234] The cost map generation unit uses images to detect people and uses the person detection results to detect crowds.

[0235] <4>

[0236] according to <2> or <3> The information processing device, wherein

[0237] The image is an image captured by a camera equipped on a mobile object or an image captured by a surveillance camera.

[0238] <5>

[0239] according to <4> The information processing device, wherein

[0240] The mobile object is an unmanned aerial vehicle (UAV).

[0241] <6>

[0242] according to <2> to <5> The information processing device according to any one of

[0243] The cost map generation unit generates a cost map using images acquired in a specific period of time.

[0244] <7>

[0245] according to <2> to <6> The information processing device according to any one of

[0246] The cost map generation unit updates, in the cost map, a cost for a region in which the cost obtained using the existing image is changed.

[0247] <8>

[0248] according to <1> to <7> The information processing device according to any one of

[0249] The cost map generation unit sets a cost for a crowd area according to the area of ​​the crowd area in the real world or on an image.

[0250] <9>

[0251] according to <1> to <8> The information processing device according to any one of

[0252] The cost map generation unit detects a crowd using the location information of the mobile terminal and generates crowd information.

[0253] <10>

[0254] according to <1> to <9> The information processing device according to any one of

[0255] The cost map generation unit generates a cost map periodically or irregularly.

[0256] <11>

[0257] according to <10> The information processing device, wherein

[0258] The cost map generation unit resets the cost for an area for which a certain period of time has passed since registration of the cost.

[0259] <12>

[0260] according to <11> The information processing device, wherein

[0261] The cost map generation unit uses event information about an event where people gather, and resets the cost for an area where a specific period of time has passed since the cost was registered, targeting only the cost for non-event areas other than the event area where the event is occurring.

[0262] <13>

[0263] according to <1> to <10> The information processing device according to any one of

[0264] The cost map generation unit generates a cost map by further using weather information.

[0265] <14>

[0266] according to <1> to <13> The information processing device according to any one of

[0267] The cost map generation unit generates a cost map by further using event information about events where people gather.

[0268] <15>

[0269] according to <1> The information processing device, wherein

[0270] Costmap Generation Unit

[0271] generating an individual map as an individual cost map for each piece of information of a plurality of types of patterns including at least information from which crowd information is obtained, and

[0272] A final cost map is generated by integrating the individual maps for each of the information for the multiple types of modes.

[0273] <16>

[0274] according to <15> The information processing device, wherein

[0275] Various types of pattern information include images or location information of mobile terminals, and

[0276] The cost map generating unit detects a crowd using an image or position information of the mobile terminal and generates crowd information.

[0277] <17>

[0278] according to <16> The information processing device, wherein

[0279] The cost map generation unit integrates the individual maps by selecting a cost indicating the highest risk from the costs registered in the individual maps for each piece of information in the plurality of types of patterns.

[0280] <18>

[0281] according to <1> to <17> The information processing device according to any one of claims , further comprising:

[0282] A path planning unit that uses the cost map to generate a path plan for the mobile body.

[0283] <19>

[0284] An information processing method, comprising:

[0285] By using crowd information, a cost map is generated that indicates the risk of traveling through an area.

[0286] <20>

[0287] A program that causes a computer to be used as

[0288] A cost map generating unit generates a cost map indicating a risk of passing through an area by using the crowd information.

[0289] Reference Signs List

[0290] 10 UTM

[0291] 11. Drones

[0292] 13 surveillance cameras

[0293] 21 Cost Map Generation Unit

[0294] 22 Path Planning Unit

[0295] 31 Information receiving unit

[0296] 32 to 34 individual map generation units

[0297] 35 integrated units

[0298] 51 Smartphone

[0299] 52 web servers

[0300] 901 Bus

[0301] 902 CPU

[0302] 903 ROM

[0303] 904 RAM

[0304] 905 Hard Drive

[0305] 906 Output Unit

[0306] 907 Input Unit

[0307] 908 Communication Unit

[0308] 909 Driver

[0309] 910 Input / Output Interface

[0310] 911 Removable Recording Media

Claims

1. An information processing device, comprising: a cost map generating unit that generates a cost map indicating a risk of passing through an area by using crowd information, The cost map generation unit uses event information about events where people gather, and in the cost map, only the costs for non-event areas other than the event area where the event is occurring are used as the object to reset the costs for areas that have passed a specific period of time since the cost was registered.

2. The information processing apparatus according to claim 1, wherein The cost map generation unit detects a crowd by using an image and generates the crowd information.

3. The information processing apparatus according to claim 2, wherein The cost map generation unit detects a person using the image, and detects the crowd using the detection result of the person.

4. The information processing apparatus according to claim 2, wherein The image is an image captured by a camera equipped on a mobile object or an image captured by a monitoring camera. The information processing apparatus according to claim 4 , wherein The mobile object is an unmanned aerial vehicle (UAV). The information processing apparatus according to claim 2 , wherein The cost map generation unit generates the cost map using images acquired in a specific period.

7. The information processing apparatus according to claim 2, wherein The cost map generation unit updates, in the cost map, a cost for a region in which a cost obtained using an existing image is changed. The information processing apparatus according to claim 1 , wherein The cost map generating unit sets a cost for a crowd area according to an area of ​​the crowd area in the real world or on an image.

9. The information processing apparatus according to claim 1, wherein The cost map generating unit detects a crowd using location information of a mobile terminal and generates the crowd information.

10. The information processing apparatus according to claim 1, wherein The cost map generating unit generates the cost map periodically or irregularly. The information processing apparatus according to claim 1 , wherein The cost map generating unit generates the cost map by further using weather information.

12. The information processing apparatus according to claim 1, wherein The cost map generation unit generates the cost map by further using event information about an event where people gather.

13. The information processing apparatus according to claim 1, wherein The cost map generation unit generating an individual map as an individual cost map for each piece of information of a plurality of types of patterns including at least information from which crowd information is obtained, and A final cost map is generated by integrating the individual maps for each of the plurality of types of modes.

14. The information processing apparatus according to claim 13, wherein The information of the plurality of types of modes includes image or location information of the mobile terminal, and The cost map generation unit detects a crowd by using the image or the position information of the mobile terminal, and generates the crowd information.

15. The information processing apparatus according to claim 14, wherein The cost map generation unit integrates the individual maps by selecting a cost indicating the highest risk from the costs registered in the individual maps for each piece of information among the plurality of types of patterns.

16. The information processing apparatus according to claim 1, further comprising: A path planning unit generates a path plan for the mobile object using the cost map.

17. An information processing method, comprising: By using crowd information to generate cost maps that indicate the risk of traveling through an area, Here, using event information about an event where people gather, the cost map resets the cost for an area where a specific period of time has passed since the cost was registered, targeting only the cost for non-event areas other than the event area where the event is occurring.

18. A computer program product for causing a computer to: a cost map generating unit that generates a cost map indicating a risk of passing through an area by using crowd information, in, The cost map generation unit uses event information about an event where people gather, and resets costs for areas where a specific period of time has passed since cost registration, targeting only costs for non-event areas other than the event area where the event is occurring.

Images