Information processing method, information processing device, and information processing program

Abstract

[Problem] To output information relating to the number of moving bodies. [Solution] This information processing method includes: acquiring a plurality of items of image data obtained by imaging a target range; extracting a plurality of items of point cloud data including position information relating to objects existing within the target range from the plurality of items of image data; superimposing the plurality of items of point cloud data on the basis of the position information relating to the objects; extracting point cloud data relating to an outlier; and outputting an existence probability of a moving body among the objects existing within the target range on the basis of the outlier point cloud data.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to an information processing method, an information processing device, and an information processing program.

[0002] Structure From Motion (SFM) / Multi-View Stereo (MVS) technology, which generates three-dimensional shape data of a target object from two-dimensional image data, is widely used to create point clouds for digital twins. However, SFM / MVS technology has the drawback of making it difficult to detect moving objects because it extracts objects by matching with other images. To address this issue, an image processing device has been proposed that distinguishes between stationary and moving object regions based on the correlation between multiple stereo images captured at different times (see, for example, Patent Document 1).

[0003] Furthermore, a measurement device has been proposed that compares a road surface shape model obtained by laser measurement with an image of the road surroundings to determine a moving object area (see Patent Document 2).

[0004] JP 2020-42727 A JP 2009-204615 A

[0005] When planning a robot's path, etc., information on the number of moving objects within a certain range, such as people flow, may be required. However, the method of Patent Document 1 merely estimates the moving object area and is unable to output information on the number of moving objects within the certain range.

[0006] Furthermore, the method of Patent Document 2 requires that a three-dimensional model of the road surface shape and the like be prepared in advance, which makes the preparations complicated.

[0007] Therefore, the present disclosure provides an information processing method, an information processing device, and an information processing program that can output information on the number of moving objects.

[0008] In order to solve the above problem, according to the present disclosure, an information processing method is provided, which includes acquiring a plurality of image data images of a target range, extracting a plurality of point cloud data having position information of objects present within the target range from the plurality of image data, overlaying the plurality of point cloud data based on the position information of the objects, extracting outlier point cloud data, and outputting the probability of the existence of a moving object among the objects present within the target range based on the outlier point cloud data.

[0009] The probability of the existence of the moving object may be displayed based on the point cloud data of the outliers.

[0010] The plurality of point cloud data may include a plurality of point data each having position information of the object, and output point cloud data having position information of stationary objects among the objects may be extracted from the plurality of point data, and the outlier point cloud data may be extracted from point data among the plurality of point data that was not extracted as the output point cloud data.

[0011] The plurality of image data may include first image data and second image data having an imaging position different from that of the first image data, and the plurality of point cloud data may include first point cloud data having position information of the object based on the imaging position of the first image data and second point cloud data having position information of the object based on the imaging position of the second image data.

[0012] The outlier point cloud data may include one or more point data, and the probability of the existence of the moving object may be displayed based on at least one of the density of the one or more point data or the number of the one or more point data.

[0013] From the plurality of image data, a plurality of image data captured in a first period and a plurality of image data captured in a second period after the first period are obtained, and the probability of the presence of a moving object in the first period is displayed based on the plurality of image data captured in the first period, and the probability of the presence of a moving object in the second period is displayed based on the plurality of image data captured in the second period.

[0014] A mapping model showing the positional relationship of one or more objects present in the target range may be formed based on the plurality of point cloud data, and a range in which a moving object may exist may be displayed on the mapping model based on the outlier point cloud data.

[0015] The probability of the existence of the moving object within the range in which the moving object may exist may be displayed by the number of the one or more point data, the density of the one or more point data, or a pixel value corresponding to the number or density of the point data.

[0016] The range in which the moving object may exist may include a first range and a second range in which the probability of the moving object existing is lower than that of the first range, and in the early mapping model, a movement path of the moving object may be acquired that passes through the early second range in preference to the early first range.

[0017] Furthermore, according to the present disclosure, there is provided an information processing device comprising: an acquisition unit that acquires a plurality of image data images of a target range; a first extraction unit that extracts a plurality of point cloud data having position information of objects present within the target range from the plurality of image data; a second extraction unit that overlays the plurality of point cloud data based on the position information of the objects and extracts outlier point cloud data; and an output unit that outputs the probability of the existence of a moving object among the objects present within the target range based on the outlier point cloud data.

[0018] Furthermore, according to the present disclosure, there is provided an information processing program comprising the steps of: acquiring a plurality of image data images of a target range; extracting a plurality of point cloud data having position information of objects present within the target range from the plurality of image data; superimposing the plurality of point cloud data based on the position information of the objects and extracting outlier point cloud data; and outputting the probability of the existence of a moving object among the objects present within the target range based on the outlier point cloud data.

[0019] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment of the present disclosure. 2 is a block diagram showing a configuration of a management unit according to a first embodiment of the present disclosure. 3 is a flowchart explaining the operation of the information processing device of FIG. 1. 4 is a flowchart explaining the operation of the management unit of FIG. 2. 5 is a flowchart showing an example of a matching process. 6 is a diagram showing an example of output data of an outlier point cloud. 7 is a diagram showing an example of a UI for setting a path plan for a robot. 8 is a flowchart showing the operation of setting a path plan for a robot.

[0020] Hereinafter, embodiments of an information processing method, an information processing device, and an information processing program will be described with reference to the drawings. The following description will focus on the main components of the information processing method, the information processing device, and the information processing program, but the information processing method, the information processing device, and the information processing program may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0021] 1 is a block diagram showing the configuration of an information processing device 1 according to a first embodiment of the present disclosure. The information processing device 1 is, for example, a device that creates a point cloud for a three-dimensional model (e.g., a digital twin) using SFM / MVS technology, and is configured, for example, by a computer. The information processing device 1 can also display the number of moving objects within a certain range on the three-dimensional model.

[0022] An information processing device 1 according to the first embodiment of the present disclosure includes a management unit 2 and an image acquisition unit (acquisition unit) 3. The management unit 2 performs predetermined image processing on images acquired from the image acquisition unit 3 to extract point cloud data. The image acquisition unit 3 acquires multiple image data from an external imaging device (e.g., a camera) or the like. Note that the imaging device may be built into the information processing device 1.

[0023] The imaging device is preferably configured to capture images of a predetermined range (target range) from two or more imaging positions in order to generate point cloud data that can be projected onto a three-dimensional model. Specifically, the imaging device may be a mobile camera mounted on a robot or drone, or may be two or more fixed cameras.

[0024] The information processing device 1 may be configured to include a time acquisition unit 4. The time acquisition unit 4 acquires time information from a clock built into the information processing device 1 or from an external device (for example, an NTP (Network Time Protocol) server).

[0025] Alternatively, time information may be added to image data by an external imaging device or an imaging device built into the information processing device 1. The image acquisition unit 3 may acquire image data to which time information has been added.

[0026] 2 is a block diagram showing the configuration of the management unit 2 according to the first embodiment of the present disclosure. The management unit 2 includes a point cloud extraction unit (first extraction unit) 11, a feature point extraction unit 12, an imaging position acquisition unit 13, a three-dimensional projection unit 14, a matching unit (second extraction unit) 15, and an outlier point cloud output unit (output unit, or outlier point cloud acquisition unit) 16. The management unit 2 may also include a time period group generation unit 17.

[0027] Fig. 3 is a flowchart illustrating the operation of the information processing device 1. Fig. 4 is a flowchart illustrating the operation of the management unit 2. Below, the functions of each component of the information processing device 1 and the management unit 2 will be explained using Fig. 3 and Fig. 4. Note that below, an example will be explained in which the information processing device 1 has a time acquisition unit 4 and the management unit 2 has a time period group generation unit 17.

[0028] First, the image acquisition unit 3 acquires image data (step S1 in FIG. 3). Then, the time acquisition unit 4 acquires time data at a timing corresponding to step S1 (step S2). For example, if the time acquisition unit 4 acquires time data at approximately the same timing as the image capture timing of the image capture device, the image capture time can be acquired. The image acquisition unit 3 may acquire image data from the image capture device in real time.

[0029] The management unit 2 associates the image data acquired in step S1 with the time information acquired in step S2 (step S3), thereby associating the image data with the image capture time.

[0030] The image acquisition unit 3 may acquire data in advance, in which the image capture time and image data are linked together, using an imaging device or the like.

[0031] The time period group generation unit 17 in the management unit 2 groups the data acquired by the image acquisition unit 3 by any time period and generates an image group (step S4). This makes it possible to analyze the number of moving objects for each time period by the process described below.

[0032] One image group includes two or more image data (for example, first image data and second image data) captured at different positions.

[0033] The management unit 2 performs the process shown in Fig. 4 for each image group generated in step S4. First, the point cloud extraction unit 11 selects one image data item from the image group (step S11).

[0034] The point cloud extraction unit 11 extracts point cloud data from the selected image data (step S12). The point cloud data is, for example, data obtained by extracting a portion of the image data where an object is located using a plurality of point data.

[0035] The feature point extraction unit 12 extracts a plurality of feature points from the selected image data (step S13). The feature point extraction unit 12 may extract a plurality of feature points from the point cloud data extracted by the point cloud extraction unit 11.

[0036] The imaging position acquisition unit 13 acquires information about the imaging position of the selected image data (step S14). Specifically, the imaging position acquisition unit 13 estimates the information about the imaging position by matching feature points of the selected image data with feature points of other image data in the image group. The information about the imaging position includes, for example, the three-dimensional position coordinates of the imaging device, the attitude (angle) of the imaging device, etc.

[0037] Note that image data serving as a reference for estimating the information on the imaging position may be prepared in advance. The imaging position acquisition unit 13 may estimate the information on the imaging position by matching feature points of the reference image data with feature points of the selected image data. In addition, the imaging position acquisition unit 13 may acquire information on the imaging position of the image data from an imaging device or the like.

[0038] The three-dimensional projection unit 14 projects the point cloud data onto the three-dimensional model based on the information on the image capture positions acquired by the image capture position acquisition unit 13 (step S15). In this specification, each point data in the point cloud data projected onto the three-dimensional model is also referred to as a projection point. Each projection point has position information of an object in the image data.

[0039] Projection points acquired from a plurality of image data belonging to the same image group are superimposed on the three-dimensional model based on the position information of the object that each of them has.

[0040] The matching unit 15 performs a matching process (step S16). Fig. 5 is a flowchart showing an example of the matching process. The matching unit 15 performs a matching determination for each projection point (step S21).

[0041] The matching unit 15 determines whether the matching of the projection points of the selected image data has been successful (step S22). For example, if there is one or more projection points whose object position information substantially matches that of the selected image data among the projection points acquired from other image data belonging to the same image group, the matching is determined to be successful. Alternatively, the matching unit 15 may determine that the matching is successful if there are a predetermined threshold or more projection points whose object position information substantially matches.

[0042] The matching unit 15 extracts the successfully matched projection points as an output point cloud (step S23). The output point cloud is a point cloud that indicates stationary objects among the objects in the image data. For example, a stationary object may be a facility whose location is fixed. Note that the stationary object may also include an object whose location is not fixed but whose movement amount is small within a predetermined time period.

[0043] The matching unit 15 extracts the projection points that have failed to be matched as an outlier point cloud (step S24). It is assumed that a portion of the image data where a moving object exists will fail to be matched with the projection points and will be extracted as an outlier point cloud.

[0044] That is, the matching unit 15 overlays at least two pieces of point cloud data on the three-dimensional model. Specifically, the matching unit 15 overlays first point cloud data (first projection point cloud data) having object position information based on the imaging position of the first image data with second point cloud data (second projection point cloud data) having object position information based on the imaging position of the second image data. The matching unit 15 may output, as an output point cloud, projection points among the multiple projection points in the first point cloud data that include one or more projection points whose object position information approximately matches the second point cloud data. Furthermore, the matching unit 15 may output, as an outlier point cloud, projection points among the multiple projection points in the first point cloud data that do not include one or more projection points whose object position information approximately matches the second point cloud data.

[0045] The matching process in step S16 in Fig. 4 may be performed using the PatchMatch algorithm, in addition to the method shown in Fig. 5. Alternatively, isolated projection points may be extracted as outlier point clouds based on their spatial positional relationships with other projection points. Alternatively, a neural network may be used to learn in advance the characteristics of projection points that will become output point clouds or the characteristics of projection points that will become outlier point clouds, and the projection points may be classified as output point clouds or outlier point clouds based on the learning results.

[0046] If the time acquisition unit 4 in Figure 1 and the time zone group generation unit 17 in Figure 2 are omitted, the management unit 2 may treat the multiple image data acquired by the image acquisition unit 3 as an image group and perform the processing of Figure 4.

[0047] The outlier point cloud output unit 16 outputs the outlier point cloud extracted by the matching unit 15. FIG. 6 is a diagram showing an example of the output data 21 of the outlier point cloud. The output data 21 in FIG. 6 shows an example of outputting outlier point clouds 23a, 23b, and 23c onto a mapping model 22. The outlier point cloud output unit 16 may display the outlier point cloud in a format that is visible to the user or recognizable by a downstream image processing device or the like, as shown in FIG. 6. Alternatively, the outlier point cloud output unit 16 may output the outlier point cloud to a downstream neural network or the like as data that is not visible to the user.

[0048] The mapping model 22 can be generated, for example, based on the output point cloud extracted by the matching unit 15. The mapping model 22 is three-dimensional model data that indicates the positional relationship of one or more objects that exist in the target area. The mapping model 22 can be used, for example, as digital twin data.

[0049] Alternatively, the mapping model 22 may be generated from one or more of the multiple image data acquired by the image acquisition unit 3 .

[0050] The range in which the outlier point cloud is located is the range in which a moving object may exist during a specific time period. The density of the outlier point cloud (hereinafter also referred to as concentration) indicates the probability of the existence of a moving object. The higher the concentration of the point cloud, the higher the probability of the existence of a moving object. The probability of the existence of a moving object may be displayed as the number of point data in the outlier point cloud.

[0051] 6, the density of the outlier point groups 23a, 23b, and 23c increases in this order. Therefore, the ranges in which the outlier point groups 23a, 23b, and 23c are located have a high probability of containing a moving object in this order.

[0052] As described above, the outlier point cloud output unit 16 may display the probability of a moving object existing within a range where the moving object can exist as the number or density of one or more pieces of point data, or may display it as pixel values ​​according to the number or density of one or more pieces of point data, more specifically, color or monochrome gradation information.

[0053] The more image data pieces acquired by the image acquisition unit 3, the more point data pieces of the outlier point cloud there are, and the higher the density of the outlier point cloud becomes. The outlier point cloud output unit 16 may thin out the point data pieces of the outlier point cloud according to the number of image data pieces before displaying them.

[0054] The outlier point cloud output unit 16 can output an output point cloud and an outlier point cloud for two or more different time periods (e.g., a first time period and a second time period) based on two or more image groups grouped by any time period. That is, the outlier point cloud output unit 16 can generate two or more output data 21 according to two or more time periods. The outlier point cloud output unit 16 can display the probability of a moving object being present in the first time period using the outlier point cloud for the first time period, and can display the probability of a moving object being present in the second time period using the outlier point cloud for the second time period.

[0055] The deviant point cloud output unit 16 may generate two or more mapping models 22 based on the corresponding output point clouds in response to two or more pieces of output data 21. Alternatively, the deviant point cloud output unit 16 may use the same mapping model 22 for some or all of the two or more pieces of output data 21.

[0056] 6 can be used to grasp the flow of people (moving objects) in a factory, for example. Alternatively, the output data 21 may be used to grasp the movements of workers, tools, construction equipment (e.g., cranes), and the like at a building construction site, etc. Alternatively, the output data 21 may be used to grasp the movements of people, food delivery robots, tableware, baggage, and the like at a restaurant, the movements of workers, construction equipment, construction objects (e.g., buildings to be demolished), and the like at an outdoor construction site, or the movements of people, cars, pets, and the like in a residential area.

[0057] The output data 21 of FIG. 6 can be applied, for example, to setting a path plan for a robot or the like, as described below. Alternatively, the output data 21 may be applied to urban planning, urban management, architectural planning, or architectural management, improving efficiency by predicting areas and time periods where there are many moving objects, such as people or vehicles. Furthermore, when placing property (e.g., sensors, etc.) in a location where regular management is difficult, the output data 21 may be used to identify areas where there are many animals or people and avoid those areas to reduce the risk of vandalism or theft. Alternatively, in event management, the output data 21 may be used to plan more efficient event management by identifying areas and time periods where people are concentrated. Furthermore, in autonomous driving or driving assistance, the output data 21 may be used to drive more safely by driving or encouraging the driver to drive more safely when passing through areas with high pedestrian flow. Alternatively, in nature observation, the output data 21 may be used to plan efficient observations by identifying areas and time periods where there are likely to be many animals. Alternatively, in the field of chemistry, the output data 21 may be used to detect accumulations of white gas or the like, or to detect areas where kinetic energy or the like is likely to be generated by chemical reactions. Alternatively, in the medical field, it may be used to determine areas in the human body through which red blood cells pass, in the meteorological field, it may be used to determine areas where clouds and the like are concentrated, or in the astronomy field, it may be used to determine areas where space debris and the like are concentrated.

[0058] In this way, the information processing device 1 according to the first embodiment of the present disclosure outputs the probability of the existence of a moving object using an outlier point cloud, which has conventionally been discarded, when creating a point cloud for a digital twin. The outlier point cloud can output and display the probability of the existence of a moving object within a predetermined range or information on the number of moving objects, based on the density of the point cloud, etc. The outlier point cloud can be applied to estimating areas with a high concentration of moving objects (for example, people), and can be applied to various fields such as path planning for robots, urban planning, etc.

[0059] Second Embodiment In a second embodiment of the present disclosure, an example will be described in which the information processing device 1 in Fig. 1 or the information processing method in Fig. 4 etc. is applied to setting a path plan for a robot. The path plan according to the second embodiment of the present disclosure is not limited to a robot, and may be used as a path plan for a drone or the like.

[0060] FIG. 7 is a diagram showing an example of a UI (User Interface) for setting a path plan for a robot. A screen 30 shown in FIG. 7 has a display section for output data 21, a time period selection section 31, an image number display section 32, and a threshold adjustment section 33. A user can use the screen 30 to set and view a path 35 for a robot 34 on a mapping model 22. The screen 30 can be displayed on, for example, a computer or a smartphone. The information processing device 1 in FIG. 1 may be configured to include a display section for displaying the screen 30.

[0061] The time period selection unit 31 can set a time period for setting a path 35 of the robot 34. The time period selection unit 31 can select a time period according to a group of multiple images generated by the time period group generation unit 17 in Fig. 1. The output data 21 in Fig. 7 displays a mapping model 22 and an outlier point cloud for the selected time period.

[0062] For example, even if the flow of people at a work site or the like varies depending on the time of day, the screen 30 of FIG. 7 can set a route 35 for the robot 34 according to the flow of people at that time of day.

[0063] The image number display unit 32 displays the number of image data used to generate the outlier point cloud. The image number display unit 32 may display the number of image data in the image group for the time period selected by the time period selection unit 31. Alternatively, the image number display unit 32 may display the total number of image data in multiple image groups. The image number display unit 32 serves as a measure of the reliability of the outlier point cloud. The greater the number of image data, the higher the reliability of the outlier point cloud.

[0064] The threshold adjustment unit 33 can adjust and refer to the threshold for determining whether or not the robot 34 is permitted to enter, based on the density of the outlier point cloud, etc. It is desirable for the robot 34 not to enter areas with a high density of outlier point clouds (i.e., areas with a high volume of people, etc.). For example, the threshold adjustment unit 33 can prohibit the robot 34 from entering an area where the density of the outlier point cloud is equal to or greater than a threshold "XX." Furthermore, the threshold adjustment unit 33 can, for example, not recommend the robot 34 from entering an area where the density of the outlier point cloud is equal to or greater than a threshold "YY."

[0065] In the following, an example will be described in which the threshold adjustment unit 33 sets the outlier point group 23a as entry prohibited and the outlier point group 23b as entry not recommended.

[0066] The path 35 of the robot 34 is automatically set by, for example, the screen 30. In this case, the screen 30 sets the path 35 so as to avoid the outlier point cloud 23a where entry is prohibited. Furthermore, the screen 30 sets the path 35 by giving priority to avoiding the outlier point cloud 23b where entry is not recommended, but if there is no other path that can be set, sets the path 35 that passes through the outlier point cloud 23b. Furthermore, the screen 30 sets the path 35 so as to give priority to passing through an area on the mapping model 22 where no outlier point cloud is located or an area where the density of the outlier point cloud is low.

[0067] The path 35 of the robot 34 may be set by a user. In this case, the screen 30 may display a warning if the path 35 set by the user passes through at least one of the outlier point group 23 a where entry is prohibited or the outlier point group 23 b where entry is not recommended.

[0068] 7 may have an approve / disapprove button 36. By pressing the approve / disapprove button 36, the user can approve or disapprove the route 35 automatically set by the screen 30 or the route 35 set by another user. The robot 34 can be driven based on the approved route 35.

[0069] Fig. 8 is a flowchart showing the operation of setting a path plan for a robot. In the following, an example of setting a path 35 using the screen 30 in Fig. 7 will be described, but the path 35 may be set using another UI.

[0070] First, the time period selection unit 31 selects a time period for setting the route 35 (step S31), and the outlier points are projected onto the mapping model (map) 22 for the selected time period (step S32).

[0071] Next, the threshold adjustment unit 33 sets a threshold for the intrusion permitted level (step S33). As a result, a range with a high density of outlier points is set as intrusion prohibited or intrusion not recommended.

[0072] Based on the intrusion level set in step S33, a route 35 for the robot 34 is set on the screen 30 (step S34). A user (e.g., a supervisor) can choose whether to approve or not approve the set route 35 (step S35). If the user approves the route 35, the screen 30 transmits the data of the route 35 to the robot 34 (step S36).

[0073] If the route is not approved by the user, the screen 30 determines whether or not to change the threshold of the intrusion permitted level (step S37). If the threshold of the intrusion permitted level is not to be changed, the screen 30 sets a new route 35 in step S34. If the threshold of the intrusion permitted level is to be changed, the threshold adjustment unit 33 resets the threshold of the intrusion permitted level in step S33, and the screen 30 sets a new route 35 based on this.

[0074] In this way, in the second embodiment of the present disclosure, the outlier point cloud output by the method of the first embodiment can be applied to set a path plan for the robot.

[0075] At least a part of the information processing device 1 described in the first and second embodiments above may be configured with hardware or software. If configured with software, a program that realizes at least a part of the functions of the information processing device 1 may be stored on a recording medium such as a flexible disk or CD-ROM, and may be read and executed by a computer. The recording medium is not limited to removable recording media such as magnetic disks and optical disks, but may also be fixed recording media such as hard disk drives and memories.

[0076] In addition, a program that realizes at least some of the functions of the information processing device 1 may be distributed via a communication line (including wireless communication) such as the Internet. Furthermore, the program may be encrypted, modulated, or compressed and distributed via a wired line or wireless line such as the Internet, or stored on a recording medium.

[0077] The present technology can be configured as follows: (1) An information processing method comprising: acquiring a plurality of image data capturing an image of a target range; extracting a plurality of point cloud data having position information of objects present within the target range from the plurality of image data; overlaying the plurality of point cloud data based on the position information of the objects to extract outlier point cloud data; and outputting a probability of existence of a moving object among the objects present within the target range based on the outlier point cloud data. (2) The information processing method described in (1), comprising: displaying the probability of existence of the moving object based on the outlier point cloud data. (3) The information processing method described in (1) or (2), wherein the plurality of point cloud data includes a plurality of point data each having position information of the object; extracting output point cloud data having position information of a stationary object among the objects from the plurality of point data; and extracting the outlier point cloud data from point data among the plurality of point data that was not extracted as the output point cloud data. (4) The information processing method according to (3), wherein the plurality of image data include first image data and second image data captured at a different imaging position from the first image data, and the plurality of point cloud data include first point cloud data having position information of the object based on an imaging position of the first image data and second point cloud data having position information of the object based on an imaging position of the second image data. (5) The information processing method according to any one of (2) to (4), wherein the outlier point cloud data includes one or more point data, and displays the existence probability of the moving object based on at least one of a density of the one or more point data or a number of the one or more point data. (6) The information processing method according to (5), wherein a plurality of image data captured in a first period and a plurality of image data captured in a second period that is later than the first period are acquired from the plurality of image data, and displays the existence probability of the moving object in the first period based on the plurality of image data captured in the first period, and displays the existence probability of the moving object in the second period based on the plurality of image data captured in the second period.(7) The information processing method according to (5) or (6), further comprising: forming a mapping model indicating the positional relationships of one or more objects existing in the target range based on the plurality of point cloud data; and displaying a range in which a moving object may exist on the mapping model based on the outlier point cloud data. (8) The information processing method according to (7), further comprising: displaying a probability of the moving object existing in the range in which the moving object may exist by the number of the one or more point data, the density of the one or more point data, or a pixel value corresponding to the number or density of the point data. (9) The information processing method according to (7) or (8), further comprising: acquiring, in the mapping model, a movement path of the moving object that passes through the second range with priority over the first range. (10) An information processing device comprising: an acquisition unit that acquires a plurality of image data of an image of a target range, a first extraction unit that extracts a plurality of point cloud data having position information of objects present within the target range from the plurality of image data, a second extraction unit that overlays the plurality of point cloud data based on the position information of the objects and extracts outlier point cloud data, and an output unit that outputs a probability of existence of a moving object among the objects present within the target range based on the outlier point cloud data. (11) An information processing program comprising: a step of acquiring a plurality of image data of an image of a target range, a step of extracting a plurality of point cloud data having position information of objects present within the target range from the plurality of image data, a step of overlaying the plurality of point cloud data based on the position information of the objects and extracting outlier point cloud data, and a step of outputting a probability of existence of a moving object among the objects present within the target range based on the outlier point cloud data.

[0078] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0079] REFERENCE SIGNS LIST 1 Information processing device, 2 Management unit, 3 Image acquisition unit, 4 Time acquisition unit, 11 Point cloud extraction unit, 12 Feature point extraction unit, 13 Imaging position acquisition unit, 14 Three-dimensional projection unit, 15 Matching unit, 16 Outlier point cloud output unit, 17 Time period group generation unit, 21 Output data, 22 Mapping model, 23a, 23b, 23c Outlier point cloud, 30 Screen, 31 Time period selection unit, 32 Image number display unit, 33 Threshold adjustment unit, 34 Robot, 35 Path, 36 Button

Claims

1. An information processing method comprising: acquiring a plurality of image data capturing an image of a target range; extracting a plurality of point cloud data having position information of objects present within the target range from the plurality of image data; superimposing the plurality of point cloud data based on the position information of the objects; extracting outlier point cloud data; and outputting the probability of the existence of a moving object among the objects present within the target range based on the outlier point cloud data.

2. The information processing method according to claim 1, further comprising displaying the probability of the existence of the moving object based on the point cloud data of the outliers.

3. The information processing method according to claim 1, wherein the plurality of point cloud data includes a plurality of point data each having position information of the object, and output point cloud data having position information of stationary objects among the objects is extracted from the plurality of point data, and the outlier point cloud data is extracted from point data among the plurality of point data that was not extracted as the output point cloud data.

4. The information processing method according to claim 3, wherein the plurality of image data includes first image data and second image data captured at a different imaging position from the first image data, and the plurality of point cloud data includes first point cloud data having position information of the object based on the imaging position of the first image data and second point cloud data having position information of the object based on the imaging position of the second image data.

5. The information processing method according to claim 2, wherein the outlier point cloud data includes one or more point data, and the probability of the existence of the moving object is displayed based on at least one of the density of the one or more point data or the number of the one or more point data.

6. The information processing method of claim 5, further comprising: acquiring from the plurality of image data sets a plurality of image data captured in a first period and a plurality of image data captured in a second period subsequent to the first period; displaying the probability of a moving object being present in the first period based on the plurality of image data captured in the first period; and displaying the probability of a moving object being present in the second period based on the plurality of image data captured in the second period.

7. The information processing method according to claim 5, further comprising forming a mapping model showing the positional relationship of one or more objects present in the target range based on the plurality of point cloud data, and displaying a range in which a moving object may exist on the mapping model based on the outlier point cloud data.

8. An information processing method according to claim 7, wherein the probability of the existence of the moving object within a range in which the moving object may exist is displayed by the number of the one or more point data, the density of the one or more point data, or a pixel value corresponding to the number or density of the point data.

9. The information processing method of claim 7, wherein the range in which the moving object may exist includes a first range and a second range in which the probability of the moving object existing is lower than that of the first range, and a movement path of the moving object is obtained in the mapping model that passes through the second range in preference to the first range.

10. An information processing device comprising: an acquisition unit that acquires multiple image data images of a target range; a first extraction unit that extracts multiple point cloud data having position information of objects present within the target range from the multiple image data; a second extraction unit that overlays the multiple point cloud data based on the position information of the objects and extracts outlier point cloud data; and an output unit that outputs the probability of the existence of a moving object among the objects present within the target range based on the outlier point cloud data.

11. An information processing program comprising the steps of: acquiring a plurality of image data images of a target range; extracting a plurality of point cloud data having position information of objects present within the target range from the plurality of image data; superimposing the plurality of point cloud data based on the position information of the objects and extracting outlier point cloud data; and outputting the probability of the existence of a moving object among the objects present within the target range based on the outlier point cloud data.

Images