Information processing device, driving space monitoring system, mobile system, information processing method, and program

Abstract

The present invention provides an information processing device that can reduce the risk of travel when a moving object passes through a non-plane surface. [Solution] The information processing device (300) includes a first generation means (311) that generates a distance image using distance measurement data acquired in the travel space of the moving body (1), a detection means (312) that detects non-planar areas in the travel space, a identification means (313) that identifies influential areas that may affect the travel of the moving body in the travel space using information on the non-planar areas and information on the drive wheels of the moving body, and a determination means (314) that determines the travel risk of the moving body using information on the direction of travel of the moving body and information on the influential areas.

Description

Technical Field

[0004] , , , , ,

[0005] , , , , ,

[0003] , , ,

[0001] The present invention relates to an information processing apparatus, a traveling space monitoring system, a moving system, an information processing method, and a program.

Background Art

[0002] Conventionally, in small moving bodies driven by humans, such as senior cars and electric wheelchairs, it is known to detect non-planar surfaces such as steps and grooves so that the moving body does not fall over or the like when crossing steps or grooves, which would affect travel. In Patent Document 1, a ranging image sensor is attached to a moving body, the distance from the floor surface to the step and the height of the step are calculated from the distance information, and it is determined whether the step can be overcome based on the drive torque of its drive wheel motor, the material of the drive wheel and the floor surface. Patent Document 2 discloses a driving support method for detecting a step existing in parallel with a lane, obtaining the entry angle of a moving body for crossing the step from the vehicle travel trajectory and the position of the step, and performing movement control.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] An information processing device as one aspect of the present invention is characterized by comprising: a first generation means for generating a distance image using distance measurement data acquired in the travel space of a moving body; a detection means for detecting a non-planar region in the travel space; a identification means for identifying an influential region that may affect the travel of the moving body in the travel space using information on the non-planar region and information on the drive wheels of the moving body; and a determination means for determining the travel risk of the moving body using information on the direction of travel of the moving body and information on the influential region.

[0007] Other objects and features of the present invention are described in the following examples. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an information processing device that can reduce the risk of travel when a moving object passes through a non-plane surface. [Brief explanation of the drawing]

[0009] [Figure 1] These are schematic diagrams of the moving body in each embodiment. [Figure 2] This is a block diagram of the driving space monitoring system in each embodiment. [Figure 3] These are explanatory diagrams of the imaging optical system and image sensor in each embodiment. [Figure 4] This is an explanatory diagram of a non-planar example in each embodiment. [Figure 5] This flowchart shows the processing of the driving space monitoring system in each embodiment. [Figure 6] This flowchart shows the processing of the region identification unit in each embodiment. [Figure 7] This diagram illustrates the relationship between the drive wheel and the non-plane in each embodiment. [Figure 8] This flowchart shows the processing of the driving risk determination unit in each embodiment. [Figure 9] This diagram illustrates the relationship between the moving body and the non-planar surface in each embodiment. [Figure 10] This is an explanatory diagram illustrating the relationship between the moving body and the non-planar surface in Example 1. [Figure 11] This is an explanatory diagram illustrating the relationship between the moving body and the non-planar surface in Example 2. [Figure 12] This is an explanatory diagram of the mobile system in Example 3. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] Figures 1(a) and 1(b) are schematic diagrams of the mobile body 1 in each embodiment. The mobile body 1 is, for example, a mobility scooter or an electric wheelchair, but is not limited to these. In each embodiment, the left-right direction of the mobile body 1 is described as the x-axis direction, the front-back direction as the y-axis direction, and the up-down direction as the z-axis direction. Furthermore, the +x direction is described as the right direction, the -x direction as the left direction, the +y direction as the front direction, the -y direction as the rear direction, the +z direction as the up direction, and the -z direction as the down direction.

[0012] Figure 1(a) is a schematic view of the mobile body 1 from the left side. Figure 1(b) is a schematic view of the mobile body 1 from above. A travel space monitoring system 100 is mounted in front of the mobile body 1 to monitor the travel space of the mobile body 1. However, each embodiment is not limited to this, and the travel space monitoring system 100 can be mounted at any position. In addition, the mobile body 1 has drive wheels, with a drive wheel 10r on the right front wheel, a drive wheel 10l on the left front wheel, a drive wheel 11r on the right rear wheel, and a drive wheel 11l on the left rear wheel.

[0013] Next, referring to FIG. 2, the configuration of the driving space monitoring system 100 in this embodiment will be described. FIG. 2 is a block diagram of the driving space monitoring system 100. The driving space monitoring system 100 includes a data acquisition unit (acquisition means) 200 and an integration processing unit (information processing device) 300. The data acquisition unit 200 also serves as an imaging unit, and details will be described later. The data acquisition unit 200 and the integration processing unit 300 may be provided in the same housing, or may be provided in separate housings.

[0014] The data acquisition unit 200 has an imaging optical system 210 and an image sensor 220. The imaging optical system 210 is the imaging lens of the data acquisition unit 200 and has a function of forming an image of a subject on the image sensor 220. The imaging optical system 210 is composed of a plurality of lens groups (not shown) and has an exit pupil (not shown) at a position a predetermined distance away from the image sensor 220. In this specification, in the imaging optical system 210, the l, m, and n axes are such that the n axis is parallel to the optical axis of the imaging optical system 210, the l axis and the m axis are perpendicular to each other, and they are axes perpendicular to the optical axis.

[0015] The image sensor 220 is composed of a CMOS (complementary metal oxide semiconductor) sensor or a CCD (charge-coupled device) sensor and has a distance measurement function by the imaging surface phase difference distance measurement method. The subject image formed on the image sensor 220 through the imaging optical system 210 is photoelectrically converted by the image sensor 220 to generate an image signal based on the subject image. By performing image processing on the acquired image signal by the image processing unit 316, a recognition processing image signal can be generated. An image for display on a display unit (not shown) may be generated. Also, the generated image signal can be stored in the buffer memory 320.

[0016] Next, referring to FIGS. 3(a) to (c), the image sensor 220 in this embodiment will be described in detail. FIGS. 3(a) to (c) are explanatory diagrams of the imaging optical system 210 and the image sensor 220.

[0017] Figure 3(a) is a lm cross-sectional view of the image sensor 220. The image sensor 220 in Figure 3(a) is composed of multiple 2x2 pixel groups 221 arranged in a grid. In the pixel group 221, green pixels 221G1 and 221G2 are arranged diagonally, and red pixels 221R and blue pixels 221B are arranged in the other two pixels.

[0018] Figure 3(b) is a schematic diagram of the I-I' cross section of the pixel group 221. Figure 3(c) shows the first light beam 211 received by the first photoelectric conversion unit 226 and the second light beam 212 received by the second photoelectric conversion unit 227. Each pixel has two photoelectric conversion units (first photoelectric conversion unit 226 and second photoelectric conversion unit 227) arranged in the light-receiving layer 225 for photoelectric conversion of the received light. The microlens 222 is arranged so that the exit pupil and the light-receiving layer 225 are optically conjugate. As a result, the first photoelectric conversion unit 226 mainly receives the first light beam 211, and the second photoelectric conversion unit 227 mainly receives the second light beam 212.

[0019] The first photoelectric conversion unit 226 converts the received light beam into electrical signals. Similarly, the second photoelectric conversion unit 227 converts the received light beam into electrical signals. The set of electrical signals generated by the first photoelectric conversion unit 226 of each pixel of the image sensor 220 generates the first distance measuring image signal (distance measuring data). Similarly, the set of electrical signals generated by the second photoelectric conversion unit 227 of each pixel of the image sensor 220 generates the second distance measuring image signal (distance measuring data). From the first distance measuring image signal, the intensity distribution of the image formed on the image sensor 220 by the first light beam 211 can be obtained, and from the second distance measuring image signal, the intensity distribution of the image formed on the image sensor 220 by the second light beam 212 can be obtained. In addition, since the pixel group 221 is equipped with color filters corresponding to the blue, green, and red wavelength ranges, the first distance measuring image signal and the second distance measuring image signal contain three color information. In other words, in this embodiment, each pixel of the image sensor 220 combines both imaging and distance measurement functions. Therefore, the data acquisition unit 200 also functions as an imaging unit. The data acquisition unit 200 and the imaging unit may be configured as a stereo camera, or a combination of an imaging camera and a LiDAR (Light Detection And Ranging) for acquiring distance information.

[0020] Next, the integrated processing unit 300 will be described. The integrated processing unit 300 processes the image signal output from the data acquisition unit 200. Note that in Figure 2, the details of the data acquisition unit 200 and the integrated processing unit 300, as well as their wiring, are omitted for convenience.

[0021] The integrated processing unit 300 includes a System On Chip (SOC) / Field Programmable Gate Array (FPGA) 103, a buffer memory 320, a CPU 330 as a computer, and memory 340 as a storage medium. The CPU 330 performs various controls on the entire driving space monitoring system 100 by executing computer programs stored in memory 340.

[0022] The SOC / FPGA310 includes a distance measurement processing unit (first generation means) 311, a non-planar detection unit (detection means) 312, an area identification unit (identification means) 313, a driving risk determination unit (determination means) 314, and a determination result generation unit (second generation means) 315. The SOC / FPGA310 also includes an image processing unit (image processing means) 316 and a recognition processing unit (recognition means) 317. The distance measurement processing unit 311 receives output data from the data acquisition unit 200 and generates a distance image. The non-planar detection unit 312 detects non-planar areas (areas other than planes, such as steps, grooves, and protrusions) based on the output of the distance measurement processing unit 311. The area identification unit 313 acquires (identifies) areas on the distance image that may affect the driving of the mobile body 1 (influenced areas) based on the output of the non-planar detection unit 312. The driving risk determination unit 314 determines the driving risk of the mobile body 1 based on the output of the area identification unit 313. The image processing unit 316 receives the output data from the data acquisition unit 200 and performs image processing for object recognition. The recognition processing unit 317 and the image processing unit 316 perform object recognition processing from the output.

[0023] The distance measurement processing unit 311 calculates the amount of disparity between the first distance measurement image signal and the second distance measurement image signal. Specifically, the distance measurement processing unit 311 sets a point of interest within the first distance measurement image signal and sets a matching range centered on the point of interest. The matching range is, for example, a rectangle with sides of a predetermined number of pixels centered on the point of interest. Next, the distance measurement processing unit 311 sets a reference point within the second distance measurement image signal and sets a reference range centered on the reference point. The reference range has the same size and shape as the matching range. The distance measurement processing unit 311 calculates the correlation between the first distance measurement image signal included in the matching range and the second distance measurement image signal included in the reference range while sequentially moving the reference point, and sets the reference point with the highest correlation as the corresponding point to the point of interest. The amount of relative positional shift between the point of interest and the corresponding point is the amount of disparity at the point of interest.

[0024] The distance measurement unit 311 can calculate the amount of disparity at multiple pixel positions by calculating the amount of disparity while sequentially moving the point of interest. Known methods can be used to calculate the correlation. For example, a method called NCC (Normalized Cross-Correlation), which evaluates the normalized cross-correlation between image signals, can be used. Alternatively, SSD (Sum of Squared Difference), which evaluates the sum of squared differences between image signals, or SAD (Sum of Absolute Difference), which evaluates the sum of absolute differences, may be used.

[0025] Next, the distance measurement unit 311 converts the parallax amount into a defocus amount, which is the distance from the image sensor 220 to the focal point of the imaging optical system 210, using a predetermined conversion coefficient. When the predetermined conversion coefficient is K, the defocus amount is ΔL, and the parallax amount is d, the parallax amount d can be converted to the defocus amount ΔL using the following equation (1).

[0026] ΔL = K × d ···(1) By performing the above distance information generation process at multiple pixel positions, it is possible to generate a distance image that includes the defocus amounts of multiple pixel positions as distance information.

[0027] Next, the defocus amount is converted to the subject distance. This conversion can be performed using the imaging relationship of the imaging optical system 210. For example, the focal length and principal point position of the imaging optical system 210 can be used for the conversion. Through the above process, a distance image containing the subject distance as distance information can be generated. The generated distance image is output to the non-planar detection unit 312 and stored in the buffer memory 320.

[0028] The non-planar detection unit 312 first obtains a planar distance image, which has been acquired by measuring the distance of only the plane, from the buffer memory 320. The non-planar detection unit 312 then uses the difference between the planar distance image and the distance image input from the distance measurement processing unit 311 (the distance image acquired while the moving object 1 is moving) to acquire the location of non-planar objects (non-planar information) on the distance image, such as steps, holes, and protrusions. The non-planar detection unit 312 also acquires the width, depth, and height (non-planar information) of the non-planar object from the distance image data for the detected non-planar location. Alternatively, the non-planar detection unit 312 may assume that an area with a contrast difference from the surrounding area (a contrast difference greater than a predetermined value) in the image acquired from the image processing unit 316 is a non-planar object. In this case, the non-planar detection unit 312 may consider a location where the output of the distance measurement processing unit 311 in the area with a contrast difference differs from the surrounding distance value as a non-planar object. The non-planar detection unit 312 may also detect non-planar objects based on the output of the recognition processing unit 317. The detected non-planar positions on the distance image, as well as the width, depth, and height of the non-planar areas, are output to the region identification unit 313 and stored in the buffer memory 320.

[0029] The region identification unit 313 takes information such as the position and shape of the drive wheels 10l and 10r from the output of the non-planar detection unit 312 to identify influential regions on the distance image that could pose a driving risk to the moving body 1, and stores the identified influential regions in the driving risk determination unit 314 and buffer memory 320.

[0030] The driving risk determination unit 314 determines the driving risk of the mobile body 1 in the driving area based on the output of the area identification unit 313 and the direction of travel of the mobile body 1, and stores the determined driving risk in the determination result generation unit 315 and the buffer memory 320.

[0031] The judgment result generation unit 315 generates the processing to be performed by the mobile body 1 from the output of the driving risk determination unit 314. For example, if the driving risk determination unit 314 determines that there is a driving risk, the processing generated by the judgment result generation unit 315 is either a warning process for the user (a warning process to stop the mobile body 1) or a process to stop the mobile body 1. Alternatively, if there is a driving risk, the processing generated by the judgment result generation unit 315 may be a vehicle driving route generation process that takes into account the affected area, which is the output of the area identification unit 313 (a process that notifies the user of a driving method that reduces the driving risk). In this case, the object type and coordinate information in the image data, which are the output of the recognition processing unit 317, may also be used. The processing generated by the judgment result generation unit 315 is stored in the buffer memory 320, and the CPU 330 controls the mobile body 1 via the mobile body control unit 400 and presents the information to the user via the information provision unit 500.

[0032] The image processing unit 316 processes the image signals output from the data acquisition unit 200. Note that part or all of the image processing unit 316 may be performed by the signal processing unit stacked on the image sensor 220. Specifically, the image processing unit 316 debayers the image data input from the image sensor 220 according to the Bayer array and converts it into RGB raster image data. Furthermore, it performs various correction processes such as white balance adjustment, distortion correction, gain / offset adjustment, gamma processing, color matrix processing, and lossless compression. However, it is desirable not to perform lossy compression. The processed image data is output to the recognition processing unit 317 and stored in the buffer memory 320.

[0033] The recognition processing unit 317 performs object recognition in the image data by applying a predetermined algorithm to the image data input from the image processing unit 316. The predetermined algorithm may be configured to detect objects using an inference model pre-trained by deep learning by inputting a dataset containing at least steps, holes, and protrusions. As a result of object recognition, the object type and coordinate information within the image data are stored in the buffer memory 320.

[0034] The CPU 330 controls the mobile unit control unit 400 and the information provision unit 500 based on the results of the judgment result generation unit 315. The CPU 330 also sets and controls the data acquisition unit 200 and the SOC / FPGA 310 (not shown). The integrated processing unit 300 may be configured to connect to multiple data acquisition units 200.

[0035] The mobile unit control unit 400 is a unit that incorporates a computer and memory for comprehensively controlling the drive and direction of the mobile unit 1, and is connected to the driving space monitoring system 100 so as to be able to communicate with each other. The mobile unit control unit 400 is configured to output mobile unit control signals to the integrated processing unit 300. The mobile unit control signals output by the mobile unit control unit 400 include information related to the movement (movement state) of the mobile unit, such as the driving speed, driving direction, shift lever status, turn signal status, and the orientation of the mobile unit as determined by a geomagnetic sensor.

[0036] The information provision unit 500 outputs information to the user based on the notified information, based on instructions from the driving space monitoring system 100 or the mobile unit control unit 400. For example, this may include display on a display unit (not shown), warning guidance from an audio output unit (not shown), and driving route guidance.

[0037] In the following, the operation of the mobile body 1 and the method for determining the driving risk based on the shape of steps and holes will be described for each embodiment. [Examples]

[0038] First, with reference to Figures 4(a) to 4(d) to 10(a) to 10(d), the method for determining travel risk to non-plane surfaces in Embodiment 1 of the present invention will be explained. Figures 4(a) to 4(d) are diagrams showing examples where non-plane surfaces exist in the space in which the mobile body 1 travels. Figure 4(a) shows an example where a projection 20 protrudes upward from a plane in the travel space. Figure 4(b) shows an example where a groove 30 is recessed downward from a plane in the travel space. Figure 4(c) shows an example where a straight step 21 exists in the travel space. Figure 4(d) shows an example where a curved step 22 exists in the travel space.

[0039] Next, with reference to Figure 5, the method for determining the driving risk of the mobile body 1 will be explained. Figure 5 is a flowchart showing the processing of the driving space monitoring system 100 (method for determining driving risk).

[0040] First, in step S101, the distance measurement processing unit 311 generates a distance image. Next, in step S102, the non-planar detection unit 312 acquires non-planar information such as the position, width, depth, and height of non-planar areas on the distance image. Then, in step S103, the region identification unit 313 uses the non-planar information acquired by the non-planar detection unit 312 to identify influencing regions on the distance image that affect driving.

[0041] Now, with reference to Figures 6 and 7(a) to (c), the processing of the region identification unit 313 in step S103 will be described. Figure 6 is a flowchart showing the processing of the region identification unit 313.

[0042] First, in step S201, the region identification unit 313 acquires one non-planar information obtained by the non-planar detection unit 312. Next, in step S202, the region identification unit 313 determines whether the non-planar area is higher than the ground. If it is determined that the non-planar area is higher than the ground, the process proceeds to step S203. On the other hand, if it is determined that the non-planar area is lower than the ground, the process proceeds to step S208.

[0043] The process of step S203 will be explained with reference to Figure 7(a). Figure 7(a) is a diagram showing the relationship between the diameter t_d of the drive wheel 10l and the height s_h of the non-planar projection 20. The region identification unit 313 compares the diameter t_d and the height s_h to determine whether the moving body 1 can overcome the projection 20. The region identification unit 313 makes the determination using, for example, the coefficient th_p11 for the diameter t_d and the following conditional equation (2).

[0044] t_d×th_p11 <s_h ···(2) The coefficient th_p11 is a coefficient used to determine the height of the step that the moving body 1 cannot overcome, using the diameter t_d. If condition (2) is met, the drive wheels 10l cannot overcome the protrusion 20, and the moving body 1 may become unable to move due to tipping over or other reasons. If condition (2) is met, proceed to step S205. On the other hand, if condition (2) is not met, proceed to step S204.

[0045] In step S204, the region identification unit 313 compares the diameter t_d and the height s_h, similar to step S203. The region identification unit 313 makes a determination using, for example, the coefficient th_p12 and the following conditional equation (3).

[0046] t_d×th_p12 <s_h ···(3) The coefficient th_p12 is a coefficient used to determine the height that the moving body 1 can overcome, taking into account its direction of travel, using the diameter t_d. If condition (3) is satisfied, proceed to step S206. On the other hand, if condition (3) is not satisfied, proceed to step S207.

[0047] In step S205, the projection 20 is at a height that the moving body 1 cannot overcome, and therefore has a significant impact on the movement of the moving body 1. For this reason, the area identification unit 313 determines that passage is not possible.

[0048] In step S206, the projection 20 is at a height that the moving body 1 can overcome if its direction of travel is taken into consideration, and therefore has little effect on travel. For this reason, the area identification unit 313 determines that passage is possible.

[0049] In step S207, the projection 20 is at a height that does not affect the movement of the moving body 1. Therefore, the region identification unit 313 determines that passage is possible on the same level as on a flat surface.

[0050] The process of step S208 will be explained with reference to Figures 7(b) and (c). Figure 7(b) shows the relationship between the diameter t_d of the drive wheel 10l, the depth h_d of the groove 30, and the depth h_l. Figure 7(c) shows the width t_w of the drive wheel 10l. The region identification unit 313 compares the diameter t_d, height h_d, and depth h_l to determine whether the moving body 1 can overcome the groove 30. The region identification unit 313 makes the determination using, for example, coefficients th_p21 and th_p31 for the diameter t_d, and the following equations (4) and (4).

[0051] t_d×th_p21 <h_d ···(4) t_d×th_p31 <h_l ···(5) The coefficient th_p21 is used to determine the depth of the groove 30 that the moving body 1 cannot overcome when passing through the groove 30, using the diameter t_d. The coefficient th_p31 is used to determine the length of the drive wheel 10l that enters the groove 30, using t_d. If both conditions (4) and (5) are met, the drive wheel 10l will either overturn or be unable to overcome the groove 30 when it enters the groove 30, rendering the moving body 1 unable to move. If both conditions (4) and (5) are met, the process proceeds to step S210. On the other hand, if at least one of conditions (4) and (5) is not met, the process proceeds to step S209.

[0052] In step S209, the diameter t_d and width t_w are compared with the height h_d and depth h_l. Using the coefficients th_p22 and th_p32, the following conditional equations (6) and (7) are used to make a determination.

[0053] t_d×th_p22 <h_d ···(6) t_w×th_p41 <h_l ···(7) The coefficient th_p22 is used to determine the depth to which the moving body 1 can overcome the groove 30, taking into account its direction of travel, using the diameter t_d. The coefficient th_p41 is used to determine the length of the groove depth to which the drive wheels 10l can overcome the groove 30 without getting stuck, taking into account the direction of travel, using the width t_w. If both conditions (6) and (7) are met, proceed to step S206. On the other hand, if at least one of conditions (6) and (7) is not met, proceed to step S207.

[0054] In step S210, the groove 30 has a depth and width that the moving body 1 cannot overcome, thus having a significant impact on its movement. For this reason, the area identification unit 313 determines that passage is impossible.

[0055] In step S211, the groove 30 has a depth and width that the moving body 1 can overcome if the direction of travel is taken into consideration, so it has little effect on travel. For this reason, the area identification unit 313 determines that passage is possible.

[0056] In step S212, the projection 20 does not affect the movement of the moving body 1. Therefore, the region identification unit 313 determines that passage is possible on the same level as on a flat surface.

[0057] In step S213, the region identification unit 313 determines whether processing has been completed for all non-planar areas. If processing has not been completed for all non-planar areas, the process proceeds to step S201. On the other hand, if processing has been completed for all non-planar areas, this flow terminates.

[0058] The coefficients used in conditional equations (2) to (7) may be determined by considering the material of the drive wheel 10l of the moving body 1, the torque of the drive motor for the drive wheel 10l, etc. Also, the drive wheel 10l may be replaced with the drive wheel 10r.

[0059] In step S104 of Figure 5, the driving risk determination unit 314 determines the driving risk using the information of the affected area identified by the area identification unit 313. Now, referring to Figures 8 to 10(a) to (d), the processing of the driving risk determination unit 314 in step S104 will be explained.

[0060] Figure 8 is a flowchart showing the processing of the driving risk determination unit 314. First, in step S301, the driving risk determination unit 314 acquires one affected area information identified by the area identification unit 313. Next, in step S302, the driving risk determination unit 314 determines whether the affected area information indicates no influence or not. If the affected area information indicates no influence, the process proceeds to step S311. On the other hand, if the affected area information indicates an influence, the process proceeds to step S303.

[0061] In step S303, the driving risk determination unit 314 determines whether the impact area information indicates a high impact. If the impact area information indicates a high impact, the process proceeds to step S304. On the other hand, if the impact area information indicates a low impact, the process proceeds to step S305.

[0062] In step S304, the driving risk determination unit 314 determines that the driving risk is A, meaning that passage is not permitted.

[0063] In step S305, the driving risk determination unit 314 uses the width information of the affected area to determine whether or not it is a width that should be avoided. If it is a width that should be avoided, the process proceeds to step S306. On the other hand, if it is not a width that should be avoided, the process proceeds to step S307. Now, with reference to Figure 9, step S305 will be explained in detail. Figure 9 shows the width s_w of the projection 20 and the direction of travel 40 of the moving body 1. First, it is determined whether the width s_w is less than the width th_w using the following formula.

[0064] s_w <th_w ···(8) Here, width th_w is the width that only one of the drive wheels 10l and 10r of the moving body 1 can overcome. In the case of such a narrow area of ​​influence, it is possible to overcome it, but from the standpoint of reducing driving risk, it is recommended to avoid the protrusion 20. Avoidance means selecting a route that does not have an area of ​​influence, for example, in the avoidance direction 41.

[0065] In step S306, the driving risk determination unit 314 determines that the driving risk C is one that should be avoided.

[0066] Now, referring to Figures 10(a) to (d), steps S307 to S310 will be explained. Figures 10(a) and (b) show a linear step 21 as shown in Figure 4(c) in front of the moving body 1, with the first distance from the drive wheel (first front wheel) 10l to the step (influence area) 21 being 50l, and the second distance from the drive wheel (second front wheel) 10r to the step 21 being 50r.

[0067] As shown in Figure 10(a), when distances 50l and 50r are approximately the same, when moving in the direction of travel 40, the drive wheels 10l and 10r enter the step 21 almost simultaneously, thus balancing the left and right sides of the moving body 1 and reducing the risk of driving. On the other hand, as shown in Figure 10(b), when distances 50l and 50r are different (when there is a difference between distances 50l and 50r), the drive wheel 10l enters the step 21 first, causing the moving body 1 to tilt to the right, thus increasing the risk of driving. Therefore, in steps S307 to S310, the risk of driving is determined based on whether the distance from the drive wheels 10l and 10r to the step is approximately the same with respect to the direction of travel 40.

[0068] In step S307, the driving risk determination unit 314 calculates the distances 50r and 50l from the drive wheels 10r and 10l, respectively, to the affected area with respect to the direction of travel 40. Subsequently, in step S308, the driving risk determination unit 314 determines whether or not it is possible to pass through the affected area by whether or not the distances 50r and 50l are approximately the same (whether or not the absolute value of the difference between the distances 50r and 50l is within the range of the first threshold), that is, using the following conditional expression (9).

[0069] |50l-50r| <th_d ···(9) th_d is a threshold value (first threshold) used to determine whether the mobile body 1 can pass over a step, and may be determined by considering the material of the drive wheels 10l and 10r of the mobile body 1, the drive torque of the motor that drives the drive wheel 10l, etc.

[0070] If condition (9) is met, the driving risk determination unit 314 determines that the distances 50r and 50l are approximately the same and proceeds to step S309. On the other hand, if condition (9) is not met, the unit determines that the distances 50r and 50l are not approximately the same and proceeds to step S310.

[0071] In step S309, the driving risk determination unit 314 determines that the driving risk is D, meaning that it is possible to pass through the affected area, as the situation is as shown in Figure 10(a). In other words, the driving risk determination unit 314 determines that the driving risk related to the affected area is low.

[0072] In step S310, the driving risk determination unit 314 determines that the vehicle is in the state shown in Figure 10(b), and that it will be possible to pass if the direction of travel toward the step 21 is changed as shown in Figure 10(a). Therefore, the driving risk determination unit 314 determines that the vehicle will be able to pass if the direction of travel toward the affected area is changed, and the driving risk is B.

[0073] Furthermore, steps S307 to S310 can be applied similarly even when there is a curved step 22 in front of the moving body 1, as shown in Figure 4(d), as shown in Figures 10(c) and (d). The same process can also be applied when steps 21 and 22 are grooves.

[0074] In step S311, the driving risk determination unit 314 determines whether processing for all affected areas has been completed. If processing for all affected areas is completed, this flow process is terminated. On the other hand, if processing for all affected areas is not completed, the process returns to step S301.

[0075] In step S105 of Figure 5, the judgment result generation unit 315 generates a driving risk judgment result to be provided to the user based on the result of the driving risk judgment unit 314. Specifically, if there are areas affected by driving risks A and C, information is provided to encourage the user to take avoidance. If there is an area affected by driving risk B, information is generated to encourage the user to change the direction of travel. At this time, in order to reduce the driving risk, a specific driving route may be calculated taking into account the result of the area identification unit 313.

[0076] Next, in step S106, the CPU 330 determines whether or not there is information to provide to the user based on the result of the judgment result generation unit 315. If there is information to provide to the user, the process proceeds to step S107. On the other hand, if there is no information to provide to the user, this flow is terminated.

[0077] In step S107, the information provision unit 500 provides information to the user based on the result of the judgment result generation unit 315, and this flow ends.

[0078] Steps S105 to S107 may be replaced with a warning to the user, where the mobile unit control 400 is controlled to stop the mobile unit 1 when it approaches the affected areas of driving risks A and B. In addition, the driving space monitoring system 100 may not only determine driving risks but also detect moving objects such as people and bicycles within the driving space and issue a warning to the user based on the results. [Examples]

[0079] Next, with reference to Figures 4(a) to (d) through 9, and Figures 11(a) and (b), the method for determining the driving risk on a non-planar surface in Embodiment 2 of the present invention will be described. In this embodiment, steps S101 to S103 in Figures 4(a) to (d) and 5, and steps S301 to S306 in Figures 6, 7(a) to (c) and 8 are the same as in Embodiment 1, so their explanations will be omitted.

[0080] Referring to Figures 11(a) and (b), the differences between steps S307 and S308 in Figure 8 and those in Embodiment 1 will be explained. Figures 11(a) and (b) are explanatory diagrams of the relationship between the moving body 1 and the non-planar region in this embodiment. Figure 11(a) shows a state in front of the moving body 1 where a straight step 21 exists as shown in Figure 4(c). In Figure 11(a), the angle at which the direction of travel 40 intersects the step 21 is denoted as Θ(°). Figure 11(b) shows a state in front of the moving body 1 where a curved step 22 exists as shown in Figure 4(d). In Figure 11(b), at the intersection of the direction of travel 40 and the step (influence region) 22, the angle at which the tangent to the step 22 intersects the direction of travel 40 is denoted as Θ(°) (0≦Θ≦180). In this case, the closer the angle Θ is to 90°, the smaller the driving risk.

[0081] In step S307, the driving risk determination unit 314 calculates the angle Θ at which the direction of travel 40 intersects with the affected area. Subsequently, in step S308, the driving risk determination unit 314 determines whether or not it is possible to pass through the affected area using the following conditional expression (9) relating to the angle Θ(°). That is, at the intersection of the direction of travel 40 of the moving body 1 and the affected area (step 22), the driving risk determination unit 314 determines whether or not the driving risk related to the step 22 is low based on whether or not the angle Θ between the direction of the tangent to the affected area and the direction of travel 40 is within the range of the second threshold.

[0082] |90-Θ| <th_an ···(9) th_an is a threshold value used to determine whether the mobile body 1 can pass over a step. It may be determined by considering factors such as the material of the drive wheels 10l and 10r of the mobile body 1, and the drive torque of the motor that drives the drive wheel 10l.

[0083] If condition (9) is met (the angle Θ is within the range of the second threshold), it is determined that the mobile body 1 can pass over the step, and the process proceeds to step S309. On the other hand, if condition (9) is not met, it is determined that the mobile body 1 cannot pass over the step, and the process proceeds to step S310. The subsequent processing is the same as in Example 1, so its explanation is omitted. [Examples]

[0084] Next, with reference to Figure 12, an example of a mobile system in which multiple travel space monitoring systems 100 are attached to a mobile body 1 will be described. Figure 12 shows a state in which a travel space monitoring system 100f is attached to the front of the mobile body 1, a travel space monitoring system 100l is attached to the left side, a travel space monitoring system 100r is attached to the right side, and a travel space monitoring system 100b is attached to the rear. The field of view of each travel space monitoring system 100 is set to 110f, 110l, 110r, and 110b. By arranging multiple travel space monitoring systems 100 in this way, it is possible to monitor 360° around the mobile body 1. In addition, a step 23 exists to the left side of the mobile body 1 and is assumed to be within the field of view of travel space monitoring system 100l. In this embodiment, the mobile system monitors the travel space of the mobile body 1 using result data from at least one of the multiple travel space monitoring systems, according to the direction of movement of the mobile body 1.

[0085] In this case, even if the mobile body 1 can move horizontally not only forward and backward but also left and right, the travel space monitoring system 100l can detect the step 23 on the left side, so the travel risk determination described in Embodiment 1 can be performed according to the direction of travel of the mobile body 1. The direction of travel of the mobile body 1 can be obtained from the mobile body control unit 400, but this embodiment is not limited to this. The direction of travel of the mobile body 1 may also be obtained, for example, by determining the movement vector from the distance image output from the distance measurement processing unit 311 of the travel space monitoring systems 100f, 100l, 100r, and 100b, or from the change in image information output from the image processing unit 316 over time.

[0086] Furthermore, when the mobile vehicle 1 detours to the left, in the configuration of Embodiment 1 or Embodiment 2, information about the step 23 cannot be acquired until the step 23 enters the field of view of the driving space monitoring system 100f, which is located in front of the mobile vehicle 1. On the other hand, according to this embodiment, since the step 23 is within the field of view of the driving space monitoring system 100l, the driving risk of the step 23 can be determined in advance, and the user can be provided with a driving experience that reduces driving risk.

[0087] Note that the step 23 may be a groove, and its shape is not limited to a straight line as shown in Figure 12. Also, when the moving body 1 changes direction or detours, the driving risk judgment results of the driving space monitoring systems 100f, 100l, 100r, and 100b may be used to calculate a driving route with less driving risk.

[0088] Furthermore, the data acquisition unit 200 and the integrated processing unit 300, which constitute the driving space monitoring system 100, may be separated from each other. In this case, instead of the driving space monitoring systems 100f, 100l, 100r, and 100b, the data acquisition units 200f, 200l, 200r, and 200b may be installed on the mobile body 1, and the integrated processing unit 300 may be attached to the mobile body 1 separately. Alternatively, the data acquisition units 200f, 200l, 200r, and 200b may be configured to use a shared integrated processing unit 300.

[0089] While each embodiment was described using small mobility devices such as senior scooters or electric wheelchairs as examples, it is not limited to these. Each embodiment can also be applied to, for example, automobiles, motorcycles, bicycles, or electric scooters.

[0090] Furthermore, in each embodiment, the driving risk determination unit 314 may further determine the driving risk using the speed of the moving body 1. For example, the driving risk determination unit 314 can increase the driving risk if the speed of the moving body 1 exceeds a predetermined speed, and decrease the driving risk if the speed of the moving body 1 falls below the predetermined speed. In addition, whether or not to consider speed when determining the driving risk may be changed depending on at least one of the information on the direction of travel of the moving body 1 or the information on the area of ​​influence.

[0091] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0092] According to each embodiment, it is possible to provide an information processing device, a travel space monitoring system, a travel system, an information processing method, and a program that can reduce the risk of travel when a moving object passes through a non-plane surface.

[0093] Each embodiment's disclosure includes the following configuration and method. (Composition 1) A first generation means generates a distance image using distance measurement data acquired in the travel space of a moving object, A detection means for detecting a non-planar region in the aforementioned travel space, A means for identifying an influential region that may affect the movement of the moving body in the travel space, using the information of the non-planar region and the information of the drive wheels of the moving body, The system includes a determination means for determining the travel risk of the moving body using information on the direction of travel of the moving body and information on the affected area. An information processing device characterized by the following: (Configuration 2) The detection means detects the information of the non-planar region using the difference between a pre-acquired planar distance image and a distance image acquired during the movement of the moving object. The information processing device according to configuration 1, characterized by the above. (Composition 3) The detection means detects a region in the distance image where the contrast difference is greater than a predetermined value as the non-planar region. An information processing device according to configuration 1 or 2, characterized by the above. (Composition 4) The system further includes recognition means that perform object recognition processing using the distance measurement data, The detection means detects the non-planar region using the object recognition process. An information processing device according to any one of configurations 1 to 3, characterized by the above. (Composition 5) The system further includes a second generation means that generates information indicating the processing to be performed by the moving body based on the driving risk information obtained by the determination means. An information processing device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) If the information regarding the aforementioned driving risk indicates that such a driving risk exists, the process is a warning process for the user. The information processing apparatus according to configuration 5, characterized by the features described herein. (Composition 7) If the information regarding the driving risk indicates that the driving risk exists, the process is the process of stopping the moving body. The information processing apparatus according to configuration 5, characterized by the features described herein. (Composition 8) If the information regarding the aforementioned driving risk indicates that the aforementioned driving risk exists, the process is to notify the user of a method of travel that reduces the aforementioned driving risk. The information processing apparatus according to configuration 5, characterized by the features described herein. (Composition 9) The drive wheels of the moving body include a first front wheel and a second front wheel. If the absolute value of the difference between the first distance from the first front wheel to the affected area and the second distance from the second front wheel to the affected area is within the range of the first threshold, the determination means determines that the driving risk with respect to the affected area is low. An information processing device according to any one of configurations 1 to 8, characterized by the above. (Composition 10) If, at the intersection of the direction of travel and the affected area, the angle between the direction of the tangent to the affected area and the direction of travel is within the range of the second threshold, the determination means determines that the driving risk with respect to the affected area is low. An information processing apparatus according to any one of configurations 1 to 9, characterized by the above. (Composition 11) The information processing device according to any one of configurations 1 to 10, characterized in that the determination means determines the driving risk using the speed of the moving body. (Composition 12) A driving space monitoring system characterized by comprising an information processing device according to any one of configurations 1 to 11, and an acquisition means for acquiring the distance measurement data. (Composition 13) Mobile and It has multiple driving space monitoring systems, Each of the aforementioned multiple driving space monitoring systems is a driving space monitoring system as described in configuration 11, The mobile body's travel space is monitored using result data from at least one of the multiple travel space monitoring systems, according to the direction of movement of the mobile body. A mobile system characterized by the following features. (Method 1) A process for generating a distance image based on distance measurement data acquired in the space where a moving object is traveling, A step of detecting a non-planar region in the aforementioned travel space, A step of identifying an influential region that may affect the movement of the moving body in the travel space, using the information of the non-planar region and the information of the drive wheels of the moving body. The process includes a step of determining the travel risk of the moving body using information on the direction of travel of the moving body and information on the affected area. An information processing method characterized by the following: (Composition 14) A program characterized by causing a computer to execute each step of the information processing method described in Method 1.

[0094] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its essence. [Explanation of Symbols]

[0095] 1 Mobile Unit 300 Integrated Processing Unit (Information Processing Unit) 311 Distance measurement processing unit (first generation means) 312 Non-planar detection unit (detection means) 313 Area identification part (identification means) 314 Driving risk determination unit (determination means)

Claims

1. A first generation means generates a distance image using distance measurement data acquired in the travel space of a moving object, A detection means for detecting a non-planar region in the aforementioned travel space, A means for identifying an influential region that may affect the movement of the moving body in the travel space, using the information of the non-planar region and the information of the drive wheels of the moving body, The system includes a determination means for determining the travel risk of the moving body using information on the direction of travel of the moving body and information on the affected area. An information processing device characterized by the following:

2. The detection means detects the information of the non-planar region using the difference between a pre-acquired planar distance image and a distance image acquired during the movement of the moving object. The information processing apparatus according to feature 1.

3. The detection means detects a region in the distance image where the contrast difference is greater than a predetermined value as the non-planar region. The information processing apparatus according to feature 1.

4. The system further includes recognition means that perform object recognition processing using the distance measurement data, The detection means detects the non-planar region using the object recognition process. The information processing apparatus according to feature 1.

5. The system further includes a second generation means that generates information indicating the processing to be performed by the moving body based on the driving risk information obtained by the determination means. The information processing apparatus according to feature 1.

6. If the information regarding the aforementioned driving risk indicates that such a driving risk exists, the process is a warning process for the user. The information processing apparatus according to feature 5.

7. If the information regarding the driving risk indicates that the driving risk exists, the process is the process of stopping the moving body. The information processing apparatus according to feature 5.

8. If the information regarding the aforementioned driving risk indicates that the aforementioned driving risk exists, the process is to notify the user of a method of travel that reduces the aforementioned driving risk. The information processing apparatus according to feature 5.

9. The drive wheels of the moving body include a first front wheel and a second front wheel. If the absolute value of the difference between the first distance from the first front wheel to the affected area and the second distance from the second front wheel to the affected area is within the range of the first threshold, the determination means determines that the driving risk with respect to the affected area is low. The information processing apparatus according to feature 1.

10. If, at the intersection of the direction of travel and the affected area, the angle between the direction of the tangent to the affected area and the direction of travel is within the range of the second threshold, the determination means determines that the driving risk with respect to the affected area is low. The information processing apparatus according to feature 1.

11. The information processing device according to claim 1, characterized in that the determination means determines the driving risk using the speed of the moving body.

12. A driving space monitoring system characterized by comprising an information processing device according to any one of claims 1 to 11, and an acquisition means for acquiring the distance measurement data.

13. Mobile and It has multiple driving space monitoring systems, Each of the aforementioned multiple driving space monitoring systems is a driving space monitoring system according to claim 11, The mobile body's travel space is monitored using result data from at least one of the multiple travel space monitoring systems, according to the direction of movement of the mobile body. A mobile system characterized by the following features.

14. A process for generating a distance image based on distance measurement data acquired in the space where a moving object is traveling, A step of detecting a non-planar region in the aforementioned travel space, A step of identifying an influential region that may affect the movement of the moving body in the travel space, using the information of the non-planar region and the information of the drive wheels of the moving body. The process includes a step of determining the travel risk of the moving body using information on the direction of travel of the moving body and information on the affected area. An information processing method characterized by the following:

15. A program characterized by causing a computer to execute each step of the information processing method described in claim 14.

Images