Obstacle detection system and obstacle detection method

Abstract

The vehicle aligns the two images using ruts on the warehouse floor to detect obstacles.SOLUTION: Selecting a past image captured at a position close to an imaging position of the current image by collating position information of the current image with position information of the past image, and specifying an area including a track generated by traveling of the vehicle as a traveling road associated area from the current image and the past image; Extracting a rut generated by traveling of the vehicle as a feature portion from the current image and the past image, creating a correspondence relationship in which the feature portion of the current image and the feature portion of the past image are associated with each other in the traveling road associated area, aligning the current image and the past image based on the created correspondence relationship, extracting a difference between the current image and the past image, and determining presence or absence of an obstacle based on the extracted difference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an obstacle detection system that detects an obstacle that obstructs the movement of a vehicle. [Background technology]

[0002] Vehicles (e.g., AGVs: Automatic Guided Vehicles) traveling within a warehouse have cameras that monitor the road ahead to detect obstacles on the road, and use images captured by the cameras to detect obstacles that have fallen onto the road.

[0003] The following prior art exists as background art in this technical field: Patent Document 1 (JP 2009-129001 A) describes a driving assistance system that includes a camera attached to a moving body for capturing images of the periphery of the moving body, and that estimates a three-dimensional object area in an image based on a camera image on a camera coordinate plane obtained from the camera, the system comprising: an image capturing means for capturing first and second camera images captured by the camera at first and second different times while the moving body is moving; a movement vector detecting means for extracting n feature points (n is an integer of 2 or more) from the first camera image and detecting a movement vector of each feature point on the camera coordinate plane between the first and second camera images; and a bird's-eye view coordinate plane that projects each camera image and each feature point and each movement vector on the camera coordinate plane onto a bird's-eye view coordinate plane that is parallel to the ground, thereby estimating a three-dimensional object area in an image based on the camera image. The document describes a driving assistance system comprising: a bird's-eye view conversion means for converting a bird's-eye view image into a first and a second bird's-eye view image and detecting the position of each feature point on the first bird's-eye view image and the movement vector of each feature point on the bird's-eye view coordinate plane between the first and second bird's-eye view images; a determination means for determining whether a feature point of interest on the first bird's-eye view image is a ground feature point using constraint conditions that ground feature points located on the ground must satisfy; a movement information estimation means for estimating movement information of the moving object between the first and second times based on the positions on the first bird's-eye view image and the movement vectors on the bird's-eye view coordinate plane for two or more feature points determined to be the ground feature points; and a three-dimensional object area estimation means for estimating the three-dimensional object area based on the first and second bird's-eye view images and the movement information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-129001 Summary of the Invention [Problem to be solved by the invention]

[0005] As mentioned above, obstacles that have fallen onto the roadway are detected using images captured by a forward-looking camera. In a typical obstacle detection method that uses images, thin obstacles (e.g., paper) that have fallen onto the roadway can be detected by aligning two images and detecting the difference between the images. However, on flat surfaces with few features, such as warehouse floors, incorrect alignment can occur, making accurate obstacle detection difficult.

[0006] Therefore, there is a need for technology that can correctly align two images and accurately detect obstacles even on floors with few features.

[0007] The present invention aims to provide an obstacle detection method that detects obstacles by aligning two images using tracks (three-dimensional patterns caused by running marks or dents) that vehicles leave on the floor of a warehouse. [Means for solving the problem]

[0008] A representative example of the invention disclosed in this application is as follows. That is, the obstacle detection system detects obstacles that hinder the traveling of both vehicles, and is configured by a computer having a processing unit that executes predetermined processing and a memory unit connected to the processing unit. The memory unit stores a current image taken by a camera mounted on the vehicle, past images taken in the past by the camera, position information of the vehicle that took the current image, and position information of the vehicle that took the past image. The processing unit compares the position information of the current image with the position information of the past image to select a past image that was taken at a position close to the position where the current image was taken. From the current image and the past image, it identifies an area that includes ruts caused by the traveling of the vehicle as a traveling path-related area. From the current image and the past image, it extracts the ruts caused by the traveling of the vehicle as characteristic parts. In the traveling path-related area, it creates a correspondence relationship that associates the characteristic parts of the current image with the characteristic parts of the past image. Based on the created correspondence relationship, it aligns the current image with the past image. It extracts the differences between the current image and the past image. It determines the presence or absence of an obstacle based on the extracted differences. [Effects of the Invention]

[0009] According to one aspect of the present invention, two images can be correctly aligned, and obstacles can be accurately detected. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] 3 is a schematic diagram of obstacle detection ahead of a vehicle in the first embodiment. FIG. [Figure 2] FIG. 1 is a diagram illustrating a configuration of a logistics system according to a first embodiment. [Figure 3] 1 is a perspective view of a vehicle according to a first embodiment, seen from above. [Figure 4] FIG. 2 is a bottom view of the vehicle of the first embodiment. [Figure 5] 1 is a diagram illustrating a configuration example of an obstacle detection system according to a first embodiment. [Figure 6] 3 is a flowchart of a process executed by the obstacle detection system according to the first embodiment. [Figure 7] FIG. 3 is a diagram showing an example of feature point pairs of a straight road in the first embodiment. [Figure 8] FIG. 3 is a diagram showing an example of feature point pairs of a straight road in the first embodiment. [Figure 9] 3A and 3B are diagrams illustrating examples of feature point pairs for a straight traveling path and a curved traveling path in the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating a configuration example of an obstacle detection system according to a second embodiment. [Figure 11] FIG. 10 is a diagram illustrating a configuration example of an obstacle detection system according to a third embodiment. [Figure 12] 11 is a flowchart of a process executed by an obstacle detection system according to a third embodiment. [Figure 13] FIG. 10 is a diagram illustrating a configuration example of an obstacle detection system according to a fourth embodiment. [Figure 14] 10 is a flowchart of a process executed by an obstacle detection system according to a fourth embodiment. [Figure 15] FIG. 10 is a diagram illustrating a configuration example of an obstacle detection system according to a fifth embodiment. [Figure 16] FIG. 13 is a diagram illustrating a configuration example of an obstacle detection system according to a sixth embodiment. [Figure 17] 13 is a flowchart of a process executed by an obstacle detection system according to a sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0012] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0013] Although various types of information may be described using expressions such as "table," "list," and "queue" as examples, the various types of information may also be expressed using data structures other than these. The various types of information may be data of any structure (for example, structured data or unstructured data), or may be a learning model such as a neural network that generates an output for an input, a genetic algorithm, or a random forest. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be referred to as "XX information." Furthermore, in the following description, the structure of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0014] Any information (for example, at least one of "ID," "name," and "number") may be adopted as information for identifying an element (identification information, identifier). When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0015] When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0016] In the following description, processing performed with a "program" as the subject may be described, but this processing is processing performed by executing a "program." Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the subject of processing performed by executing a program may be the processor. Similarly, the subject of processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of processing performed by executing a program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0017] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium (e.g., a non-transitory recording medium). When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0018] A "processor" may refer to one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be other types of processor devices such as a GPU (Graphics Processing Unit). The at least one processor device may be single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a broader processor device such as a circuit that is a collection of gate arrays written in a hardware description language that performs part or all of the processing (e.g., an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit)).

[0019] The "memory" refers to one or more memory devices, which are an example of one or more "storage devices," and may typically be a primary storage device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

[0020] The “persistent storage device” may be one or more persistent storage devices, which are an example of one or more “storage devices.” The persistent storage device may typically be a non-volatile storage device (e.g., an auxiliary storage device), and specifically may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a non-volatile memory express (NVMe) drive, or a storage class memory (SCM).

[0021] An "interface device" may be one or more interface devices, which may be at least one of the following:

[0022] One or more I / O (Input / Output) interface devices. The I / O interface devices are interface devices for at least one of the I / O devices and the display computer. The I / O devices may be user interface devices, for example, input devices such as a keyboard and a pointing device, or output devices such as a display device. The I / O interface device for the display computer may be a communication interface device.

[0023] One or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)), or two or more heterogeneous communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

[0024] The unit of "date and time" may be coarser or finer than the year, month, day, hour, minute.

[0025] Example 1 FIG. 1 is a schematic diagram of obstacle detection ahead of a vehicle according to the first embodiment.

[0026] The vehicle (AGV) 0001 uses the mounted camera 0002 to capture an image of the floor surface 0004 in front of the vehicle 0001 and detects an obstacle 0003. If the obstacle 0003 is a three-dimensional object with height, it can be easily detected from a single image captured by the camera 0002. However, if the obstacle 0003 is thin and flat (e.g., paper), it is difficult to detect it from a single image captured by the camera 0002. However, if the vehicle 0001 travels over the flat obstacle 0003 without detecting it, it may get caught in the flat obstacle 0003 and break down. Furthermore, if the vehicle 0001 travels over the flat obstacle 0003, one of the left or right drive wheels may slip, causing it to travel in an unintended direction.

[0027] To detect thin obstacles, the two images captured by the camera 0002 are aligned, and obstacles are detected based on whether there is a difference between the images. Therefore, in this embodiment of the present invention, the two images are aligned using the tracks (three-dimensional patterns caused by driving marks and dents) that the vehicle 0001 leaves on the warehouse floor, and the two images are aligned correctly even on a flat surface with few features, such as a warehouse floor, to accurately detect obstacles.

[0028] FIG. 2 is a diagram illustrating the configuration of the logistics system according to the first embodiment.

[0029] The logistics system includes a control server 101 and one or more vehicles 0001. The control server 101 and the vehicles 0001 can communicate with each other via a network 100.

[0030] The vehicle 0001 includes a drive unit 91, a storage unit 92, an interface unit 93, a sensor 94, a battery (not shown), and a controller 90 connected to these.

[0031] The driving device 91 has driving wheels 96, auxiliary wheels 97 (see FIG. 4), and an actuator (not shown) such as a motor for driving the driving wheels 96 to rotate.

[0032] The storage device 92 stores, for example, route information 920. The route information 920 includes, for example, a travel instruction received from the control server 101 and information indicating a travel route specified in the travel instruction.

[0033] The interface device 93 is a device for communicating with the control server 101 via the network 100 in accordance with a predetermined wireless communication method, and may be configured, for example, by a wireless LAN card.

[0034] The sensor 94 is a device for collecting information about the floor surface on which the vehicle 0001 is traveling and various information about the vehicle 0001. The sensor 94 is capable of reading information about markers on the floor surface, for example. The sensor 94 may be, for example, a camera 0002 that acquires information about the environment surrounding the vehicle 0001, a camera that captures images of the markers, a vibration sensor that detects vibrations received by the vehicle 0001, a speed sensor that measures the speed of the vehicle 0001, an acceleration sensor that measures the acceleration of the vehicle 0001, a gyro sensor that measures the orientation of the vehicle 0001, a weight sensor that measures the weight of a load (cargo) carried by the vehicle 0001, or the like.

[0035] The controller 90 is a control device that controls the operation of the vehicle 0001 in accordance with movement instructions from the control server 101, the charge state of the built-in battery, and the like. The controller 90 includes, for example, a processor and a storage device (e.g., memory). A control program 900 is stored in the storage device of the controller 90, and functions as a "storage unit." The processor of the controller 90 executes the control program 900, thereby operating as a "processing unit" and performing various controls of the vehicle 0001.

[0036] The control program 900 has a function (communication program) of transmitting and receiving commands and information to and from the control server 101 via the interface device 93. The control program 900 transmits at least some of the information among the route information 920, device information 921, map information 922, measurement information 923, and performance information 924 to the control server 101 in response to a request from the control server 101 (or without a request). The control program 900 may transmit each of these pieces of information to the control server 101 periodically or irregularly. The control program 900 has a function (movement control program) of controlling the movement of the vehicle 0001 in response to a movement instruction received from the control server 101. The control program 900 controls the drive device 91 so that the vehicle 0001 moves along the movement route specified in the movement instruction. The control program 900 has a function (measurement program) of setting the sensor 94 and storing the output (measurement results) of the sensor 94 in the measurement information 923. The control program 900 has a function (position estimation program) of estimating the position of the vehicle 0001 based on marker information acquired by the sensor 94. The control program 900 is equipped with an obstacle detection system 1000, which will be described later, and the obstacle detection system 1000 constitutes a part of the control program 900.

[0037] The control server 101 has a processor 110, a memory 111, an auxiliary storage device 112, an input device 113, an output device 114, and an interface device 115. The processor 110 operates as a "processing unit," and the memory 111 and the auxiliary storage device 112 operate as a "storage unit."

[0038] The control server 101 may be one or more physical computers having hardware such as a processor 110, a memory 111, an auxiliary storage device 112, an input device 113, an output device 114, and an interface device 115, or may be a system (e.g., a cloud computing system) implemented on one or more physical computers (e.g., a cloud platform). The control server 101 may also be a control system.

[0039] Each device of the control server 101 may be located on a single physical computer, or may be distributed across multiple physical computers. The programs and information stored in the auxiliary storage device 112 may be stored in a single storage device, or may be stored separately across multiple storage devices. The control server 101 may be able to input and output information via a client system that can communicate with the control server 101 through the interface device 115.

[0040] The processor 110 is a device that executes a program to perform predetermined arithmetic processing and controls the overall operation of the control server 101. The processor 110 may be configured with a processing unit, an arithmetic unit, an arithmetic processing unit, and a controller. The memory 111 is a volatile storage area used as a work memory for the processor 110 and may include a non-volatile storage area for storing unchanging data. The input device 113 may be configured with, for example, a mouse or a keyboard, and is used by an operator to input necessary information and instructions to the control server 101. The output device 114 may be configured with a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The interface device 115 is a device that communicates with the vehicle 0001 via the network 100 in accordance with a predetermined wireless communication method, and may be configured with, for example, a wireless LAN card.

[0041] The auxiliary storage device 112 is a non-volatile storage medium that stores various programs and various information. For example, the auxiliary storage device 112 stores map information 104 that describes location information within the warehouse and device information 106 that describes the specifications of the vehicle 0001.

[0042] The control server 101 periodically or irregularly receives information about each vehicle 0001 (for example, at least some of the route information 920, device information 921, map information 922, measurement information 923, and performance information 924) from the vehicle 0001. The control server 101 also receives information about the progress of work input into a terminal of a work station (not shown) installed in the warehouse.

[0043] A WCS (Warehouse Control System) program is stored in the auxiliary storage device 112. The processor 110 executes the WCS program to realize the WCS 102. The WCS 2 comprehensively controls a logistics system including various vehicles 0001 and various material handling facilities.

[0044] 3 and 4 are diagrams showing the appearance of the vehicle 0001 of the first embodiment.

[0045] 3 is a perspective view of the vehicle 0001 as seen from above. The vehicle 0001 is formed in a rectangular parallelepiped shape as a whole. A disk-shaped loading device 95 that can be raised, lowered, and rotated is provided in the center of the top surface of the vehicle 0001. The loading device 95 may also be called a lifting device, a rotation device, a lifting and rotation device, or a table.

[0046] 4 is a bottom view of the vehicle 0001. Drive wheels 96 for turning and moving forward are arranged on the underside (bottom) of the vehicle 0001, and auxiliary wheels 97 are arranged at the four corners of the underside of the vehicle 0001. The drive wheels 96 and auxiliary wheels 97 are the "wheels" of the vehicle 0001.

[0047] The vehicle 0001 shown in Figures 3 and 4 rotates the drive wheels 96 to move underneath the object (e.g., a shelf), then raises the loading device 95 to lift the object after moving, and travels through the warehouse with the object lifted to transport it. The vehicle 0001 can change the direction of the object by rotating the loading device 95 with the object lifted. When the vehicle 0001 itself turns, the vehicle 0001 can change the traveling direction of the vehicle 0001 by rotating the loading device 95 in the direction opposite to the turning direction, thereby turning without rotating the lifted object.

[0048] FIG. 5 is a diagram illustrating an example of the configuration of an obstacle detection system 1000 according to the first embodiment.

[0049] The storage device 203 stores a past image 1002, a photographing position 1003 of the past image 1002, a current image 1004, and a photographing position 1005 of the current image 1004. The past image 1002 is an image previously photographed by a camera 0002 of a vehicle 0001. The photographing position 1003 is the position of the vehicle 0001 that photographed the past image 1002, and is represented by a position within the warehouse. The current image 1004 is an image currently being photographed by a camera 0002 of the vehicle 0001. The photographing position 1005 is the current position of the vehicle 0001, i.e., the position of the vehicle 0001 that photographed the current image 1004, and is represented by a position within the warehouse. The past image 1002 and the current image 1004 may be color images or grayscale images. The storage device 203 has an output unit 1006 that outputs the current image 1004 and the photographing position 1005 to the control device 1001.

[0050] The control device 1001 includes a nearest neighbor image selection unit 1007 , a road area extraction unit 1009 , a feature pair creation unit 1013 , an image transformation unit 1014 , and an obstacle detection unit 1016 .

[0051] The nearest neighbor image selection unit 1007 acquires the current image 1004 stored in the memory device 203, selects the nearest previous image 1008 from the previous images 1002 that was taken at the closest position, and outputs it to the road area extraction unit 1009 and the image transformation unit 1014.

[0052] The travel road area extraction unit 1009 extracts feature points 1010 of the nearest previous image 1008 output from the nearest image selection unit 1007, extracts feature points 1011 of the current image 1004 acquired from the storage device 203, and outputs the extracted feature amounts to the feature pair creation unit 1013. The travel road area extraction unit 1009 may extract feature points 1010 in the travel road related area 10001.

[0053] The feature pair creation unit 1013 compares the extracted feature points 1010 of the nearest previous image 1008 with the feature points 1011 of the current image 1004 to create a feature points pair 1012, and outputs the created feature points pair 1012 to the image transformation unit 1014.

[0054] The image transformation unit 1014 uses the feature point pairs 1012 output from the feature pair creation unit 1013 to transform the positions of each feature point of the nearest previous image 1008 so that it is consistent with the current image 1004, generates an aligned previous image 1015, and outputs the generated previous image 1015 to the obstacle detection unit 1016.

[0055] The obstacle detection unit 1016 compares the past image 1015 output from the image transformation unit 1014 with the current image 1004 to detect the presence or absence 1017 of an obstacle, and outputs the detected presence or absence 1017 of the obstacle.

[0056] The obstacle detection system 1000 may be implemented in the controller 90 of the vehicle 0001 and constitute a part of the control program 900, but may also be implemented in the control server 101 and constitute a part of the WCS .

[0057] FIG. 6 is a flowchart of the process executed by the obstacle detection system 1000 according to the first embodiment.

[0058] First, the control device 1001 acquires a current image 1004 and a photographing position 1005 of the current image 1004 (S1101). The acquired current image 1004 is input to the travel road area extraction unit 1009 and the obstacle detection unit 1016, and the acquired photographing position 1005 is input to the nearest image selection unit 1007.

[0059] Then, the nearest image selection unit 1007 searches for the shooting position 1003 of the past image 1002 stored in the storage device 203 using the shooting position 1005 of the current image 1004 as a key, and selects the past image 1002 whose shooting position 1003 is closest to the shooting position 1005 of the current image 1004 (S1102).

[0060] Then, the travel path area extraction unit 1009 extracts feature points 1010 from the past image 1002 and extracts feature points 1011 from the current image 1004 (S1103). For example, the travel path area extraction unit 1009 may identify the travel path-related area 10001 that includes the travel path by multiplying the pixel values ​​of a predetermined mask image (for straight driving or turning), or may identify the travel path-related area 10001 using a machine learning model trained with data that identifies the travel path area in an image captured by vehicle 0001. Then, the travel path area extraction unit 1009 extracts key points as feature points in the travel path areas of the two images using, for example, the SIFT algorithm.

[0061] Then, the feature pair creating unit 1013 matches the extracted feature points 1010 of the nearest previous image 1008 with the feature points 1011 of the current image 1004 to create feature point pairs 1012 (S1104). For example, the feature pair creating unit 1013 uses the SIFT algorithm to compare the feature amounts of the extracted key points between the two images and associate points with similar feature amounts (see FIGS. 7, 8, and 9).

[0062] Then, the image deformation unit 1014 uses the feature point pairs 1012 to transform the positions of the feature points of the nearest previous image 1008 so that they are aligned with the current image 1004, and generates an aligned previous image 1015 (S1105). The image deformation unit 1014 may use a transformation matrix generated by, for example, homography transformation to align the nearest previous image 1008 with the current image 1004.

[0063] Then, the obstacle detection unit 1016 compares the past image 1015 output from the image transformation unit 1014 with the current image 1004 to determine whether or not an obstacle exists (S1106). For example, the obstacle detection unit 1016 generates a difference image in which the difference in pixel values ​​corresponding to each other between the current image 1004 and the past image 1015 is determined using a predetermined threshold, and determines that an obstacle exists in an area where the area of ​​a group of adjacent pixels, each of which has a pixel value difference exceeding the predetermined threshold, exceeds the predetermined threshold.

[0064] When the control device 1001 determines that there is an obstacle, it determines whether the obstacle is on the travel route of the vehicle 0001. Then, when it determines that there is an obstacle on the travel route of the vehicle 0001, the control device 1001 stops the vehicle 0001 or controls the travel of the vehicle 0001 on a changed travel route. On the other hand, when it determines that there is no obstacle on the travel route of the vehicle 0001, the control device 1001 controls the travel of the vehicle 0001 without changing the travel route.

[0065] In the above-described processing, feature points pairs 1012 are created in the roadway-related region 10001, but feature points pairs 1012 may be created in the entire region of the image. For example, in step S1103, the roadway region extraction unit 1009 extracts feature points 1010 and 1011 in the entire region of the image, in step S1104, the feature pair creation unit 1013 creates feature points pairs 1012 in the entire region of the image, and in step S1105, the image deformation unit 1014 aligns the images using the feature points pairs 1012 in the entire region of the image.

[0066] Alternatively, feature points 1010 and 1011 may be extracted from the entire region of the image, and feature points pair 1012 between the images may be created in the travel path-related region 10001. For example, in step S1103, the travel path region extraction unit 1009 extracts feature points 1010 and 1011 from the entire region of the image, in step S1104 the feature pair creation unit 1013 creates feature points pair 1012 in the travel path-related region 10001, and in step S1105 the image deformation unit 1014 aligns the images using the feature points pair 1012 in the travel path-related region 10001.

[0067] 7 is a diagram showing an example of a feature points pair 1012 of a straight road in Example 1. As shown in the figure, feature points having similar feature amounts among feature points extracted in a road-related area 10001 from an image capturing wheel tracks on a straight road are associated with each other. In the feature points pair shown in FIG. 7, the feature points extracted in the road-related area 10001 are matched, so the area from which feature points must be extracted is narrowed, which reduces the computer resources required for feature amount extraction and enables feature amounts to be extracted quickly.

[0068] Fig. 8 is a diagram showing an example of a feature points pair 1012 of a straight road in Example 1. As shown in the figure, of the feature points extracted from the entire image in which the ruts on the straight road are captured, feature points having similar feature amounts in a road-related area 10001 are associated with each other. In the feature points pair shown in Fig. 7, the feature points extracted from the entire image are matched in the road-related area 10001, so the area from which feature points must be extracted is wide, but an existing feature point extraction engine can be used, reducing the number of development steps.

[0069] 9 is a diagram showing an example of feature point pairs 1012 of a straight traveling path and a turning traveling path in Example 1. Among the feature points of a straight traveling path extracted in a traveling path-related area 10001 from an image in which wheel tracks on the straight traveling path are captured, feature points having similar feature amounts are associated with each other, and among the feature points of a turning traveling path extracted in a traveling path-related area 10002 from an image in which wheel tracks on the turning traveling path are captured, feature points having similar feature amounts are associated with each other.

[0070] As described above, according to the first embodiment of the present invention, thin obstacles can be detected from images of the road captured by vehicle 0001. Furthermore, since a previous image taken at a position close to the current image is used for detection, obstacles can be detected with high accuracy. Furthermore, since image misalignment due to a difference between the capture positions of the previous image and the current image is corrected, obstacles can be detected with high accuracy even if the capture positions of the previous image and the current image are different. Furthermore, since feature pairs are created by extracting areas that have features different from the floor surface, such as wheel ruts, erroneous matching of feature point pairs 1012 can be reduced, and obstacles can be detected with high accuracy.

[0071] Since obstacles are accurately detected, vehicle 0001 can be prevented from traveling over obstacle 0003, preventing breakdowns in vehicle 0001 and reducing repair costs for vehicle 0001. In addition, schedule delays due to vehicle 0001 being stopped can be reduced, reducing transportation losses due to work delays. In addition, stops due to vehicle 0001 deviating from its travel path can be reduced, reducing work delays due to rescue of stopped vehicle 0001.

[0072] <Example 2> Next, a second embodiment of the present invention will be described. The storage device 203 of the second embodiment updates a past image at the same position with a current image. In the second embodiment, differences from the first embodiment will be mainly described, and the same configurations and processes as those of the first embodiment will be assigned the same reference numerals, and descriptions thereof will be omitted.

[0073] FIG. 10 is a diagram illustrating an example of the configuration of an obstacle detection system 1000 according to the second embodiment.

[0074] The obstacle detection system 1000 of the second embodiment is configured by a computer having a control device 1001 and a storage device 203 .

[0075] The storage device 203 stores a previous image 1002, a shooting position 1003 of the previous image 1002, a current image 1004, and a shooting position 1005 of the current image 1004. The storage device 203 has an output unit 1006 and a previous image update unit 2001. The previous image 1002, the shooting position 1003, the current image 1004, the shooting position 1005, and the output unit 1006 are the same as those in the first embodiment described above.

[0076] The past image update unit 2001 searches for the shooting position 1003 of the past image 1002 stored in the storage device 203 using the shooting position 1005 of the current image 1004 as a key, selects a past image 1002 that is within a predetermined error range from the shooting position 1005 of the current image 1004, updates the selected past image 1002 with the current image 1004, and updates the shooting position 1003 of the selected past image 1002 with the shooting position 1005 of the current image 1004. Note that the past image update unit 2001 may update the past image 1002 with the current image 1004 when it is determined that no obstacle is present in the current image 1004 or when it is erroneously determined that an obstacle is present (i.e., the obstacle detection system 1000 detects an obstacle, but the operator determines that there is no obstacle).

[0077] The control device 1001 includes a nearest neighbor image selection unit 1007, a road area extraction unit 1009, a feature pair creation unit 1013, an image transformation unit 1014, and an obstacle detection unit 1016. The configuration of the control device 1001 is the same as that of the first embodiment described above.

[0078] As described above, according to the second embodiment of the present invention, the current image is recorded as a past image, so that a past image taken at a time close to the current image can be used for detection, enabling obstacles to be detected with high accuracy.

[0079] Example 3 Next, a third embodiment of the present invention will be described. A control device 1001 of the third embodiment controls whether to output an obstacle detection result according to the reliability of the obstacle detection result. In the third embodiment, differences from the first embodiment will be mainly described, and the same configurations and processes as those of the first embodiment will be assigned the same reference numerals, and description thereof will be omitted.

[0080] FIG. 11 is a diagram illustrating an example of the configuration of an obstacle detection system 1000 according to the third embodiment.

[0081] The obstacle detection system 1000 of the third embodiment is configured by a computer having a control device 1001 and a storage device 203 .

[0082] The storage device 203 stores a previous image 1002, a shooting position 1003 of the previous image 1002, a current image 1004, and a shooting position 1005 of the current image 1004. The storage device 203 has an output unit 1006. The configuration of the storage device 203 is the same as that of the first embodiment described above.

[0083] The control device 1001 includes a nearest image selection unit 1007, a roadway area extraction unit 1009, a feature pair creation unit 1013, an image deformation unit 1014, an obstacle detection unit 1016, and a detection result usability determination unit 3001. The configurations of the nearest image selection unit 1007, the roadway area extraction unit 1009, the feature pair creation unit 1013, the image deformation unit 1014, and the obstacle detection unit 1016 are the same as those in the first embodiment described above.

[0084] The feature pair creation unit 1013 compares the extracted feature points 1010 of the nearest previous image 1008 with the feature points 1011 of the current image 1004 to create a feature points pair 1012, and outputs the created feature points pair 1012 to the image transformation unit 1014 and the detection result usability determination unit 3001.

[0085] The detection result usability determining unit 3001 calculates the reliability of the feature points pair 1012 from the feature points pair 1012 acquired from the feature pair creating unit 1013 , and outputs the calculated reliability to the obstacle detecting unit 1016 .

[0086] FIG. 12 is a flowchart of the process executed by the obstacle detection system 1000 according to the third embodiment.

[0087] Steps S1101 to S1106 are the same as those in the first embodiment described above.

[0088] After step S1104, the detection result usability determination unit 3001 calculates the reliability of the feature points pairs 1012 from the statistics of the feature points pairs 1012 (S3102). For example, if the number of feature points pairs 1012 created by the feature pair creation unit 1013 is equal to or less than a predetermined threshold, the detection result usability determination unit 3001 determines that the number of generated feature points pairs 1012 is small, that is, the feature points are not associated sufficiently, and the reliability of the feature points pairs 1012 is low. Furthermore, if there is a large variation in the movement amounts of the feature points pairs 1012, it can be determined that the feature points are not associated sufficiently, and the reliability of the feature points pairs 1012 is low. More specifically, the movement amounts of the feature points pairs 1012 are clustered, and if the feature points pairs 1012 whose movement amounts are within a predetermined range are equal to or less than a predetermined threshold (for example, 50%), the detection result usability determination unit 3001 determines that the reliability of the feature points pairs 1012 is low.

[0089] After determining whether an obstacle is present 1017 in step S1106 and calculating the reliability in step S3102, the detection result usability determination unit 3001 compares the calculated reliability with a first threshold (S3103). If the calculated reliability is equal to or greater than the first threshold, the obstacle detection unit 1016 outputs the obstacle presence / absence 1017 determined in step S1106 (S3105). On the other hand, if the calculated reliability is smaller than the first threshold, the detection result usability determination unit 3001 compares the calculated reliability with a second threshold (S3104).

[0090] If the calculated reliability is smaller than the second threshold, the obstacle detection unit 1016 suppresses output of the presence / absence of an obstacle and outputs a message indicating that the presence / absence of an obstacle cannot be determined (S3106). On the other hand, if the calculated reliability is smaller than the second threshold (i.e., if the calculated reliability is smaller than the first threshold but equal to or greater than the second threshold), the detection result usability determination unit 3001 changes the parameters for the process of extracting feature points 1010 from the past image 1002 and the process of extracting feature points 1011 from the current image 1004 (S3101), and executes the process again from step S1103. The detection result usability determination unit 3001 may, for example, generate a large number of parameters using random numbers and select the parameter with the highest reliability through an iterative process. Alternatively, a known optimization method (e.g., steepest descent method) may be used to search for a parameter with the highest reliability. Alternatively, multiple parameter sets may be prepared in advance, and the parameter with the highest reliability may be selected through an iterative process using those parameters.

[0091] As described above, according to the third embodiment of the present invention, even if there are no characteristic ruts or the like on the floor, it is possible to automatically determine whether or not the obstacle detection result can be used, and it is possible to notify the timing when an obstacle can be detected at the beginning of the introduction of the conveyance system or immediately after the floor has been replaced, and it is possible to output an obstacle with high accuracy.In addition, since the conditions of the rut characteristic region extraction unit are changed and detection is attempted again, it is possible to detect an obstacle even when the ruts are unclear.

[0092] Example 4 Next, a fourth embodiment of the present invention will be described. A control device 1001 of the fourth embodiment determines an obstacle by excluding known objects that are not obstacles captured in the travel lane but are temporarily present. In the fourth embodiment, differences from the first embodiment will be mainly described, and the same configurations and processes as those of the first embodiment will be assigned the same reference numerals, and their description will be omitted.

[0093] FIG. 13 is a diagram illustrating an example of the configuration of an obstacle detection system 1000 according to the fourth embodiment.

[0094] The obstacle detection system 1000 of the fourth embodiment is configured by a computer having a control device 1001 and a storage device 203 .

[0095] The storage device 203 stores a past image 1002, a shooting position 1003 of the past image 1002, a shooting time 4018 of the past image 1002, a current image 1004, a shooting position 1005 of the current image 1004, and information 4001 acquired from the WCS. The storage device 203 has an output unit 1006 and an in-path information output unit 4002. The information 4001 acquired from the WCS is information on known objects that may be present on the path, such as information on other vehicles 0001 that can be acquired from the WCS, location information on shelves, information on markers installed on the path, and information on the traveling direction and route of the host vehicle 0001. The in-path information output unit 4002 outputs the information 4001 acquired from the WCS to the control device 1001. Other configurations of the storage device 203 are the same as those in the first embodiment described above.

[0096] The control device 1001 includes a nearest image selection unit 1007, a roadway area extraction unit 1009, a feature pair creation unit 1013, an image deformation unit 1014, an obstacle detection unit 1016, and an in-road road information exclusion unit 4003. The configurations of the nearest image selection unit 1007, the roadway area extraction unit 1009, the feature pair creation unit 1013, and the obstacle detection unit 1016 are the same as those in the first embodiment described above.

[0097] The image transformation unit 1014 uses the feature point pair 1012 output from the feature pair creation unit 1013 to transform the position of each feature point of the nearest previous image 1008 so that it is consistent with the current image 1004, generates an aligned previous image 1015, and outputs the generated previous image 1015 to the lane information exclusion unit 4003.

[0098] The lane information exclusion unit 4003 receives the current image 1004, the aligned past image 1015, the shooting position 1003 of the past image 1002, the shooting time 4018 of the past image 1002, information 4001 obtained from the WCS, and the shooting position 1005 of the current image 1004, and generates an aligned past image 4004 from which the lane information has been excluded and a current image 4005 from which the lane information has been excluded from these input data.

[0099] FIG. 14 is a flowchart of the process executed by the obstacle detection system 1000 according to the fourth embodiment.

[0100] Steps S1101 to S1105 are the same as those in the first embodiment described above.

[0101] The in-road information output unit 4002 acquires information 4001 from the WCS (S4001). The information 4001 acquired by the in-road information output unit 4002 from the WCS includes information on other vehicles 0001, shelf position information, information on markers installed on the road, and information on the traveling direction and route of the vehicle 0001.

[0102] The intra-roadway information exclusion unit 4003 generates intra-roadway information (S4002).

[0103] After the images are aligned in step S1105 and the inside lane information is generated in step S4002, the inside lane information exclusion unit 4003 determines whether the current image 1004 includes inside lane information, and if the current image 1004 includes inside lane information, excludes the inside lane information from the current image 1004 and generates a past image 4004 from which the inside lane information has been excluded (S4003). Note that the inside lane information exclusion unit 4003 may also determine whether the aligned past image 1015 includes inside lane information, and if the aligned past image 1015 includes inside lane information, acquire inside lane information for the shooting position 1003 and shooting time of the past image 1015, exclude the acquired inside lane information from the past image 1015, and generate an aligned past image 4004 from which the inside lane information has been excluded.

[0104] Furthermore, when the intra-roadway information exclusion unit 4003 generates intra-roadway information from known objects detected using a known object determination model, it can exclude the intra-roadway information from the image without acquiring the intra-roadway information from the WCS. This known object determination model may be a machine learning model trained with images of known objects, or a model that detects known objects by pattern matching with images of known objects stored in advance.

[0105] Furthermore, when recording the past image in the storage device 203, the inside lane information may be excluded from the past image, and the past image from which the inside lane information has been excluded may be stored in the storage device 203.

[0106] The intra-lane information exclusion unit 4003 may generate a mask image of the range of the intra-lane information and set the range of the intra-lane information to an area where obstacles are not detected. Also, the intra-lane information exclusion unit 4003 may delete the intra-lane information from the current image 1004 and correct the deleted range to generate an image in which the intra-lane information does not exist.

[0107] Then, the obstacle detection unit 1016 compares the past image 1015 or 4004 with the current image 1004 or 4005 in an area outside the range of the in-road lane information to determine whether or not an obstacle exists (S4004). For example, the in-road lane information exclusion unit 4003 generates a difference image in which the difference in pixel values ​​corresponding to each other between the past image 4004 and the current image 4005 is determined using a predetermined threshold, and determines that an obstacle exists in an area where the area of ​​a group of adjacent pixels, each of which has a pixel value difference exceeding the predetermined threshold, exceeds the predetermined threshold.

[0108] As described above, according to the fourth embodiment of the present invention, an object that appears temporarily in a roadway image is not determined to be an obstacle, thereby enabling accurate detection of an obstacle.

[0109] <Example 5> Next, a fifth embodiment of the present invention will be described. A control device 1001 of the fifth embodiment identifies the photographing position of an image using markers installed in a warehouse. In the fifth embodiment, differences from the first embodiment will be mainly described, and the same configurations and processes as those of the first embodiment will be assigned the same reference numerals, and description thereof will be omitted.

[0110] FIG. 15 is a diagram illustrating an example of the configuration of an obstacle detection system 1000 according to a fifth embodiment.

[0111] The obstacle detection system 1000 of the fifth embodiment is configured by a computer having a control device 1001 and a storage device 203 .

[0112] The storage device 203 stores a past image 1002, a photographing position 1003 of the past image 1002, a current image 1004, a photographing position 1005 of the current image 1004, and traveling marker information 5001 of the vehicle 0001. The storage device 203 has an output unit 1006 and a photographing position calculation unit 5002. The traveling marker information 5001 of the vehicle 0001 is information on floor markers photographed by the camera 0002 of the vehicle 0001. The floor markers are provided in the vicinity of the traveling path-related area 10001 (for example, between the traveling path-related areas 10001 when traveling in a straight line or in the center of the traveling path-related area 10001 when turning). When the camera 0002 of the vehicle 0001 photographs the floor markers, the exact positions of the floor markers can be determined. The photographing position calculation unit 5002 estimates the position of the photographed marker and the distance and direction of the photographed marker from the center of the vehicle 0001, and estimates the center position of the vehicle 0001. The photographing position calculation unit 5002 also outputs the photographing position including the marker identification information, distance from the marker, and direction from the marker to the control device 1001, based on the identification information of the floor marker photographed by the camera 0002 of the vehicle 0001, the travel distance from the marker position measured by the encoder attached to the running wheels, and the travel direction from the marker position instructed by the travel command sent from the WCS to the vehicle 0001. The other configuration of the storage device 203 is the same as that of the first embodiment described above.

[0113] The control device 1001 includes a nearest neighbor image selection unit 1007, a roadway area extraction unit 1009, a feature pair creation unit 1013, an image deformation unit 1014, an obstacle detection unit 1016, and an in-road road information exclusion unit 4003. The configurations of the roadway area extraction unit 1009, the feature pair creation unit 1013, the image deformation unit 1014, and the obstacle detection unit 1016 are the same as those in the first embodiment described above.

[0114] The nearest neighbor image selection unit 1007 acquires the current image 1004 stored in the storage device 203, selects the nearest previous image 1008 having the closest shooting position among the previous images 1002, and outputs it to the road area extraction unit 1009 and the image transformation unit 1014. In the fifth embodiment, the shooting position is configured by the identification information of the floor marker, the traveling distance from the marker position, and the traveling direction from the marker position, so that the previous images 1002 can be narrowed down using the identification information of the floor marker, and therefore the nearest previous image 1008 can be easily selected.

[0115] Although an example in which the traveling marker information 5001 of the vehicle 0001 is used has been described above, position information identified by Visual SLAM may be used instead of the traveling marker information 5001 of the vehicle 0001.

[0116] As described above, according to the fifth embodiment of the present invention, the image capturing positions can be managed with the nearest marker as the origin, and the nearest image can be easily selected. In addition, the accumulation of errors in the calculation unit of the capturing position can be suppressed, and the nearest image can be correctly selected.

[0117] Example 6 Next, a sixth embodiment of the present invention will be described. The control device 1001 of the fifth embodiment identifies a roadway-related area 10001 by using vehicle body information. In the sixth embodiment, differences from the first embodiment will be mainly described, and the same configurations and processes as those of the first embodiment will be assigned the same reference numerals, and the description thereof will be omitted.

[0118] FIG. 16 is a diagram illustrating an example of the configuration of an obstacle detection system 1000 according to the sixth embodiment.

[0119] The obstacle detection system 1000 of the sixth embodiment is configured by a computer having a control device 1001 and a storage device 203 .

[0120] The storage device 203 stores a past image 1002, a photographing position 1003 of the past image 1002, a current image 1004, a photographing position 1005 of the current image 1004, and vehicle body information 6001. The past image 1002, the photographing position 1003, the current image 1004, the photographing position 1005, and the output unit 1006 are the same as those in the above-described first embodiment. The vehicle body information 6001 includes the wheel width and wheel spacing of the vehicle 0001, and can be acquired from the device information 106 of the control server 101.

[0121] The control device 1001 includes a nearest neighbor image selection unit 1007, a roadway area extraction unit 1009, a feature pair creation unit 1013, an image deformation unit 1014, and an obstacle detection unit 1016. The configurations of the nearest neighbor image selection unit 1007, the feature pair creation unit 1013, the image deformation unit 1014, and the obstacle detection unit 1016 are the same as those in the first embodiment described above.

[0122] The travel road area extraction unit 1009 generates a travel road-related area 10001 using the vehicle body information 6001. Then, the travel road area extraction unit 1009 extracts feature points 1010 of the nearest previous image 1008 output from the nearest image selection unit 1007 using the generated travel road-related area 10001, extracts feature points 1011 of the current image 1004 acquired from the storage device 203, and outputs the extracted feature amounts to the feature pair creation unit 1013.

[0123] FIG. 17 is a flowchart of the process executed by the obstacle detection system 1000 according to the sixth embodiment.

[0124] Steps S1101 to S1102 and steps S1104 to S1106 are the same as those in the first embodiment described above.

[0125] The storage device 203 acquires the vehicle body information 6001 from the WCS that manages the vehicle 0001 (S6001). The storage device 203 may acquire the vehicle body information 6001 from the vehicle 0001.

[0126] Then, the roadway area extraction unit 1009 generates a roadway-related area 10001 in the image where wheel ruts may exist, based on the wheel width and wheel spacing included in the vehicle body information 6001 (S6002). The roadway-related area 10001 can be obtained, for example, by calculating in which area of ​​the image wheel ruts exist, when the positional relationship between the camera 0002 and the wheels is known and the internal parameters of the camera 0002 are also known.

[0127] After the roadway-related area 10001 is generated in step S6002 and the past image is selected in step S1102, the roadway area extraction unit 1009 extracts past image features from the past image within the area where road traces may exist and extracts current image features from the current image (S6003).

[0128] As described above, according to the sixth embodiment of the present invention, the traveling path-related area 10001 is generated using the vehicle body information 6001, and therefore, by using design information when extracting feature points related to the traveling path, erroneous extraction can be reduced. Also, there is no need to set the traveling path-related information in advance, and the obstacle detection system 1000 can be operated efficiently. Furthermore, the traveling path-related area 10001 can be generated by combining the vehicle body information 6001 of multiple types of vehicles 0001.

[0129] Although a number of embodiments have been described in this specification, these embodiments can be implemented in any combination.

[0130] The present invention is not limited to the above-described embodiments, but includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0131] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0132] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0133] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0134] 0001 Vehicle 0002 Camera 0003 Obstacle 0004 Floor 90 Controller 91 Drive unit 92 Storage device 93 Interface Device 94 Sensors 95 Loading device 96 Drive wheels 97 Training wheels 100 Network 101 Control Server 104 Map Information 106 Device information 110 processors 111 memory 112 Auxiliary storage 113 Input Device 114 Output Device 115 Interface Device 203 Storage device 900 Control Program 920 Route Information 921 Device information 922 Map Information 923 Measurement Information 924 Performance Information 1000 Obstacle Detection System 1001 Control device 1002 Past Images 1003 Location of past images 1004 Current Image 1005 Current image capture location 1006 Output section 1007 Nearest image selection unit 1008 Nearest Neighbor Past Images 1009 Traveling road area extraction part 1010, 1011 minutiae 1012 feature point pairs 1013 Feature Pair Creation Unit 1014 Image transformation unit 1015 Past Images 1016 Obstacle detection unit 1017 Obstacles 2001 Past Image Update Section 3001 Detection result availability determination unit 4001 Information 4002 Roadway information output unit 4003 Roadway information exclusion unit 4004 Past Images 4005 Current Image 5001 Travel marker information 5002 Shooting position calculation unit 6001 Vehicle Information 10001, 10002 Road-related areas

Claims

1. An obstacle detection system that detects an obstacle that hinders vehicle travel, The computer is configured with a processing unit that executes a predetermined process and a storage unit connected to the processing unit, the storage unit stores a current image captured by a camera mounted on the vehicle, a past image captured in the past by the camera, location information of the vehicle that captured the current image, and location information of the vehicle that captured the past image; The processing unit comparing the position information of the current image with the position information of the past image to select a past image captured at a position close to the capturing position of the current image; Identifying an area including ruts caused by the vehicle traveling as a traveling path-related area from the current image and the past image; extracting, from the current image and the past image, ruts created by the running of the vehicle as characteristic portions; creating a correspondence relationship between a characteristic part of the current image and a characteristic part of the past image in the travel path-related region; aligning the current image with the past image based on the created correspondence relationship, and extracting a difference between the current image and the past image; An obstacle detection system that determines the presence or absence of an obstacle based on the extracted difference.

2. 2. The obstacle detection system of claim 1, The obstacle detection system is characterized in that the processing unit extracts the feature portion from the travel path-related region.

3. 2. The obstacle detection system of claim 1, The processing unit stores the current image as the past image in the memory unit when it is determined that there is no obstacle in the current image or when an obstacle is erroneously detected.

4. 2. The obstacle detection system of claim 1, The processing unit Calculating the reliability of the created correspondence; An obstacle detection system characterized in that, when the calculated reliability is low, output of the determination result of the presence or absence of the obstacle is suppressed.

5. 2. The obstacle detection system of claim 1, The obstacle detection system is characterized in that the processing unit determines the obstacle by excluding known objects that appear in the travel path-related area.

6. 2. The obstacle detection system of claim 1, The processing unit identifies the photographing position of the current image using the position of a marker installed in the vicinity of the travel path-related area, and stores the identified photographing position in the storage unit.

7. 2. The obstacle detection system of claim 1, The processing unit determining whether the obstacle is on the travel path of the vehicle; When it is determined that the obstacle is on the travel route of the vehicle, the vehicle is stopped or the travel of the vehicle is controlled along a changed travel route; An obstacle detection system, characterized in that, when it is determined that the obstacle is not on the travel route of the vehicle, the system controls the travel of the vehicle without changing the travel route.

8. 2. The obstacle detection system of claim 1, The obstacle detection system is characterized in that the processing unit identifies the roadway-related area using vehicle body information including wheel width and wheel spacing of the vehicle.

9. An obstacle detection method in which an obstacle detection system detects an obstacle that hinders vehicle travel, comprising: the obstacle detection system is configured by a computer having a processing unit that executes predetermined processing and a storage unit connected to the processing unit, the storage unit stores a current image captured by a camera mounted on the vehicle, a past image captured in the past by the camera, location information of the vehicle that captured the current image, and location information of the vehicle that captured the past image; The obstacle detection method includes: the processing unit compares the position information of the current image with the position information of the past image, and selects a past image that was captured at a position close to the capturing position of the current image; The processing unit identifies an area including ruts caused by the vehicle traveling as a traveling path related area from the current image and the past image. the processing unit extracts, from the current image and the past image, ruts generated by the running of the vehicle as feature portions; the processing unit creates a correspondence relationship that associates a characteristic part of the current image with a characteristic part of the past image in the travel path-related region; the processing unit aligns the current image and the past image based on the created correspondence relationship and extracts a difference between the current image and the past image; The obstacle detection method, wherein the processing unit determines the presence or absence of an obstacle based on the extracted difference.

Images