Path detection device

Abstract

The purpose of an embodiment of the present invention is to highly accurately detect routes in various environments. A route detection system 1 is provided with: a camera 2 that acquires a two-dimensional color picture reflecting a two-dimensional image in a visual field and acquires depth information of an image corresponding to each pixel in the two-dimensional color picture; and a computer 3 that processes the two-dimensional color picture and the depth information. The computer 3 detects the positions of pixels of a line segment image which is an image approximated by line segments on the two-dimensional color picture, identifies the depth corresponding to the positions of the pixels of the line segment image, calculates the height and the slope of the line segment image in a three-dimensional space on the basis of the positions of the pixels of the line segment image and the depth corresponding to the positions, and detects a route candidate in the three-dimensional space on the basis of the height and the slope.

Description

[Technical Field]

[0001] The embodiment relates to a route detection device that detects a route. [Background technology]

[0002] Conventionally, there have been known devices that detect feature points based on the detection results of sensors such as cameras for the purpose of robot control, etc. For example, there are known devices that detect planes in three-dimensional coordinates based on distance images detected by a distance image sensor (see Patent Document 1 below), and devices that process images captured by a television camera to detect feature points such as corners of contours (see Patent Document 2 below). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-85940 [Patent Document 2] Japanese Patent Application Publication No. 6-139357 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional detection methods used in the devices described in Non-Patent Document 1 and Non-Patent Document 2, it is desirable to be able to accurately detect routes in a variety of environments, both indoors and outdoors.

[0005] The present embodiment has been made in view of the above-mentioned problems, and aims to provide a route detection device that can detect routes with high accuracy in various environments. [Means for solving the problem]

[0006] In order to solve the above problem, a path detection device according to one embodiment of the present disclosure includes an acquisition device that acquires a two-dimensional image that reflects a two-dimensional image within a field of view and depth information of the image corresponding to each pixel of the two-dimensional image, and at least one processor that processes the two-dimensional image and the depth information, wherein the at least one processor detects the position of a line segment image, which is an image that approximates a line segment on the two-dimensional image, identifies a depth corresponding to the position of the line segment image from the depth information, calculates the height and inclination of the line segment image in three-dimensional space based on the position of the line segment image and the depth corresponding to the position, and detects path candidates in the three-dimensional space based on the height and inclination. Note that the "line segment" in "line segment image" here is not limited to a straight line, but is a concept that broadly includes curves, broken lines, etc.

[0007] According to the above aspect, the position of a line segment image is detected on a two-dimensional image acquired by an acquisition device, a depth corresponding to the position of the line segment image is identified based on depth information acquired by the acquisition device, the height and inclination of the line segment image in three-dimensional space are calculated from the position of the line segment image and the corresponding depth, and path candidates in the three-dimensional space are identified based on the calculated height and inclination. At this time, by determining the height and inclination of the line segment image according to the environment in which the subject is present, path candidates according to the environment can be detected with high accuracy. [Effects of the Invention]

[0008] According to the embodiment, it is possible to detect a route with high accuracy under various environments. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing a schematic configuration of a path detection system according to a preferred embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of the computer shown in FIG. 1. [Figure 3] 1 is a flowchart showing the procedure of a path detection method according to a preferred embodiment of the present invention. [Figure 4] 1 is a diagram showing a two-dimensional color image acquired by a camera 2 and the positions of line segment images detected on the image by a computer 3. FIG. [Figure 5] 10 is a graph showing the distribution of heights and inclinations of a plurality of line segment images calculated by a computer 3. [Figure 6] FIG. 10 is a diagram showing an example of route candidates output by the computer 3. [Figure 7] FIG. 1 is an external view showing a schematic configuration of a robot 4 showing an application example of this embodiment. [Figure 8] 1 shows a two-dimensional color image captured by a camera 2 and the positions of potential routes detected by a computer 3 on the image. [Figure 9] 1 shows a two-dimensional color image captured by a camera 2 and the positions of potential routes detected by a computer 3 on the image. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, preferred embodiments of the path detection device according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding parts are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] 1, which is a preferred embodiment of the present invention, is a computer system that detects travel routes such as roads, sidewalks, passageways, and stairs in the surrounding environment for the purpose of robot control, etc. The path detection system 1 includes a camera 2, which is an acquisition device that acquires images of the external environment within its field of view, and a computer 3 that processes the images.

[0012] Camera 2 incorporates a distance measuring camera 2a and a color camera 2b. Camera 2 has two functions: (1) operating color camera 2b under the control of computer 3 to capture a two-dimensional image within its field of view and acquire a two-dimensional color image reflecting that image; and (2) operating distance measuring camera 2a under the control of computer 3 to measure the distance from camera 2 corresponding to the position of the two-dimensional image within its field of view and acquire depth information for each pixel of the image. For example, camera 2 acquires data including brightness information for each pixel of each RGB color component as a two-dimensional color image, and simultaneously acquires a depth image showing the depth information for each pixel. The distance measuring function for acquiring a depth image can be realized, for example, by using the principle of infrared triangulation with a stereo camera, by irradiating an infrared pattern and measuring the distance from the change in the pattern, by using an infrared time-of-flight (TOF) camera, or by radar-based distance measurement.

[0013] The computer 3 is a data processing device that detects the positions in three-dimensional space of candidate routes within the field of view of the camera 2 by processing the two-dimensional color images and depth images acquired by controlling the camera 2. Here, in this embodiment, the computer 3 is configured as a single device, but it may also be configured as a plurality of devices, and these plurality of devices may be connected to each other via a wired or wireless network so that data communication is possible.

[0014] Fig. 2 shows the hardware configuration of computer 3. As shown in Fig. 2, computer 3 is physically a computer including a CPU (Central Processing Unit) 101, which is a processor, a RAM (Random Access Memory) 102 or ROM (Read Only Memory) 103, which is a recording medium, a communication module 104, and an input / output module 106, each of which is electrically connected. Note that computer 3 may include a display, keyboard, mouse, touch panel display, etc. as input / output module 106, or may include a data recording device such as a hard disk drive or semiconductor memory. Furthermore, computer 3 may be composed of multiple computers.

[0015] Returning to FIG. 1 , the functional configuration of the computer 3 will be described. The computer 3 includes an image conversion unit 31, an edge detection unit 32, a Hough transform unit 33, a two-dimensional position calculation unit 34, a matching unit 35, a three-dimensional position calculation unit 36, and a path evaluation unit 37. The functional units of the computer 3 shown in FIG. 1 are realized by loading a program onto hardware such as a CPU 101 and RAM 102, which operates a communication module 104, an input / output module 106, and the like under the control of the CPU 101, and by reading and writing data from and to the RAM 102. The CPU 101 of the computer 3 executes the computer program to cause the computer 3 to function as the functional units shown in FIG. 1 and sequentially execute processes corresponding to a path detection method described below. All of the various data required for executing the computer program and the various data generated by the execution of the computer program are stored in internal memories such as the ROM 103 and RAM 102, or in storage media such as a hard disk drive.

[0016] The image conversion unit 31 controls the operation of the camera 2 to acquire two-dimensional color images and depth images, and converts the resolution, which is a parameter used when acquiring the two-dimensional color images, into multiple types to generate two-dimensional color images with converted resolutions. For example, the image conversion unit 31 converts the resolution into a resolution of 640 pixels x 480 pixels and a resolution of 128 pixels x 96 pixels to generate two-dimensional color images with multiple resolutions. However, the types and number of resolutions converted by the image conversion unit 31 can be set arbitrarily. Furthermore, the image conversion unit 31 converts each of the two-dimensional color images with multiple resolutions into grayscale to generate two-dimensional grayscale images with multiple resolutions. Examples of grayscale conversion methods include extracting one of the RGB color components of the two-dimensional color image, or weighting the brightness of the RGB color components to calculate a weighted average.

[0017] The edge detection unit 32 performs edge detection on the two-dimensional grayscale image generated by the image conversion unit 31, and outputs two-dimensional coordinates of points detected as edges on the two-dimensional grayscale image. The edge detection unit 32 uses, for example, the Canny method as a method for edge detection.

[0018] The Hough transform unit 33 detects the positions of multiple line segment images that approximate line segments on the two-dimensional grayscale image based on the two-dimensional coordinates of the edges output from the edge detection unit 32. The Hough transform unit 33 detects the positions of the line segment images using, for example, a probabilistic Hough transform technique, and outputs the two-dimensional coordinates of both ends of the detected line segment images.

[0019] The two-dimensional position calculation unit 34 obtains a mathematical formula representing the position of the line segment image based on the two-dimensional coordinates of both ends of the line segment image output from the Hough transform unit 33. For example, when the two-dimensional coordinates of both ends of the line segment image are detected as (x1, y1) and (x2, y2) in an xy coordinate system based on the image plane of the camera 2, the two-dimensional position calculation unit 34 obtains a mathematical formula representing the position of the line segment image from the following formula: y=(y2-y1){(x-x1) / (x2-x1)}+y1 Then, when x1≠x2, the two-dimensional position calculation unit 34 changes x from x1 to x2 in the above formula to find the x-coordinates and y-coordinates of all pixels of the line segment image on the xy coordinate system. On the other hand, when x1=x2, the two-dimensional position calculation unit 34 sets x=x1 and changes y from y1 to y2 to find the x-coordinates and y-coordinates of all pixels of the line segment image on the xy coordinate system.

[0020] The correspondence unit 35 extracts depth information of the pixel position corresponding to the coordinates (x coordinate, y coordinate) of the line segment image on the two-dimensional coordinate system calculated by the two-dimensional position calculation unit 34 from the depth image. For example, when the depth information included in the depth image is expressed as the z coordinate value of an xyz three-dimensional coordinate system, which is the distance from the xy plane that is the image surface of the two-dimensional color image, the correspondence unit 35 extracts and identifies the z coordinate value as the value indicating the depth of the pixel position of the line segment image. Then, the correspondence unit 35 generates and outputs combinations of the pixel positions (x coordinate, y coordinate) of the line segment image on the two-dimensional coordinate system and the depth information (z coordinate) of the pixel for all pixels calculated by the two-dimensional position calculation unit 34.

[0021] The three-dimensional position calculation unit 36 ​​calculates the position of the line segment image in a predetermined three-dimensional space based on the pixel positions (x coordinates, y coordinates) of the line segment image and the depth (z coordinate) corresponding to the pixel output from the association unit 35. For example, the three-dimensional position calculation unit 36 ​​calculates the coordinates of the line segment image in three-dimensional space in a predetermined three-dimensional coordinate system (a three-dimensional coordinate system in which the horizontal plane is the xy plane, a three-dimensional coordinate system in which the ground surface is the xy plane, etc.) based on the pixel positions of the line segment image and the corresponding depth. The three-dimensional position calculation unit 36 ​​then outputs information about the position of the line segment image in three-dimensional space. The position information to be output may be coordinate information of the start point and end point of the line segment image in three-dimensional space, information about an approximation formula of a straight line, curve, or polygonal line that approximates the line segment image in three-dimensional space, or coordinate information of each pixel of the line segment image in three-dimensional space.

[0022] The path evaluation unit 37 detects path candidates in three-dimensional space from among the multiple line segment images detected by the Hough transform unit 33, based on information about the positions of the multiple line segment images in three-dimensional space calculated by the three-dimensional position calculation unit 36. Specifically, the path evaluation unit 37 calculates the height H [m] and inclination θ [degrees] of the multiple line segment images from a predetermined plane (such as a horizontal plane or the ground) in three-dimensional space. The height H may be calculated as the height of the center of gravity of the line segment image, or as the height of the starting point or ending point of the line segment image. The inclination θ may be calculated as the inclination of a tangent to the center of gravity, starting point, or ending point of the line segment image. The path evaluation unit 37 then extracts path candidates in three-dimensional space from among the multiple line segment images based on the results of comparing the height H and inclination θ of the multiple line segment images with a first threshold and a second threshold, respectively. For example, 0 m and 0.6 m are preset as the first threshold, and 0 degrees and 10 degrees are preset as the second threshold, respectively, and the route evaluation unit 37 calculates the route using the following formula: 0 degrees≦θ≦10 degrees, and 0m≦H≦0.6m The position of the line segment image that satisfies the above condition is extracted as a candidate position of the path.

[0023] Furthermore, the route evaluation unit 37 outputs information on the positions of the extracted route candidates. Examples of output modes include displaying the positions of the route candidates highlighted on a display or the like on the two-dimensional color image acquired by the camera 2, adding the information on the route candidates to map data or the like and outputting the data to an internal or external recording medium, or outputting information on the three-dimensional positions of the route candidates to an internal or external control unit that controls a robot or the like.

[0024] Next, we will explain the steps of the route detection method executed by the above-mentioned route detection system 1. Fig. 3 is a flowchart showing the processing steps of the route detection method. The route detection processing shown in Fig. 3 is started in response to an operation by a user of the route detection system 1, and is repeatedly executed at regular intervals, or when a predetermined state is detected by various sensors, etc.

[0025] 3, when the path detection process is started, the computer 3 controls the camera 2 to acquire a two-dimensional color image and a depth image within the field of view (step S1). Next, the image conversion unit 31 of the computer 3 converts the acquired two-dimensional color image to a predetermined resolution and then converts it into a two-dimensional grayscale image (step S2).

[0026] Thereafter, the edge detection unit 32 of the computer 3 performs edge detection on the two-dimensional grayscale image, and outputs the two-dimensional coordinates of points detected as edges (step S3).Then, the Hough transform unit 33 of the computer 3 performs a probabilistic Hough transform on the two-dimensional coordinates of the points detected as edges, thereby detecting the positions of multiple line segment images (step S4).Thereafter, the two-dimensional position calculation unit 34 of the computer 3 calculates the two-dimensional coordinates of all pixels of the multiple line segment images based on the positions of the multiple line segment images (step S5).

[0027] Next, the correspondence unit 35 of the computer 3 corresponds depth information to the two-dimensional coordinates of the pixels of the multiple line segment images based on the depth image and outputs the corresponding information (step S6). Furthermore, the three-dimensional position calculation unit 36 ​​of the computer 3 calculates the positions (three-dimensional coordinates) of the multiple line segment images in a predetermined three-dimensional space (step S7). Thereafter, the path evaluation unit 37 of the computer 3 calculates the height and inclination of the line segment images from a predetermined plane in the three-dimensional space, and compares the calculation result with a predetermined threshold to extract candidate positions of paths from the positions of the multiple line segment images, and outputs the extracted candidate positions of paths in a predetermined output format (step S8). Furthermore, the computer 3 determines whether to convert the two-dimensional color image to the next resolution (step S9). If conversion to the next resolution is required (step S9; Yes), the processes of steps S2 to S8 are repeated for the two-dimensional color image converted to the next resolution. If conversion to all resolutions is completed (step S9; No), the path detection process ends.

[0028] Next, an example of the results of the route detection process according to the embodiment described above will be shown with reference to Figures 4 to 6. In Figure 4, part (a) shows an example of a two-dimensional color image acquired by the camera 2, part (b) shows an example of the positions of line segment images detected by the computer 3, Figure 5 is a graph showing the distribution of heights and inclinations of multiple line segment images calculated by the computer 3, and Figure 6 is a diagram showing an example of route candidates output by the computer 3.

[0029] As shown in these figures, the route detection process of this embodiment detects the two-dimensional positions of multiple line segment images in a two-dimensional color image acquired by camera 2, and then calculates the height and inclination of the multiple line segment images relative to a predetermined plane in three-dimensional space (FIGS. 4 and 5). In part (b) of FIG. 4, the two-dimensional positions of the detected line segment images are indicated by numerous white lines, and among these white lines, line segment images that have a predetermined length or more and are route candidates are indicated by thick white lines. At this stage, some line segment images that cannot be route candidates (e.g., vertical line segment images) are also displayed as thick white lines. Then, route candidates are extracted from the multiple line segment images based on the calculated height and inclination of the line segment images, and their positions are output. For example, from the distribution of heights and inclinations of line segment images shown in Fig. 5, line segment images in a range (encircled range) where the height H [m] and inclination θ [degrees] of the line segment image in three-dimensional space satisfy, for example, 0 degrees ≦ θ ≦ 10 degrees and 0 m ≦ H ≦ 0.6 m are extracted as route candidates. The results are reflected in three-dimensional space, and as shown in Fig. 6, two line segment images CR on either side of the aisle (displayed on the actual display in a color other than white, for example, so that they can be distinguished from other line segment images as route candidates) are extracted and output as route candidates.

[0030] 7 shows a schematic configuration of a robot 4 illustrating an application example of this embodiment. In addition to the camera 2 and computer 3 that serve as a path detection system 1, the robot 4 further includes a drive mechanism 5 that enables the entire robot 4 to move autonomously under the control of the computer 3. The computer 3 controls the autonomous movement of the drive mechanism 5 based on the positions of the path candidates output by a path evaluation unit 37. The drive mechanism 5 includes, for example, wheels, a steering unit, a drive unit for the wheels or steering unit, and a drive power source such as a battery.

[0031] The effects of the route detection system 1 according to the above-described embodiment will be described.

[0032] According to this embodiment, the position of a line segment image is detected in a two-dimensional color image acquired by camera 2, and a depth corresponding to the position of a pixel of the line segment image is identified based on the depth information acquired by camera 2. The height and inclination of the line segment image relative to a predetermined plane in three-dimensional space are calculated from the pixel position of the line segment image and the corresponding depth, and path candidates in the three-dimensional space are identified based on the calculated height and inclination. By determining the height and inclination of the line segment image in three-dimensional space according to the environment in which the subject is located, path candidates according to the environment can be detected with high accuracy. For example, when controlling a robot to move along a corridor inside a building or an outdoor sidewalk, a path suitable for the robot's movement can be detected. Also, when controlling the robot to move along a wall or pillar of an outdoor building, a path suitable for the robot to move can be detected.

[0033] Here, the computer 3 repeatedly detects route candidates from the two-dimensional color image with the changed resolution while changing the resolution of the two-dimensional image in multiple ways. In this case, by using two-dimensional images with a resolution changed to suit the environment, such as indoors or outdoors, route candidates according to the environment can be detected with higher accuracy. For example, when the image, such as indoors, tends to have linear changes in shading, a route can be detected with high accuracy using an image with a higher resolution, and when the image, such as outdoors, tends to have random changes in shading, a route can be detected with high accuracy using an image with a lower resolution.

[0034] For example, Figures 8 and 9 show examples of detection results based on two-dimensional color images acquired at multiple resolutions in an outdoor environment. Part (a) of Figure 8 shows a two-dimensional color image acquired at a resolution of 640 pixels by 480 pixels and a frame rate of approximately 5 fps, and part (b) of Figure 8 shows the results of detecting route candidates using this two-dimensional color image. Part (a) of Figure 9 shows a two-dimensional color image acquired at a resolution of 128 pixels by 96 pixels and a frame rate of approximately 15 fps, and part (b) of Figure 9 shows the results of detecting route candidates using this two-dimensional color image. In parts (b) of Figure 8 and (b) of Figure 9, the positions extracted as route candidates from the line segment images indicated by thick white lines are indicated by the symbol CR. As these results show, in an environment where straight lines are difficult to capture in an image, such as outdoors, only one route candidate, the clear boundary of the paved road, is detected in a high-resolution image, whereas three route candidates are detected in a low-resolution image, including the ambiguous boundary of the paved road. In this way, highly accurate route detection is achieved depending on how the route is captured in the image depending on the environment.

[0035] The computer 3 also converts the two-dimensional color image to grayscale to generate a two-dimensional grayscale image, and then performs edge detection on the two-dimensional grayscale image to detect the position of the line segment image. This reduces the processing load of edge detection processing from the two-dimensional image, and as a result, the processing load of the entire path detection can be reduced.

[0036] Furthermore, the computer 3 detects the positions of line segment images by performing edge detection on the two-dimensional grayscale image and then performing a probabilistic Hough transform. In this case, all line segment images can be detected regardless of the resolution of the two-dimensional color image, resulting in highly accurate route detection.

[0037] Furthermore, the computer 3 detects route candidates in three-dimensional space based on the results of comparing the height of the line segment image with a first threshold value and the results of comparing the inclination of the line segment image with a second threshold value. In this case, by setting threshold values ​​according to the environment, such as outdoors or indoors, route candidates according to the environment can be detected with higher accuracy.

[0038] Furthermore, according to the configuration of the robot 4 equipped with the above-described path detection system 1, highly accurate autonomous movement control can be achieved using the path detection results.

[0039] The present invention is not limited to the above-described embodiment, and the configuration of the above-described embodiment can be modified in various ways.

[0040] For example, in the above embodiment, the computer 3 repeatedly detects route candidates from a 2D color image converted to multiple resolutions. However, this process is not limited to this. The computer 3 may use some of the RGB color components of the 2D color image to convert those color components to grayscale to generate a 2D grayscale image, and then repeatedly detect route candidates from an image obtained by converting the brightness of each pixel of the extracted color components into a grayscale image while changing the color components extracted from the 2D color image. These color components are parameters for acquiring the 2D grayscale image to be processed. In this case, by using image data of color components appropriate for the environment, such as indoors or outdoors, route candidates appropriate for that environment can be detected with higher accuracy.

[0041] Furthermore, the first and second thresholds used in the computer 3 to detect candidate paths may be changed to any values. For example, when detecting a path on a horizontal plane, the thresholds and criteria are set so that the conditions are that the height of the line segment image is close to the wheel position of the robot and that the inclination of the line segment image is close to the horizontal direction. When detecting the path of a climbing robot, the thresholds and criteria are set so that the conditions are that the inclination of the line segment image is close to the vertical direction and that the bottom end of the line segment image is close to the wheel position of the robot.

[0042] In addition, in order to detect ambiguous line segments as line segment images, the computer 3 may add a process of clustering short line segments to combine line segments with similar direction vectors or positions and detect them as line segment images.

[0043] When a shadow is detected outdoors or the like in the two-dimensional color image acquired by the camera 2, the computer 3 may add the following processing to prevent erroneous detection of the route. -Comparison between candidates (voting between candidate paths) integrating other image features (such as color region clustering). -Comparing the results of 3D point cloud processing (surface detection, etc.) with the path detected using LiDAR (Light Detection and Ranging). Use of image classifiers (pattern recognition technology such as deep learning) to distinguish between shadows and non-shadows.

[0044] When detecting the position of a curved line segment image, the computer 3 may represent it as a large number of short straight lines obtained by processing such as an LSD (Line Segment Detector), or may use a Hough transform to detect curves.

[0045] In addition, the computer 3 may use deep learning to improve the diversity and accuracy of route feature detection in the process of extracting linear or curved line segment images as local features.

[0046] Furthermore, the computer 3 may combine detection data from multiple sensors other than the camera 2, or multiple feature extraction methods, and use clustering to automatically select features such as data and parameters. For example, multiple sensors using infrared or lasers (LiDAR, TOF camera, etc.) may be simultaneously used to acquire depth information, and automatic selection of detection data may be performed (either one of the sensors may be selected depending on the environment, or the output of both sensors may be used) to improve accuracy. In this way, when the environment is different, the accuracy can be improved by compensating for each other's weaknesses.

[0047] Additionally, various processes in the path detection system 1 may be performed in combination with techniques such as artificial intelligence (AI), machine learning, and neural networks.

[0048] The two-dimensional images acquired and processed by the computer 3 are not limited to color images, but may be black and white images or infrared images. By acquiring infrared images, route detection becomes possible even in environments such as at night or in a dark room.

[0049] In the above embodiment, it is preferable that at least one processor, when acquiring a two-dimensional image or depth information, selects from among a plurality of data or parameters and detects route candidates based on the two-dimensional image or depth information obtained as a result of the selection. In this case, data or parameters appropriate for an environment such as outdoors or indoors are selected, and by using the two-dimensional image or depth information obtained as a result, route candidates according to the environment can be detected with higher accuracy.

[0050] Preferably, at least one processor changes the resolution of the two-dimensional image and detects route candidates from the two-dimensional image with the changed resolution. In this case, by using a two-dimensional image with a resolution changed to suit the environment, such as outdoors or indoors, route candidates according to the environment can be detected with higher accuracy.

[0051] It is also preferable that the acquisition device acquires the two-dimensional image as a color image including the brightness of each of a plurality of colors, and that at least one processor changes the color to be extracted from the two-dimensional image and detects route candidates based on the brightness of each pixel of the extracted color. In this case, by using image data of color components appropriate for the environment, such as outdoors or indoors, route candidates appropriate for that environment can be detected with higher accuracy.

[0052] It is also preferable that at least one processor converts the two-dimensional image to a grayscale image to generate a grayscale image and detects the position of the line segment image by detecting edges in the grayscale image. This reduces the processing load of edge detection processing from the two-dimensional image, and as a result, the processing load of the entire path detection can be reduced.

[0053] It is also preferable that at least one processor detects the positions of line segment images by performing edge detection on the grayscale image and then performing a Hough transform, which makes it possible to detect all line segment images regardless of the resolution of the two-dimensional image, resulting in highly accurate route detection.

[0054] It is also preferable that the at least one processor detects route candidates in the three-dimensional space based on the result of comparing the height with the first threshold and the result of comparing the tilt with the second threshold. In this case, by setting the threshold depending on the environment, such as outdoors or indoors, route candidates according to the environment can be detected with higher accuracy.

[0055] Preferably, the vehicle further comprises a drive mechanism that enables autonomous movement, and at least one processor controls the autonomous movement by the drive mechanism based on the detected route candidates. With this configuration, highly accurate autonomous movement control is realized using the route detection results. [Industrial Applicability]

[0056] One aspect of the present disclosure is directed to a route detection device that detects routes and is capable of detecting routes with high accuracy in a variety of environments. [Explanation of symbols]

[0057] 1...Path detection system (path detection device), 2...Camera (acquisition device), 3...Computer, 4...Robot, 5...Drive mechanism, 31...Image conversion unit, 32...Edge detection unit, 33...Hough transform unit, 34...Two-dimensional position calculation unit, 35...Matching unit, 36...Three-dimensional position calculation unit, 37...Path evaluation unit.

Claims

1. a camera device incorporating a camera that captures a two-dimensional image that reflects a two-dimensional image within a field of view and a distance measuring camera that captures depth information of the image corresponding to each pixel of the two-dimensional image; at least one processor for processing the two-dimensional image and the depth information; The at least one processor Detecting the position of a line segment image, which is an image that is approximated to a line segment on the two-dimensional image; Identifying a depth corresponding to the position of the line segment image from the depth information; calculating the positions of the start point and end point of the line segment image in three-dimensional space based on the position of the line segment image and the depth corresponding to the position; calculating the inclination of the tangent at the start point or the end point from a predetermined plane and the height of the start point or the end point from the predetermined plane based on the positions of the line segment image in three-dimensional space; detecting the line segment image that satisfies the conditions that the height is within a range of two predetermined threshold values ​​and the inclination is within a range of two predetermined threshold values ​​as a candidate for a route in three-dimensional space; Path detection device.

2. The at least one processor When acquiring the two-dimensional image or the depth information, a selection is made from a plurality of data or parameters, and the path candidates are detected from the two-dimensional image or the depth information obtained as a result of the selection.

2. The path detection device according to claim 1.

3. The at least one processor changing a resolution of the two-dimensional image, and detecting the route candidates from the two-dimensional image with the changed resolution; 3. The path detection device according to claim 2.

4. the camera device acquires the two-dimensional image as a color image including brightness for each of a plurality of colors; The at least one processor changing the color to be extracted from the two-dimensional image, and detecting the route candidates based on the brightness of each pixel of the extracted color; 3. The path detection device according to claim 2.

5. The at least one processor converting the two-dimensional image into a grayscale image to generate a grayscale image, and detecting edges of the grayscale image to detect the position of the line segment image; The path detection device according to any one of claims 1 to 4.

6. The at least one processor detecting the edge of the grayscale image and then performing a Hough transform to detect the position of the line segment image; The path detection device according to claim 5 .

7. Further provided with a drive mechanism that enables autonomous movement; The at least one processor controlling the autonomous movement by the drive mechanism based on the detected route candidates; The path detection device according to any one of claims 1 to 6.

Images