Line detection device, line detection method, program, and storage medium

Abstract

It accurately detects lines of a specific color from images that include road paths. [Solution] The line detection device 10 comprises a first detection unit 100 and a second detection unit 200. The first detection unit 100 extracts pixels whose color is located within a predetermined range from an image including the road on which a moving object travels, and detects lines of a first color contained in the image using the distribution of the extracted pixels in the image. After the first processing unit 100 has performed processing, the second detection unit 200 extracts pixels whose brightness is located within a predetermined brightness range from the image, and identifies lines of a second color different from the first color contained in the image using the distribution of the extracted pixels in the image.

Description

Technical Field

[0001] The present invention relates to a line detection device, a line detection method, a program, and a storage medium for detecting a line drawn on a traveling road.

Background Art

[0002] In recent years, technological development for automatically moving a moving object such as a vehicle has been promoted. In such technologies, it is important to accurately detect a line drawn on a traveling road. Since the meaning of this line differs depending on the color, it is necessary to be able to detect the color of the line as well. For example, in Patent Document 1, a line of a specific color is detected by extracting an edge of a color change in an image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Lines of different colors are used on the traveling road, and each color line has a meaning. Therefore, it is preferable to be able to accurately detect a line of a specific color.

[0005] As an example of the problem to be solved by the present invention, accurately detecting a line of a specific color from an image including a traveling road can be mentioned.

Means for Solving the Problems

[0006] The invention according to claim 1 extracts pixels located within a predetermined range of colors from an image including a traveling road on which a moving object travels, and uses the distribution of the extracted pixels in the image to detect a line of a first color included in the image, a first processing unit, After the first processing unit has performed processing, the second processing unit extracts pixels from the image whose brightness is located within a predetermined brightness range, and uses the distribution of the extracted pixels in the image to identify lines of a second color that are included in the image and are different from the first color. Equipped with, The second processing unit is a line detection device that removes pixels constituting the first color lines detected by the first processing unit from the image, and then performs a process to identify the second color lines in the image after the removal.

[0007] Other inventions include computers, By processing an image that includes the path on which a moving object is traveling, the first colored line contained in the image is detected. After performing the detection process for the first color line, pixels located within a predetermined brightness range are extracted from the image, and lines of a second color different from the first color, which are included in the image, are identified using the distribution of the extracted pixels within the image. This line detection method involves removing pixels that constitute the first colored lines detected by the computer from the aforementioned image, and then performing a process to identify the second colored lines in the image after the removal of the pixels.

[0008] Other inventions involve computers, A first process involves processing an image that includes a road on which a moving object is traveling, thereby detecting a line of a first color contained in the image. After performing the line detection process of the first color, a second process is performed to extract pixels from the image whose brightness is located within a predetermined brightness range, and to identify lines of a second color different from the first color that are included in the image using the distribution of the extracted pixels in the image. Make it run, The program in the second process removes pixels that constitute the first color line detected in the first process from the image, and then identifies the second color line in the image after the removal.

[0009] Another invention is a storage medium that stores a program executable by a computer, The aforementioned program is installed on the computer. A first process involves processing an image that includes a road on which a moving object is traveling, thereby detecting a line of a first color contained in the image. After performing the line detection process of the first color, a second process is performed to extract pixels from the image whose brightness is located within a predetermined brightness range, and to identify lines of a second color different from the first color that are included in the image using the distribution of the extracted pixels in the image. Make it run, In the second process, the storage medium removes pixels that constitute the first color line detected in the first process from the image, and identifies the second color line in the image after the removal. [Brief explanation of the drawing]

[0010] The aforementioned objectives, as well as other objectives, features, and advantages, will become even clearer from the preferred embodiments described below and the accompanying drawings.

[0011] [Figure 1] This figure shows the functional configuration of the line detection device according to the first embodiment. [Figure 2] This figure shows a mobile body equipped with a line detection device. [Figure 3] This figure shows an example of the hardware configuration of a line detection device. [Figure 4] This is a flowchart showing the processes performed by the line detection device. [Figure 5] Figure 4 is a diagram illustrating step S20. [Figure 6] (A) and (B) are diagrams illustrating steps S40 and S60 shown in Figure 4. [Figure 7] (A) and (B) are diagrams illustrating step S80 shown in Figure 4. [Figure 8] This is a diagram illustrating the processing performed by the divided section in the second embodiment. [Figure 9] This figure shows the functional configuration of the line detection device according to the third embodiment. [Figure 10] It is a diagram showing a modification example of FIG. 9. [Figure 11] It is a diagram showing the configuration of a line detection device according to a fourth embodiment. [Figure 12] (A), (B), and (C) are diagrams for schematically explaining the processes performed by the region setting unit. [Figure 13] It is a diagram showing the functional configuration of a line detection device according to a fifth embodiment. [Figure 14] It is a diagram showing an example of the functional configuration of the second detection unit. [Figure 15] It is a flowchart showing an example of the process performed by the line detection device. [Figure 16] It is a diagram for explaining an example of the region to be processed by the second detection unit. [Figure 17] It is a diagram showing the functional configuration of a line detection device according to a sixth embodiment. [Figure 18] (A) and (B) are diagrams for explaining an example of the process performed by the determination unit.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and the description will be omitted as appropriate.

[0013] (First Embodiment) Figure 1 is a diagram showing the functional configuration of the line detection device 10 according to the first embodiment. Figure 2 is a diagram showing a mobile body 40 on which the line detection device 10 is mounted. The line detection device 10 is a device that detects lines drawn on the road on which the mobile body 40 travels, and comprises a division unit 120, an estimation information generation unit 160, and a specific processing unit 180 (first specific processing unit). The division unit 120 generates a plurality of first divided images by dividing an image including the road on which the mobile body travels (hereinafter referred to as the processed image) along a first direction that includes a component in the direction in which the road extends. The estimation information generation unit 160 performs a process of selecting a first candidate pixel for each of the plurality of first divided images. The first candidate pixel is a pixel that is estimated to be a part of the first line drawn on the road. The specific processing unit 180 identifies the first line included in the processed image based on the positions of the plurality of first candidate pixels in the processed image. The first detection unit 100 then outputs information indicating the identified first line (first line information).

[0014] Furthermore, the line detection device 10 includes a distribution information generation unit 140. Therefore, the line detection device 10 can also be considered to include a distribution information generation unit 140, an estimation information generation unit 160, and a specific processing unit 180. The distribution information generation unit 140 generates first distribution information from the processed image, which shows the distribution of pixels that satisfy the criteria. The estimation information generation unit 160 selects first candidate pixels using the first distribution information. The specific processing unit 180 identifies first lines included in the processed image based on the positions of the first candidate pixels in the image described above.

[0015] The embodiment will now be described in detail.

[0016] The moving object 40 is, for example, a vehicle such as an automobile or a motorcycle. In this case, an example of a travel path is a road, and the first line and the second line described later are lines that define lanes. However, the moving object 40 may also be an airplane. In this case, the travel path is a runway.

[0017] As shown in Figure 2, the mobile body 40 is equipped with an imaging device 20 and a control device 30. The imaging device 20 generates a video including the road by photographing the area in front of the mobile body 40. Multiple frame images constituting this video are output to the line detection device 10. The line detection device 10 detects a first line included in the road for each frame image and generates first line information indicating the detected first line. The first line information includes the position of the first line and the color of the first line. This process of generating first line information is performed for each of the multiple frame images. However, the process of generating first line information may be performed only for some of the frame images constituting the video.

[0018] The first-line information is output to the control device 30. The control device 30 is a device that controls the movement of the mobile body 40. If the mobile body 40 is an automobile, the control device 30 is a control device for autonomous driving. The level of autonomous driving performed by the control device 30 is, for example, level 3 or higher, but is not limited to this. The control device 30 uses the first-line information when controlling the movement of the mobile body 40. The control device 30 may not directly control the movement of the mobile body 40, but may generate the information necessary when controlling the movement of the mobile body 40. In this case as well, the control device 30 generates the necessary information using the first-line information. The information generated here may include, for example, information to notify (e.g., display) whether or not a lane change is permitted, or information to notify (e.g., display) whether or not a stop is necessary. This information may be displayed, for example, on a display device that can be visually seen by the operator of the mobile body 40 (e.g., the display of the in-car navigation system).

[0019] As shown in Figure 1, the line detection device 10 includes a division unit 120, a distribution information generation unit 140, an estimation information generation unit 160, and a specific processing unit 180. Details of the processing performed by these units will be described later using a flowchart.

[0020] In the example shown in Figure 2, the line detection device 10 is mounted on the mobile body 40. However, the line detection device 10 may be located outside the mobile body 40. In this case, the line detection device 10 is connected to the imaging device 20 and the control device 30 via a wireless communication line. The control device 30 may also be located outside the mobile body 40, or it may be detachably attached to the mobile body 40. For example, if the line detection device 10 is located outside the mobile body 40 (e.g., on an external server), it is possible to acquire video from the imaging device 20 and analyze the frame images that make up this video to identify lines (e.g., lane markings) drawn on the road traveled by the mobile body 40, and then update the map information using the information used to identify the identified lines. In this case, the map information can be easily maintained.

[0021] Figure 3 shows an example of the hardware configuration of the line detection device 10. The main components of the line detection device 10 are implemented using an integrated circuit. This integrated circuit has a bus 402, a processor 404, a memory 406, a storage device 408, an input / output interface 410, and a network interface 412. The bus 402 is a data transmission path for the processor 404, memory 406, storage device 408, input / output interface 410, and network interface 412 to send and receive data to and from each other. However, the method of connecting the processor 404 and the other components is not limited to bus connection. The processor 404 is an arithmetic processing unit implemented using a microprocessor or the like. The memory 406 is a memory implemented using RAM (Random Access Memory) or the like. The storage device 408 is a storage device implemented using ROM (Read Only Memory) or flash memory or the like.

[0022] The input / output interface 410 is an interface for connecting the line detection device 10 to peripheral devices. In this figure, the imaging device 20 and the control device 30 are connected to the input / output interface 410.

[0023] The network interface 412 is an interface for connecting the line detection device 10 to a communication network. This communication network is, for example, a CAN (Controller Area Network) communication network. The method by which the network interface 412 connects to the communication network may be wireless or wired.

[0024] The storage device 408 stores program modules for realizing each functional element of the line detection device 10. The processor 404 reads these program modules into memory 406 and executes them to realize each function of the line detection device 10.

[0025] Note that the hardware configuration of the integrated circuit described above is not limited to the configuration shown in this figure. For example, the program module may be stored in memory 406. In this case, the integrated circuit does not need to include the storage device 408.

[0026] Figure 4 is a flowchart showing the process performed by the line detection device 10. Figures 5 to 7 are diagrams illustrating the process shown in Figure 4. First, when the imaging device 20 generates frame images that make up the video, the line detection device 10 acquires these frame images as processing images 50. Then, each time the line detection device 10 acquires a frame image, it performs the process shown in Figure 4.

[0027] First, the division unit 120 of the line detection device 10 generates multiple divided images 52 (first divided images) by dividing the processed image 50 along a first direction, as shown in Figure 5 (step S20 in Figure 4). The number of divided images 52 generated from one processed image 50 is, for example, 10 to 30, but is not limited to this. When the imaging device 20 is capturing images in front of the moving object 40, the y-axis direction includes a component in the direction in which the road (e.g., a road) extends. Therefore, in the example shown in Figure 5, the division unit 120 generates divided images 52 by dividing the processed image 50 in the y-axis direction. In this case, the load required for the generation of divided images 52 is reduced.

[0028] Next, the distribution information generation unit 140 performs a process to generate first distribution information for each of the divided images 52, which shows the distribution of pixels that satisfy the criteria (step S40 in Figure 4). The criteria used here is, for example, having a color that belongs to a predetermined range in the color space. For example, when the line detection device 10 detects a yellow line, the "predetermined range" is the range that is recognized as yellow. For example, the distribution information generation unit 140 converts pixels belonging to the predetermined range to 1 and other pixels to 0 (binarization process). The first distribution information then shows the distribution of pixels in a second direction that intersects the first direction described above. For example, if the first direction is the y-axis direction as shown in Figure 5, the second direction is, for example, the x-axis direction. The first distribution information then shows the distribution of the number of pixels that satisfy the criteria in the second direction, as shown in Figures 6(A) and (B).

[0029] Next, the estimation information generation unit 160 uses the first distribution information generated by the distribution information generation unit 140 to select pixels that are estimated to be part of the first line drawn on the road (hereinafter referred to as the first pixel) (step S60 in Figure 4). This process is performed for each of the divided images 52. For example, if the distribution information shows a distribution of the number of pixels that satisfy the criteria in the second direction, as shown in Figure 6, the estimation information generation unit 160 selects the first image using at least one of the standard deviation and variance in the first distribution information. For example, if the standard deviation is less than or equal to the criterion value, or if the variance is less than or equal to the criterion value, the estimation information generation unit 160 selects all of the first candidate pixels as the first pixel. In this case, the estimation information generation unit 160 has determined that the first line is included in the divided image 52. For example, in the case shown in Figure 6(A), since the standard deviation and variance are small, all of the first candidate pixels are selected as the first pixel. On the other hand, as shown in Figure 6(B), if the standard deviation or variance is large, the first pixel is not selected from the first candidate pixels.

[0030] However, the estimation information generation unit 160 may select a portion of the first candidate pixels as the first pixel. In this case, as shown in Figure 6(A), the estimation information generation unit 160 selects, for example, pixels included in the region where the number of pixels is equal to or greater than a reference value in the first distribution information as the first pixel.

[0031] Next, the identification processing unit 180 estimates the position of the first line included in the processed image 50 using the position of the first pixel in the processed image 50 (or its position in the divided image 52) selected by the estimation information generation unit 160 (step S80 in Figure 4). For example, the identification processing unit 180 estimates the position of the first line in the divided image 52 as the average value of the position of the first pixel in the first direction. Then, by connecting the positions of the first line in multiple divided images 52 or by performing regression analysis, the first line included in the processed image 50 is estimated. Alternatively, when estimating the position of the first line in the divided image 52, the mode or median may be used instead of the average value of the position of the first pixel in the first direction.

[0032] Each figure in Figure 7 is an example of step S80 in Figure 4, illustrating the case where the specific processing unit 180 estimates the first line using regression analysis. In this example, a regression line is used. Figure 7(A) is a plot of the estimated position of the first line. The specific processing unit 180 then generates a regression line in this figure and estimates that the generated regression line represents the first line. Figure 7(B) is an example of applying the estimated first line to the processed image 50. In this case, for example, the estimated first line is located in the lower part of the processed image 50 (where the plot in Figure 7(A) exists) and extends to the upper edge of the upper part of the processed image 50.

[0033] As shown in Figure 7(B), the identification processing unit 180 may identify pixels that overlap with the regression line and pixels that are continuous with this pixel as pixels constituting the first line. In other words, the identification processing unit 180 may identify, in each of the divided images 52, the clusters of first pixels that overlap with the regression line as pixels constituting the first line.

[0034] Furthermore, the distribution information generation unit 140 can also select pixels whose brightness meets a predetermined criterion and generate distribution information for the selected pixels (hereinafter referred to as the second distribution information). This criterion may be, for example, greater than or equal to a lower limit, less than or equal to an upper limit, or greater than or equal to both the lower limit and the upper limit. In this case, if the estimation information generation unit 160 and the identification processing unit 180 use the second distribution information instead of the first distribution information, the line detection device 10 can detect white lines.

[0035] As described above, according to this embodiment, the processing performed by the division unit 120, the estimation information generation unit 160, and the identification processing unit 180 does not include edge detection processing. Therefore, the amount of computation required to identify the first line is reduced. Consequently, a high-speed computing device is not required, and as a result, the manufacturing cost of the line detection device 10 is reduced.

[0036] Furthermore, the distribution information generation unit 140 of the line detection device 10 generates first distribution information. The estimation information generation unit 160 uses this first distribution information to select the first pixel (i.e., the pixel estimated to constitute the first line). As a result, the amount of computation performed by the estimation information generation unit 160, i.e., the amount of computation required to select the pixel estimated to constitute the first line, is reduced.

[0037] (Second embodiment) The line detection device 10 according to this embodiment is the same as the line detection device 10 shown in the first embodiment, except for the processing performed by the division unit 120.

[0038] Figure 8 is a diagram illustrating the processing performed by the division unit 120 in this embodiment. In this embodiment, the line detection device 10 does not target the entire processing image 50 for the generation of the divided image 52, but only targets a portion 54 of the processing image 50 for the generation of the divided image 52. For example, the line detection device 10 extracts a portion 54 of the processing image 50 and generates the divided image 52 by dividing this portion 54. In this way, the area that the distribution information generation unit 140, the estimation information generation unit 160, and the specific processing unit 180 should process is limited to a portion 54 of the processing image 50, thus further reducing the amount of computation performed by the line detection device 10.

[0039] The position of part 54 in the processed image 50 is predetermined. For example, if the imaging device 20 is mounted on the mobile body 40, the road is likely to be visible at the bottom of the processed image 50. Therefore, it is preferable that part 54 be set below the mobile body 40.

[0040] (Third embodiment) Figure 9 shows the functional configuration of the line detection device 10 according to the third embodiment. The line detection device 10 according to this embodiment has the same configuration as the line detection device 10 according to the first or second embodiment, except that it includes a data conversion unit 110.

[0041] The processed image 50 generated by the imaging device 20 is an image represented in the RGB color space. The data conversion unit 110 converts this processed image 50 into an image represented in a color space defined by the indicators of hue, lightness (luminance), and saturation, for example, an image represented in the HLS color space (converted image). Note that the HSV color space or Lab color space may be used instead of the HLS color space. The division unit 120, distribution information generation unit 140, estimation information generation unit 160, and specific processing unit 180 then perform processing using this converted processed image 50.

[0042] Depending on the color of the first line, the distribution information generation unit 140 may find it easier to process (and perform binarization on) an image represented in a color space defined by hue, lightness (luminance), and saturation, such as the HLS color space, than an image represented in the RGB color space. In such cases, the line detection device 10 according to this embodiment can detect the first line with higher accuracy compared to directly processing the processed image 50 represented in the RGB color space. This tendency is particularly pronounced when the first line is yellow.

[0043] As shown in Figure 10, the data conversion unit 110 may perform the above-described data conversion process on the segmented image 52 instead of the processed image 50.

[0044] (Fourth embodiment) Figure 11 shows the configuration of the line detection device 10 according to the fourth embodiment. The line detection device 10 according to this embodiment is the same as any of the line detection devices 10 according to the first to third embodiments, except that it has a region setting unit 130. Figure 11 shows a case similar to the third embodiment.

[0045] If the specific processing unit 180 can detect the first line in the first frame image, the region setting unit 130 narrows the area to be processed by the distribution information generation unit 140 in the frame image processed after the first frame image (for example, the next frame image, hereinafter referred to as the second frame image). Specifically, the region setting unit 130 narrows the area to be processed by the distribution information generation unit 140 based on the position of the first line detected in the first frame image in the second direction (for example, the position in the x-axis direction in Figure 7). For example, the region setting unit 130 makes the position of the first line in the second direction in the first frame image the center of the area to be processed. Then, the region setting unit 130 narrows the width of the area to be processed. The region setting unit 130 performs the above processing on, for example, the segmented image 52.

[0046] Each figure in Figure 12 is a schematic diagram illustrating the processing performed by the region setting unit 130. As shown in Figure 12(A), when the identification processing unit 180 identifies a first line L1 in the first frame image (processed image 50a), the region setting unit 130 obtains information indicating the position of that line L1 (for example, information indicating a line obtained by regression analysis) from the identification processing unit 180. Then, as shown in Figure 12(B), the region setting unit 130 sets the region 56 to be processed by the distribution information generation unit 140 for the second frame image (processed image 50b). Subsequently, if the identification processing unit 180 is able to identify the first line L1 in this second frame image (processed image 50b) as well, the region 56 in the third frame image (processed image 50c), which is after the second frame image (processed image 50b), is made narrower than the region 56 in the second frame image (processed image 50b).

[0047] In this way, as detection of the first line L1 continues, the region 56 gradually narrows, but there is a lower limit to its size (e.g., width). That is, the region setting unit 130 ensures that the size (e.g., width) of the region 56 does not fall below the lower limit. It is preferable that the lower limit used here is set to be larger than the width corresponding to the standard deviation reference value that serves as the criterion for determining whether or not the divided image 52 contains a part of the line.

[0048] Furthermore, if the region setting unit 130 fails to identify the first line in any frame image (processed image 50) after setting region 56, it will either expand region 56 or cancel the setting of region 56 in a subsequent frame image (for example, the next frame image).

[0049] The region setting unit 130 may perform the above-described region setting on the processed image 50 before it is processed by the division unit 120 (or on the processed image 50 after it has been returned by the data conversion unit 110), or on the first distribution information generated by the distribution information generation unit 140.

[0050] According to this embodiment, the region setting unit 130 narrows the region 56 that the distribution information generation unit 140 is to process. In this case, the region setting unit 130 sets the region 56 based on the position of the first line L1 in the processed frame image. Therefore, the amount of computation of the line detection device 10 can be reduced while maintaining the detection accuracy of line L1. Furthermore, by narrowing the region 56 as the processing target of the distribution information generation unit 140, the influence of noise that is unnecessary for line L1 detection can also be suppressed. For example, noise can include characters drawn with yellow lines on the road that are different from line L1, or signs drawn with yellow lines.

[0051] (Fifth embodiment) Figure 13 shows the functional configuration of the line detection device 10 according to the fifth embodiment. The line detection device 10 according to this embodiment includes a second detection unit 200 in addition to the first detection unit 100. The first detection unit 100 is the same as in any of the embodiments described above.

[0052] The second detection unit 200 detects lines drawn on the road using the brightness of pixels that make up the processed image 50. Specifically, the second detection unit 200 selects pixels from the processed image 50 whose brightness meets a standard, and uses the selected pixels to detect lines (second lines). One example of this process is binarization.

[0053] In this case, if the brightness standard is not set appropriately, the second detection unit 200 may detect the first line detected by the first detection unit 100 along with other lines. For example, if the first detection unit 100 is intended to detect yellow lines and the second detection unit 200 is intended to detect white lines, the second detection unit 200 may detect yellow lines along with white lines. Therefore, in this embodiment, the second detection unit 200 sets the brightness standard using the brightness of the pixels that constitute the first line detected by the first detection unit 100.

[0054] For example, if the second detection unit 200 is intended to detect a white line, the brightness standard described above is the lower limit. That is, the second detection unit 200 selects pixels with a brightness equal to or greater than the standard value. Conversely, the second detection unit 200 may select pixels with a brightness equal to or less than the standard value. In this case, the second detection unit 200 can indirectly detect the target line by selecting pixels that constitute an area other than the target line. The second detection unit 200 then sets the lower limit based on a value obtained by statistically processing the brightness of the pixels constituting the first line detected by the first detection unit 100. Examples of statistically processed values ​​include the mean and the mode. The second detection unit 200 may also set the lower limit as a value obtained by adding a constant to this statistically processed value.

[0055] Furthermore, the second detection unit 200 may detect white lines by excluding pixels constituting the first line identified by the first detection unit 100 from the processed image 50, and selecting pixels with a brightness higher than a reference value from the excluded image. In this case, the lower limit is, for example, a fixed value, and a value higher than the brightness value of a typical road surface is used.

[0056] Figure 14 shows an example of the functional configuration of the second detection unit 200. In the example shown in this figure, the second detection unit 200 includes a division unit 220, a distribution information generation unit 240, an estimation information generation unit 260, and a specific processing unit 280 (an example of a second specific processing unit, a third specific processing unit, or a white line detection processing unit). The processing performed by the division unit 220 is the same as the processing performed by the division unit 120. Furthermore, the processing performed by the distribution information generation unit 240, the estimation information generation unit 260, and the specific processing unit 280 is the same as the processing performed by the distribution information generation unit 140, the estimation information generation unit 160, and the specific processing unit 280, respectively, except that the pixel selection criterion is brightness. For this reason, in this embodiment, the first detection unit 100 can also perform the functions of the second detection unit 200. In addition, the second detection unit 200 may have a data conversion unit 110 before the division unit 220.

[0057] The distribution information generation unit 240 selects pixels whose brightness meets a predetermined standard and generates distribution information (second distribution information) for the selected pixels. The distribution information generation unit 240 sets the standard used in this process using the brightness of the pixels that constitute the first line detected by the first detection unit 100, as described above.

[0058] Alternatively, the distribution information generation unit 240 may exclude the pixels detected by the first detection unit 100 from the segmented image 52 and generate second distribution information using the segmented image 52 after the exclusion.

[0059] Figure 15 is a flowchart showing an example of the processing performed by the line detection device 10 according to this embodiment. The line detection device 10 performs the processing shown in this figure for each of the multiple frame images. First, the first detection unit 100 performs the detection processing for the first line (the line of the first color). Then, the second detection unit 200 receives the brightness information of the first line from the first detection unit 100 and then performs the detection processing for the second line (the line of the second color). Here, the brightness of the first line is lower than the brightness of the second line.

[0060] If the second detection unit 200 processes before the first detection unit 100 processes, that is, if the second line is detected before the first line, the second detection unit 200 may detect the first line along with the second line due to reasons such as the brightness standard used when detecting the second line being too low. In contrast, in this embodiment, the second detection unit 200 processes after the first detection unit 100 has processed. Therefore, the possibility of detecting the first line along with the second line is reduced. For example, as described above, when the second detection unit 200 processes, it can exclude the pixels that make up the first line (yellow line) detected by the first detection unit 100, making it possible for the second detection unit to detect only the white line.

[0061] The details of the detection process for the first line are as shown in one of the embodiments described above. The details of the detection process for the second line are also as described above.

[0062] In addition, there may be cases where the first detection unit 100 cannot detect the first line. In this case, the distribution information generation unit 240 of the second detection unit 200 performs the generation process of second distribution information using a predetermined brightness standard.

[0063] Furthermore, the second detection unit 200 may perform the second line detection process on the entire processed image 50, or it may perform the second line detection process on only a part of the processed image 50. In the latter case, the second detection unit 200 may determine the area in which the second line detection process is performed based on the position of the first line. Doing so reduces the amount of calculation processing performed by the second detection unit 200.

[0064] Figure 16 is a diagram illustrating an example of an area to be processed by the second detection unit 200. As shown in this figure, on a road (e.g., a road), there may be a second line L2 (e.g., a white line) drawn parallel to a first line L1 (e.g., a yellow line). In such a case, the second detection unit 200 determines the area to be processed using the first line L1 as a reference in the direction intersecting the first line L1, and detects the second line L2 located within that area. For example, in the horizontal direction (x-axis direction) of the processed image 50, the second detection unit 200 takes a first width W1 in the + direction and a second width W2 in the - direction, with the center of the first line L1 as the center. This area is then set as area 58. The first width W1 and the second width W2 may be equal or different. Also, the width of area 58 may be changed along the direction in which line L1 extends. For example, if the width of the region included in the processed image 50 narrows as you move towards the top of the processed image 50, then the width of region 58 may also narrow as you move towards the top of the processed image 50.

[0065] As described above, according to this embodiment, the first line can be detected with high accuracy, and the second line can also be detected with high accuracy.

[0066] (Sixth embodiment) Figure 17 shows the functional configuration of the line detection device 10 according to the sixth embodiment. The line detection device 10 according to this embodiment has the same configuration as the line detection device 10 according to the fifth embodiment, except that it includes a determination unit 300.

[0067] As described above, the first detection unit 100 detects a first line contained in the processed image 50, and the second detection unit 200 detects a second line contained in the processed image 50. The processed image 50 is each of the frame images that make up the video. The determination unit 300 uses the processing results of the first detection unit 100 for each of the frame images to calculate the detection period of the first line and determines whether the first line is a dotted line or not. The determination unit 300 performs the same processing for the second line.

[0068] The following describes the case where the determination unit 300 processes the first line. The determination unit 300 identifies the first line contained in the frame image (processed image 50) each time the first detection unit 100 processes the frame image. The determination unit 300 then determines the detection period of the first line by processing the transition of this identification result between frame images.

[0069] For example, consider the case where the estimation information generation unit 160 of the first detection unit 100 performs a process for each divided image 52 to determine whether or not the first line is included in the divided image 52, as described above. In this case, the determination unit 300 calculates the number of divided images 52 that are determined to contain the first line. Then, as shown in the figures of Figure 18, the determination unit 300 uses the change in the number of divided images 52 to determine whether or not the first line is a dotted line. Specifically, the determination unit 300 determines that the first line is a dotted line if the increase or decrease in the number of divided images 52 is repeated at a constant cycle. On the other hand, the determination unit 300 determines that the first line is a continuous line if the calculated number of divided images 52 is greater than or equal to a standard number (for example, 70% or more of the number of divided images 52 included in a single frame image) and this state continues for a certain number of frame images or more. Furthermore, the determination unit 300 determines that the first line is a continuous line but is partially missing (or faint) if the number of divided images 52 is repeatedly increasing or decreasing irregularly.

[0070] Figure 18(A) shows the result of plotting the number of divided images 52 for each frame image, and Figure 18(B) shows the result of plotting the trend of the average value of the number of divided images 52 over multiple consecutive frames (for example, 5 frames). In either case, the increase and decrease in the number of divided images 52 is repeated at a constant period, so it can be processed by the judgment unit 300. Figure 18(B) also shows that the average value of the divided images 52 falls within a certain range.

[0071] The line detection device 10 may also acquire information to determine the speed of the moving object 40 when the imaging device 20 generates the processed image 50. For example, the line detection device 10 acquires information indicating the time when the processed image 50 was generated for each processed image 50, and also acquires information indicating the speed of the moving object 40 for each time period. In this case, the determination unit 300 can further use this speed to determine whether or not the first line is a dotted line.

[0072] Specifically, the determination unit 300 calculates the length of the first line using the detection period of the first line and the speed of the moving object 40 described above. For example, consider the case where a portion 54 is set in the processed image 50, as shown in Figure 8. The determination unit 300 counts the number of frames from the frame image in which the number of divided images 52 determined to contain the first line is equal to or greater than a reference number, to the frame image in which the number of divided images 52 is equal to or less than a reference number, and calculates the length of the first line by multiplying this number by the frame rate and the speed of the moving object 40. The determination unit 300 also counts the number of frames from the frame image in which the number of divided images 52 determined to contain the first line is equal to or less than a reference number, to the frame image in which the number of divided images 52 is equal to or greater than a reference number, and calculates the interval of the first line by multiplying this number by the frame rate and the speed of the moving object 40. The determination unit 300 then determines that the first line is a dotted line if the calculated variation in the length of the first line and the variation in the spacing between the first lines are below a certain value. On the other hand, if the calculated variation in the length of the first line is above a certain value, it determines that the first line is a continuous line but is partially missing (or faded).

[0073] When the determination unit 300 calculates at least one of the length of the first line and the spacing of the first line, it may use the degree of agreement between this length and the reference length to determine whether the first line is a dotted line or not. In this case, as shown in Figure 17, the determination unit 300 may acquire type information to identify the type of road and determine the above-mentioned reference length using the type information. This type information is, for example, information to identify whether the road is a general road, a toll road, or an expressway. In the case of an expressway, the reference length is longer than in the case of a general road. The above-mentioned type information may be information that directly identifies the road, or it may be information indicating the position of the moving object 40 (for example, information indicating latitude and longitude such as GPS information) and map information.

[0074] The determination result from the determination unit 300 is output to the control device 30 of the mobile body 40. The control device 30 uses this determination result, that is, information indicating whether the first line (or second line) is a continuous line or a dotted line, to control the movement of the mobile body 40 (for example, whether or not to change the travel line). Specific examples of this determination are determined, for example, based on traffic rules.

[0075] As described above, according to this embodiment, the determination unit 300 can accurately determine whether the first line and the second line are dotted lines or not. The control device 30 of the mobile body 40 then uses this determination result to control the movement of the mobile body 40, thereby enabling a high level of automatic operation of the mobile body 40.

[0076] Furthermore, the line detection device 10 according to the first to fourth embodiments may also include the judgment unit 300 described above.

[0077] The embodiments and examples described above with reference to the drawings are illustrative examples of the present invention, and various other configurations can also be adopted.

Claims

1. A first processing unit extracts pixels whose color is located within a predetermined range from an image including the road on which a moving object is traveling, and detects lines of a first color included in the image using the distribution of the extracted pixels in the image. After the first processing unit has performed processing, the second processing unit extracts pixels from the image whose brightness is located within a predetermined brightness range, and uses the distribution of the extracted pixels in the image to identify lines of a second color that are included in the image and are different from the first color. Equipped with, The second processing unit is a line detection device that removes pixels constituting the first color lines detected by the first processing unit from the image, and then performs a process to identify the second color lines in the image after removal.

2. In the line detection device according to claim 1, The second processing unit is a line detection device that determines the predetermined brightness range using the brightness of the detected first color line.

3. In the line detection device according to claim 1, A line detection device in which the first color is yellow and the second color is white.

4. In the line detection device according to any one of claims 1 to 3, The second processing unit is a line detection device that identifies a line of the second color when the first processing unit fails to detect a line of the first color.

5. In the line detection device according to any one of claims 1 to 4, The aforementioned image is a line detection device captured by an imaging device mounted on the moving body.

6. In the line detection device according to claim 5, The image above is an image taken in front of the moving object, a line detection device.

7. Computers By processing an image that includes the path on which a moving object is traveling, the first colored line contained in the image is detected. After performing the detection process for the first color line, pixels located within a predetermined brightness range are extracted from the image, and lines of a second color different from the first color, which are included in the image, are identified using the distribution of the extracted pixels within the image. A line detection method comprising: removing pixels constituting the first color line detected by the computer from the aforementioned image, and then performing a process to identify the second color line in the image after the removal.

8. On the computer, A first process involves processing an image that includes a road on which a moving object is traveling, thereby detecting a line of a first color contained in the image. After performing the line detection process of the first color, a second process is performed to extract pixels from the image whose brightness is located within a predetermined brightness range, and to identify lines of a second color different from the first color that are included in the image using the distribution of the extracted pixels in the image. Make it run, A program that, in the second process, removes pixels that constitute the first color line detected in the first process from the image, and identifies the second color line in the image after removal.

9. A storage medium that stores a program that can be executed by a computer, The aforementioned program is installed on the computer. A first process involves processing an image that includes a road on which a moving object is traveling, thereby detecting a line of a first color contained in the image. After performing the line detection process of the first color, a second process is performed to extract pixels from the image whose brightness is located within a predetermined brightness range, and to identify lines of a second color different from the first color that are included in the image using the distribution of the extracted pixels in the image. Make it run, A storage medium that, in the second process, removes pixels constituting the first color lines detected in the first process from the image, and identifies the second color lines in the image after the removal.

Images