Target portion detection device and traveling truck

Abstract

A target portion detection device (41) detects a square target portion included in a captured image (IM) captured by an imaging device (8). The target portion detection device (41) comprises: a division unit (49) that divides a captured image (IM) into a plurality of sections (M); a plurality of computation units (41C) that are provided corresponding to each of the plurality of sections (M) and that can each independently execute image processing of the corresponding section (M); and a detection unit (45) that detects a target portion included in the captured image (IM) on the basis of the computation results from the plurality of computation units (41C).

Description

Target part detection device and traveling vehicle

[0001] One aspect of the present invention relates to an object portion detection device and a traveling vehicle that detect an object portion included in an image.

[0002] There is known a target portion detection device (rectangle detection device) that detects a rectangular shape from image data captured by an imaging device. For example, Patent Document 1 discloses a detection device that detects a rectangular shape from an image as a target portion contained in the image. The rectangle detection device of Patent Document 1 acquires image data from an image captured by an on-board camera, in which edges have been detected, and performs a partial matching process between multiple pieces of image data at different magnifications and a single square template to determine the degree of partial matching between the image data and each side of the template at the scanning position where the image data is scanned. The target portion detection device then integrates the partial matching degrees stored for each piece of image data at different magnifications to detect a rectangular shape or an image area where a rectangular shape is present from the image data.

[0003] Japanese Patent Application Laid-Open No. 2015-219820

[0004] In the conventional target portion detection device described above, a matching process is performed to compare images to detect the target portion, but such a process generally requires time. Furthermore, in devices such as transport vehicles that incorporate such target portion detection devices, there is a demand for a detection range that is as wide as possible. In order to widen the detection range, it is necessary to increase the resolution of the captured image captured by the imaging device. However, the higher the resolution of the captured image, the greater the number of target portions, and the longer the processing time required to detect the target portions.

[0005] Therefore, an object of one aspect of the present invention is to provide an object portion detection device and a traveling vehicle that can reduce the time required to detect a rectangular object portion included in an image.

[0006] (1) A target portion detection device according to one aspect of the present invention is a target portion detection device that detects a rectangular target portion included in an image captured by an imaging device, and includes a division unit that divides the captured image into a plurality of sections, a plurality of calculation units that are provided corresponding to each of the plurality of sections and are capable of independently performing image processing in the corresponding section, and a detection unit that detects the target portion included in the captured image based on the calculation results by the plurality of calculation units.

[0007] In this object portion detection device, a captured image is divided into multiple sections, and independent image processing is performed on each of the multiple sections. This reduces the load on the calculation unit compared to performing image processing on the entire image. As a result, the time required to detect a rectangular object portion contained in the image can be shortened.

[0008] (2) In the target portion detection device described in (1) above, the detection unit may detect a target portion included in the captured image by executing a plurality of processes, and the division unit may divide the captured image into a plurality of sections so that the division patterns are different for each of the plurality of processes. In this configuration, the sections are divided in a pattern that matches the process executed in the target portion detection device, so that the time required for the process can be efficiently reduced.

[0009] (3) In the target portion detection device described in (1) or (2) above, the detection unit detects the target portion included in the captured image through first-stage and second-stage processing, the division unit divides the captured image into different sections according to each of the first-stage and second-stage processing, and the multiple calculation units may include multiple first calculation units provided corresponding to each of the multiple sections corresponding to the first stage and capable of independently performing image processing on each section, and multiple second calculation units provided corresponding to each of the multiple sections corresponding to the second stage and capable of independently performing image processing on each section. With this configuration, the sections are divided in a manner that matches the stage of processing executed in the target portion detection device, thereby efficiently reducing the time required for the processing.

[0010] (4) In the target portion detection device described in any one of (1) to (3) above, the multiple calculation units may include a calculation unit that executes a captured image acquisition process to acquire a captured image from an imaging device, a calculation unit that executes a vertex candidate extraction process to extract vertex candidates that can become vertices of a quadrangular shape from the captured image, a calculation unit that executes a line segment candidate derivation process to derive line segment candidates that can become sides of the quadrangular shape from the vertex candidates, a calculation unit that executes a pattern candidate derivation process to derive quadrangular pattern candidates that can become the four sides of the quadrangular shape from the line segment candidates, and a calculation unit that executes a target portion detection process to determine, from the multiple quadrangular pattern candidates, a quadrangular pattern candidate that matches the quadrangular pattern that corresponds to the target portion of the quadrangular shape. In this configuration, the captured image acquisition process, the vertex candidate extraction process, the line segment candidate derivation process, the pattern candidate derivation process, and the target portion detection process can each be executed independently.

[0011] (5) The object portion detection device according to any one of (1) to (4) above may further include a plurality of storage units provided corresponding to the plurality of sections, each of which stores information relating to the image of the corresponding section. This configuration reduces the time required to read and write information relating to the image of the corresponding section.

[0012] (6) The object portion detection device described in any one of (1) to (5) above includes a storage device that stores a rectangular pattern corresponding to the object portion; an extraction unit that extracts multiple vertex candidates that can be vertices of the rectangular pattern included in the captured image; a detection unit that derives a combination of vertex candidates that can form the vertices of the rectangular pattern from the multiple vertex candidates extracted by the extraction unit and detects the rectangular object portion based on the derived combination of vertex candidates and the rectangular pattern; and a creation unit that reduces the captured image to create a reduced image. The detection unit may derive a combination of vertex candidates that can form the vertices of the rectangular pattern that fit within a specified pattern that defines lower and upper size limits from each of the original image and the reduced image. This configuration reduces the number of comparisons of vertex candidate combinations during the detection process compared to detecting all rectangular object portions included in the original image and the reduced image. As a result, the time required to detect the rectangular object portion can be further reduced.

[0013] (7) In the object portion detection device described in (6) above, the specified pattern may be configured with a first frame portion that defines a lower limit value and a second frame portion that is arranged to surround the first frame portion and defines an upper limit value, and the detection unit may derive combinations of vertex candidates that can form vertices of a quadrangular pattern that fits within the enclosed area of ​​the first frame portion and the second frame portion. This configuration can reduce the number of comparisons of combinations of vertex candidates that can form vertices of a quadrangular pattern that fits within the range of the specified pattern.

[0014] (8) In the object portion detection device according to any one of (1) to (7), the detection unit may detect a rectangular two-dimensional code. This configuration allows for more rapid detection of the two-dimensional code.

[0015] (9) A traveling carriage according to one aspect of the present invention may include a traveling unit that travels along a predetermined traveling path, the target portion detection device according to any one of (1) to (8) above, and a controller that controls the traveling of the traveling unit based on information obtained from the rectangular target portion detected by the target portion detection device. Since the traveling carriage can quickly detect the rectangular target portion, it can respond even if it is necessary to control the traveling unit within a limited time based on information acquired by the imaging device 8 while traveling.

[0016] According to one aspect of the present invention, it is possible to reduce the time required to detect a target portion included in an image.

[0017] FIG. 1 is a schematic diagram showing a configuration of a traveling vehicle system in which a traveling vehicle equipped with an object portion detection device according to an embodiment travels. FIG. 2 is a side view of the traveling vehicle of FIG. 1 as seen from the side. FIG. 3 is a rear view of the main body of the traveling vehicle of FIG. 1 as seen from behind in the traveling direction. FIG. 4 is a block diagram showing the functional configuration of the traveling vehicle. FIG. 5 is a diagram explaining an overview of processing in the object portion detection device of FIG. 1. FIG. 6(A) is a diagram showing a rectangular pattern of the traveling vehicle and markers of FIG. 1. FIG. 6(B) is a diagram showing a logo pattern composed of a plurality of rectangular patterns. FIG. 6(C) is a diagram showing a rectangular pattern composed of a plurality of rectangular patterns. FIG. 6(D) is a diagram showing a rectangular pattern of the traveling vehicle, black areas of the marker, and white areas of the marker. FIG. 6(E) is a diagram showing vertex candidates that may constitute the rectangular pattern of the traveling vehicle and black areas of the marker. FIG. 6(F) is a diagram showing vertex candidates that may constitute the rectangular pattern of the logo. FIG. 7(A) is a diagram showing a determination region including four partitioned regions separated by two mutually perpendicular virtual lines. FIG. 7(B) is a diagram showing an example of scanning a marker portion. FIG. 7(C) is an enlarged diagram of a determination region obtained when scanning a marker portion. FIG. 8(A) is a diagram showing a combination pattern of light and dark in four pixels consisting of two rows and two columns. FIG. 8(B) is a diagram showing an edge detected when an edge extraction filter is applied in the left-right direction. FIG. 8(C) is a diagram showing an edge detected when an edge extraction filter is applied in the up-down direction. FIG. 9(A) is a diagram explaining a prescribed pattern for scanning a captured image. FIG. 9(B) is a diagram showing whether or not detection is performed by the prescribed pattern in a captured image and a reduced image in the case of short, medium, and long distances. FIG. 10 is a block diagram showing the functional configuration of a target portion detection device including a creation unit. FIGS. 11(A) to 11(D) are example circuit diagrams for generating a reduced image. FIG. 12 is a diagram explaining a process for correcting the vertices of a rectangular pattern to the positions of the vertices of a rectangular pattern corresponding to a marker. Fig. 13 is a diagram showing the relationship between a plurality of regions and a plurality of storage processing units. Fig. 14(A) to Fig. 14(C) are conceptual diagrams of a captured image divided into a plurality of sections.FIGS. 15A and 15B are diagrams illustrating an example of ports connected to multiple arithmetic processes. FIG. 16A is a diagram illustrating a quadrangular target portion detected by a target portion detection device. FIG. 16B is a flowchart illustrating the processing flow when the target portion detection device detects a quadrangular target portion. FIG. 17 is a diagram illustrating an example of the processing outline of a target portion detection device according to an embodiment. FIG. 18 is a diagram illustrating another example of the processing outline of a target portion detection device according to an embodiment. FIG. 19 is a diagram illustrating yet another example of the processing outline of a target portion detection device according to an embodiment. FIGS. 20A to 20C are diagrams illustrating the processing flow of a target portion detection device according to an embodiment. FIG. 21 is an example of the configuration of a target portion detection device. FIG. 22A is a diagram illustrating the direction of a vertex candidate. FIG. 22B is a diagram illustrating an example of the processing outline of a target portion detection device according to Modification 1. FIG. 23 is a diagram illustrating an example of the processing of a target portion detection device according to Modification 1.

[0018] Hereinafter, an overhead traveling vehicle (traveling carriage) 6 (hereinafter also referred to as "traveling vehicle 6") equipped with a target portion detection device 41 according to one embodiment will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted.

[0019] 1 to 3, a mobile vehicle system 1 will be described as an example in which multiple mobile vehicles 6 travel along a one-way track 4 installed on the ceiling or the like of a factory. The track 4 is suspended from, for example, the ceiling. The mobile vehicles 6 transport items 10, such as containers such as FOUPs (Front Opening Unified Pods) that store multiple semiconductor wafers and reticle pods that store glass substrates, as well as general parts.

[0020] The traveling vehicle 6 travels along the track 4 and transports the article 10. The traveling vehicle 6 is configured to be able to transfer the article 10. The traveling vehicle 6 is an overhead traveling automatic guided vehicle. The traveling vehicle 6 has a traveling unit 20, a main body unit 7, an imaging device 8, a marker 60, and a controller 40. The traveling unit 20 is configured to include a motor and the like, and causes the traveling vehicle 6 to travel along the track 4. The main body unit 7 has a main body frame, a lateral feed unit, a θ drive, an elevation drive unit, an elevation unit, a front cover 34, and a rear cover 35.

[0021] The imaging device 8 is provided on the front cover 34 of the main body 7 so that its imaging range is in front of the host traveling vehicle 6. The imaging device 8 is a device that includes a lens and an imaging element that converts light entering through the lens into an electrical signal. Examples of the imaging device 8 include a visible light camera and a near-infrared camera. The imaging device 8 captures an image of the area in front of the host traveling vehicle 6. For example, the imaging device 8 captures an image of the leading traveling vehicle 6A, which is a traveling vehicle 6 located in front of the host traveling vehicle 6, so that a marker 60 provided on the leading traveling vehicle 6A is included in the captured image IM. The captured image IM (see FIG. 5 ) captured by the imaging device 8 is acquired by a controller 40, which will be described in detail later.

[0022] The imaging device 8 may be integrated with or configured separately from a target portion detection device 41 (see FIG. 4 ) as an image processing device including a storage processing unit and an arithmetic processing unit. The storage processing unit may be, for example, a static random access memory (SRAM), a dynamic random access memory (DRAM), or a storage unit configured with a combination of SRAM and DRAM, or may be configured with multiple storage units to enable parallel processing. The arithmetic processing unit may be, for example, a central processing unit (CPU), a field programmable gate array (FPGA), or an arithmetic unit configured with a combination of a CPU and an FPGA, or may be configured with multiple arithmetic units to enable parallel processing. In this embodiment, the target portion detection device 41 is configured as part of the controller 40, i.e., separately from the imaging device 8, as described in detail below.

[0023] The marker 60 is provided on the rear cover 35 so as to be visible from a rear traveling vehicle 6B, which is a traveling vehicle 6 located behind the own traveling vehicle 6. The marker 60 is a two-dimensional code, for example, an AR marker or a QR code (registered trademark). In this embodiment, an example will be described in which the two-dimensional code is an AR marker. The marker 60 is formed to include a black area consisting of a square and a white area arranged in part of the black area. The white area formed inside the black area forms a patterned graphic.

[0024] To mount the markers 60 on the rear cover 35, for example, a panel on which the markers 60 are printed may be attached to the rear cover 35, or the markers 60 may be printed directly on the rear cover 35 or laser marked. The markers 60 may also be realized by at least one of welding or fitting a material having different reflectance in a specific wavelength range, and processing that results in different reflectance in a specific wavelength range, so that the imaging device 8 can recognize the difference between light and dark.

[0025] The controller 40 shown in FIG. 4 is an electronic control unit including a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The controller 40 controls various operations of the traveling vehicle 6. Specifically, the controller 40 controls the traveling unit 20, the traverse unit, the θ drive, the lift drive unit, the lift unit, and the imaging device 8. The controller 40 also communicates with the traveling vehicle controller 12 using a communication line (feeder line) of the track 4 or the like. The controller 40 can be configured as software in which a program stored in a ROM is loaded onto a RAM and executed by a CPU. The controller 40 may also be configured as hardware including an electronic circuit or the like. A part of the controller 40 of this embodiment constitutes a target portion detection device 41. The target portion detection device 41 mainly includes an input processing unit 41A, a storage processing unit 41B, an arithmetic processing unit (arithmetic unit) 41C, an output processing unit 41D, and a storage device (storage unit) 41E.

[0026] The input processing unit 41A is a part that interfaces with external devices. For example, the input processing unit 41A receives the captured image IM from the imaging device 8. The storage processing unit 41B is composed of, for example, a DRAM, an SRAM, etc. The storage processing unit 41B stores the captured image IM processed by the target portion detection device 41, information necessary for the detection process, etc. The arithmetic processing unit 41C is composed of, for example, a CPU, an FPGA, etc. The arithmetic processing unit 41C reads the captured image IM, etc. from the storage processing unit 41B and performs target portion detection process. The output processing unit 41D is a part that interfaces with external devices. The output processing unit 41D outputs the target portion detection result by the arithmetic processing unit 41C to the outside. The storage processing unit 41B and the arithmetic processing unit 41C will be described in detail later.

[0027] The storage device 41E is configured with a hard disk drive (HDD), a solid state drive (SSD), etc. The storage device 41E stores, for example, a rectangular pattern corresponding to a rectangular target portion, which will be described in detail later. The storage device 41E may also temporarily store the captured image IM input from the imaging device 8.

[0028] The target portion detection device 41 detects a rectangular target portion from the captured image IM. Examples of rectangular target portions include, for example, a moving object, a fixed object, and portions (areas) contained therein. Examples of rectangular target portions contained in the captured image IM obtained from the imaging device 8 provided on the traveling vehicle 6 include, for example, another traveling vehicle 6 (rear cover 35), an item 10 transported by the traveling vehicle 6, the track 4 on which the traveling vehicle 6 travels, attachments to the track 4, a buffer (not shown) for holding the item 10, a marker 60 (see FIG. 3 ) provided on the traveling vehicle 6 or the track 4, a logo 160 (see FIG. 6B ), a figure, a symbol, etc. The controller 40 can execute various controls based on the rectangular target portion acquired by the target portion detection device 41. Note that the various controls referred to here will be described in detail later.

[0029] 5, the target portion detection device 41 (1) inputs a captured image IM from the imaging device 8, (2) extracts vertex candidates that can become the vertices of a quadrangle from the input captured image IM, (3) detects line segment candidates that can become the sides of the quadrangle from the extracted vertex candidates, (4) detects quadrangle pattern candidates that can become the four sides of the quadrangle from the detected line segment candidates, and (5) determines a quadrangle pattern candidate (solid line) that matches the quadrangle pattern corresponding to the quadrangle target portion from the detected multiple quadrangle pattern candidates (dashed lines), thereby detecting the quadrangle target portion. In the target portion detection device 41, hardware such as a CPU, RAM, and ROM cooperate with software such as a program to form an extraction unit 43, a detection unit 45, a creation unit 47, and a division unit 49 that perform the above processes (1) to (5).

[0030] The storage device 41E stores a rectangular pattern corresponding to a rectangular target portion. As shown in FIG. 6A , the storage device 41E stores, for example, a rectangular pattern PA6 corresponding to the traveling vehicle 6 (rear cover 35) imaged from a predetermined position (each relative distance). If the rectangular target portion is a fixed object, the storage device 41E may store a rectangular pattern corresponding to the rectangular target portion for each image capture position. The storage device 41E also stores, for example, a rectangular pattern PA60 corresponding to a marker 60 provided on the rear cover 35 of the traveling vehicle 6 imaged from a predetermined position. The storage device 41E may store, for example, the rectangular pattern PA6 and the rectangular pattern PA60 individually or integrally.

[0031] For example, if a logo 160 such as that shown in FIG. 6B is depicted on the rear cover 35 of the traveling vehicle 6 captured from a predetermined position, the storage device 41E may store a rectangular pattern PA160 corresponding to the logo. The rectangular pattern PA160 corresponding to the logo may be stored as a combination of multiple rectangular patterns RS1 to RS3. The marker 60 (see FIG. 3) provided on the rear cover 35 is formed by a black area 61 that is the outline of the marker 60 and a white area 63 that indicates the pattern of the marker 60. As shown in FIG. 6D, the storage device 41E may store the rectangular pattern corresponding to the marker 60 as a rectangular pattern PA61 corresponding to the black area 61. The storage device 41E may also store the rectangular pattern corresponding to the marker 60 as a rectangular pattern PA63 corresponding to the white area 63. Furthermore, the storage device 41E may store the rectangular pattern PA61 and the rectangular pattern PA63 integrally.

[0032] The extraction unit 43 extracts all of a plurality of vertex candidates that can be vertices of a quadrilateral shape from the captured image IM. The vertex candidates are managed as information (data), such as position information (e.g., coordinate information) and an ID (identifier). The extraction unit 43 is mainly formed by the calculation processing unit 41C. For example, the vertices of the quadrilateral shape are the four vertices P6 in the rear cover 35 of the traveling vehicle 6 as shown in FIG. 6E, and the four vertices P60 in the black region 61 of the marker 60 affixed to the rear cover 35. Furthermore, for the logo 160, the vertices are the vertices P160 of each of the multiple quadrilateral patterns RS1 to RS3 as shown in FIG. 6F. Note that the vertices of the white region 63 of the marker 60 may also be extracted using a method similar to that for the logo 160.

[0033] The extraction unit 43 may acquire the captured image IM for extracting the vertices P6 and P60 directly from the imaging device 8, or may read it from the storage processing unit 41B or the storage device 41E in which the captured image IM captured by the imaging device 8 is stored. Furthermore, the extraction unit 43 may limit the range in which vertex candidates are detected from the entire captured image IM before extracting the vertex candidates. Furthermore, the extraction unit 43 may extract vertex candidates based on a cropped image in which a range in which vertex candidates are detected is cropped from the captured image IM captured by the imaging device 8. Furthermore, the extraction unit 43 may perform the vertex candidate detection process on an image obtained by performing image correction processing, reduction processing, etc. on the captured image IM captured by the imaging device 8. Furthermore, the extraction unit 43 may store intermediate images for performing image correction processing, reduction processing, etc. on the captured image IM captured by the imaging device 8 in the storage processing unit 41B.

[0034] The extraction unit 43 can extract vertex candidates from, for example, the brightness difference in a determination area A1, which is a small range in the captured image IM. For example, as shown in FIG. 7A , the extraction unit 43 sets four segmented areas R1, R2, R3, and R4 for the determination area A1 on the captured image IM, the segmented areas being divided by two mutually orthogonal imaginary straight lines SL1 and SL2 that pass through the center position CP of the determination area A1. In other words, the extraction unit 43 sets the four segmented areas R1, R2, R3, and R4 so as to include one of the four corners formed by the two mutually orthogonal imaginary straight lines SL1 and SL2. The extraction unit 43 determines whether the determination area A1 is a vertex candidate based on the brightness and darkness patterns in the four segmented areas R1, R2, R3, and R4.

[0035] For example, if the difference between the average brightness value of one of the four segmented regions R1, R2, R3, and R4 (e.g., segmented region R4) and the average brightness values ​​of the remaining three segmented regions (e.g., segmented regions R1, R2, and R3) is equal to or greater than a predetermined threshold, the extraction unit 43 determines that the judgment region A1 is a vertex candidate. Note that, when determining whether the judgment region A1 is a vertex candidate, the extraction unit 43 may take into account deviations and the like in addition to the average brightness value. Alternatively, the extraction unit 43 may perform a binarization process on the captured image IM before determining whether the judgment region A1 is a vertex candidate.

[0036] The extraction unit 43 scans the captured image IM in units of determination regions A1 to search for vertex candidates. For example, the extraction unit 43 extracts vertex candidates from an image of the determination region A1 obtained by scanning the marker 60 portion shown in FIG. 7B, which is a part of the captured image IM captured by the imaging device 8. As shown in FIG. 7C, the extraction unit 43 determines that the determination region A1 is not a vertex candidate based on the fact that the difference between the average luminance value of one segment in the determination region A1 obtained at scanning position SP1 and the average luminance values ​​of the remaining three segment areas is not equal to or greater than a predetermined threshold. Similarly, the extraction unit 43 determines that the determination region A1 obtained at scanning position SP3 is not a vertex candidate. The extraction unit 43 determines that the determination region A1 is not a vertex candidate based on the fact that the difference between the average luminance value of one segment in the determination region A1 obtained at scanning position SP2 and the average luminance values ​​of the remaining three segment areas is equal to or greater than a predetermined threshold. Similarly, the extraction unit 43 determines that the determination area A1 obtained at the scanning position SP4 is a vertex candidate.

[0037] As described above, when using a method for determining whether the determination region A1 is a vertex candidate based on the light and dark patterns in the four segmented regions R1, R2, R3, and R4, it is also possible to determine whether the determination region A1 is a vertex candidate based on the light and dark combination pattern of four pixels PX arranged in two rows and two columns, as shown in FIG. 8A. This is because, when the center of the four pixels PX is taken as the central position, the four pixels PX adjacent to the central position can be considered to be four segmented regions R1, R2, R3, and R4. As a result, when the difference between the average luminance value of one pixel PX and the average luminance values ​​of the remaining three pixels PX is equal to or greater than a predetermined threshold, the extraction unit 43 determines that the determination region A1 formed by the four pixels PX is a vertex candidate.

[0038] The extraction unit 43 may determine whether the determination area A1 is a vertex candidate by a method other than the method of determining whether the determination area A1 is a vertex candidate based on the light and dark patterns in the four divided areas R1, R2, R3, and R4 as described above. For example, the extraction unit 43 may apply an edge extraction filter to the captured image IM to detect edges, and determine the detected edges as vertex candidates.

[0039] Specifically, as shown in Fig. 8(B), the extraction unit 43 applies an edge extraction filter in the left-right direction to detect an edge E1 (dashed circle). Also, as shown in Fig. 8(C), the extraction unit 43 applies an edge extraction filter in the up-down direction to detect an edge E2 (dashed circle). The extraction unit 43 extracts, as vertex candidates, the edges detected by both the edge E1 detected when the edge extraction filter is applied in the left-right direction and the edge E2 detected when the edge extraction filter is applied in the up-down direction.

[0040] The extraction unit 43 may directly input the extracted vertex candidates to the detection unit 45 or may temporarily store them in the storage processing unit 41B. The following describes an example in which the extraction unit 43 temporarily stores the extracted vertex candidates in the storage processing unit 41B. The extraction unit 43 may divide the storage area of ​​the storage processing unit 41B for storing the extracted vertex candidates based on the light-dark pattern in the image of the determination area A1, i.e., the direction of the vertex of the rectangular pattern. The extraction unit 43 may divide the storage area of ​​the storage processing unit 41B for storing the extracted vertex candidates based on the coordinate range of the extracted vertex candidates in the captured image IM. The extraction unit 43 may also calculate and store information other than coordinates that can be used for detecting pattern rectangle candidates in a subsequent stage. The extraction unit 43 may also divide the storage area of ​​the storage processing unit 41B for storing this information, as described above.

[0041] The detection unit 45 derives a combination of vertex candidates that can form a quadrangular pattern based on the multiple vertex candidates extracted by the extraction unit 43. The detection unit 45 detects a quadrangular target portion based on the derived combination of vertex candidates. The detection unit 45 can limit the quadrangular target portion that can be detected based on the tilt, aspect ratio, or size that the quadrangular pattern can assume based on the actual positional relationship between the detection target and the imaging device 8.

[0042] The detection unit 45 also derives, from each of the original image IM and the reduced image RIM, a combination of vertex candidates that can form the vertices of a quadrangular pattern that fits within a specified pattern 70 that defines a lower limit and an upper limit for the size. As shown in Fig. 9A , the specified pattern 70 is composed of a first frame portion 71 that defines a lower limit and a second frame portion 72 that surrounds the first frame portion 71 and defines an upper limit. The detection unit 45 derives a combination of vertex candidates that can form the vertices of a quadrangular pattern that fits within a surrounding area 70A defined by the first frame portion 71 and the second frame portion 72.

[0043] The creation unit 47 creates a reduced image RIM by reducing the original image IM, which is the captured image. A method for creating a reduced image by the creation unit 47 will now be described. As shown in FIGS. 4 and 10 , the creation unit 47 is mainly formed by the calculation processing unit 41C. The original image IM acquired by the creation unit 47 may be input directly from the imaging device 8 or may be input via the storage device 41E. The scale ratio used by the creation unit 47 to create the reduced image RIM can be set arbitrarily. For example, the creation unit 47 generates a reduced image RIM with a scale ratio of 1 / 2 and a reduced image RIM with a scale ratio of 1 / 4. The scale ratio and the combination of scale ratios to be used are set appropriately by the creation unit 47.

[0044] 11A , for example, one arithmetic processing unit 41C may sequentially create reduced images RIM at various scales based on the original image IM input from the imaging device 8, the storage processing unit 41B, or the storage device 41E. The creation unit 47 creates multiple reduced images RIM through this process. The arithmetic processing unit 41C outputs the created reduced images RIM at the multiple scales to the extraction unit 43 and the detection unit 45.

[0045] 11(B), for example, one arithmetic processing unit 41C may first create a reduced image RIM1 at a relatively large scale from the input original image IM, and then use the same arithmetic processing unit 41C to create a reduced image RIM2 at a smaller scale from the reduced image RIM1. The creation unit 47 may repeat this process multiple times to create multiple reduced images RIM1 and RIM2. The arithmetic processing unit 41C outputs the created reduced images RIM at multiple scales to the extraction unit 43 and the detection unit 45.

[0046] 11(C), for example, the creation unit 47 may have a plurality of arithmetic processing units 41C arranged in parallel, each of which may create reduced images RIMa, RIMb, and RIMc at each scale (scale a, scale b, and scale c) from the original image IM. That is, the creation unit 47 may have a plurality of arithmetic processing units 41C simultaneously create images RIMa, RIMb, and RIMc at each scale. The arithmetic processing units 41C output the created reduced images RIMa, RIMb, and RIMc at each scale to the extraction unit 43 and the detection unit 45.

[0047] 11(D), the creation unit 47 may have a first-stage arithmetic processing unit 41C arranged in series to create a reduced image RIMa with a relatively large scale a, a next-stage arithmetic processing unit 41C to create a reduced image RIMb with a scale b smaller than that of the reduced image RIMa, and a next-stage arithmetic processing unit 41C to create a reduced image RIMc with a scale c smaller than that of the reduced image RIMb. The creation unit 47 repeats this process multiple times to create multiple reduced images RIMa, RIMb, and RIMc.

[0048] The creation unit 47 may also be configured by selectively combining the circuit diagrams of Figures 11(A) to 11(D) described above. Each of the reduced images RIM created during the circuit diagrams of Figures 11(A) to 11(D) may be stored in the storage device 41E or the like, and then an even smaller scale image may be generated again by the arithmetic processing unit 41C, or may be used to detect rectangular pattern candidates. In addition to image reduction processing, the creation unit 47 may also perform image correction processing such as a sharpening filter or morphological transformation, or acquire information that can be used to detect rectangular patterns.

[0049] The following description will be given taking as an example a case where the target portion detection device 41 detects a marker 60 as a rectangular target portion. When the distance to the preceding traveling vehicle 6A is short, a medium distance farther than short, or a long distance farther than medium, the captured image IM including the marker 60 captured by the imaging device 8 provided on the traveling vehicle 6 differs, as shown in FIG. 9(B). In the case of the captured image IM shown in FIG. 9(B), the detection unit 45 can obtain the rectangular shape corresponding to the marker 60 only from the long-distance image. That is, when the specified pattern 70 shown in FIG. 9(A) is scanned over the captured image IM, all of the vertices of the rectangular shape corresponding to the marker 60 in the long-distance image fall within the enclosing area 70A. At most, one vertex of the quadrangle corresponding to the marker 60 in the close-up image fits within the surrounding area 70A, and at most, one vertex of the quadrangle corresponding to the marker 60 in the medium-distance image fits within the surrounding area 70A.

[0050] Furthermore, in the case of half-scale reduced images RIM corresponding to the close-distance image, the middle-distance image, and the long-distance image created by the creation unit 47, the detection unit 45 can acquire the rectangular shape corresponding to the marker 60 only from the middle-distance image. That is, when the defined pattern 70 shown in Fig. 9A is scanned over the captured image IM, all of the vertices of the rectangular shape corresponding to the marker 60 in the middle-distance image fit within the surrounding area 70A. At most, one vertex of the rectangular shape corresponding to the marker 60 in the close-distance image fits within the surrounding area 70A, and at most, three vertices of the rectangular shape corresponding to the marker 60 in the long-distance image fit within the surrounding area 70A.

[0051] Furthermore, in the case of quarter-scale reduced images RIM corresponding to the close-distance image, the middle-distance image, and the long-distance image created by the creation unit 47, the detection unit 45 can acquire the quadrangle corresponding to the marker 60 only from the long-distance image. That is, when the defined pattern 70 shown in Fig. 9A is scanned over the captured image IM, all of the vertices of the quadrangle corresponding to the marker 60 in the long-distance image fall within the surrounding area 70A. Of the vertices of the quadrangle corresponding to the marker 60 in the middle-distance image and the long-distance image, at most three vertices fall within the surrounding area 70A.

[0052] Furthermore, if the position (range) at which a rectangular pattern corresponding to a rectangular detection target can be captured on a scale drawing at each scale can be limited based on the actual positional relationship of the imaging device 8, the detection range of the rectangular detection target may be limited to the above range for each scale drawing. However, because the variation in the orientation of the rectangular shape corresponding to the rectangular detection target varies depending on the distance (corresponding to the detectable scale) between the imaging device 8 and the rectangular detection target, it is difficult to switch (adjust) the criteria for evaluating the degree of tilt or distortion of the rectangular shape within a single image. For example, when the imaging device 8 and the rectangular detection target are on the same linear trajectory, the tilt of the rectangular detection target in the image relative to the tilt in the depth direction varies depending on the distance between the two. Specifically, there is almost no tilt when the distance is far, and the tilt is greater when the distance is close. Therefore, the larger the range of sizes of the rectangular detection target that can be captured, the more parameters are required for range limitation. Therefore, by limiting the range of size of the rectangular detection object for each scale drawing, it is possible to limit the orientation of the rectangular detection object that may appear, which is expected to make it easier to switch (adjust) the above judgment criteria.

[0053] 12 , when an attempt is made to associate the position on the image of a rectangular pattern detected from a reduced image RIM with a small scale (e.g., 1 / 4) at the scale of the original image IM (e.g., 1 / 1), the rectangular pattern PA60 corresponding to the marker 60 and the rectangular pattern candidate PAC60 may be misaligned due to an error (rounded-down pixels) generated by the reduction process. Therefore, the creation unit 47 may perform an additional search process to correct the vertex P0 of the rectangular pattern candidate PAC60 to the position of the vertex P1 of the rectangular pattern PA60 corresponding to the marker 60.

[0054] The dividing unit 49 divides the captured image IM into a plurality of sections M as shown in FIG. 13 . The dividing unit 49 divides the captured image IM into a mesh pattern. Examples of division patterns in which the dividing unit 49 divides the captured image IM include a pattern in which the captured image IM is divided equally, as shown in FIG. 14A . The size and shape of the sections M thus formed can be determined based on, for example, the size or shape of the specified pattern 70 shown in FIG. 9A . More specifically, the size and shape can be set to be approximately the same as or slightly larger than the maximum size of a rectangle detected at the scale, such as the specified pattern 70. By using such a shape and size, for example, when finding a line segment candidate, the mesh containing a vertex candidate to be compared can be limited to the mesh located at the same position as the mesh containing the reference vertex candidate or the mesh immediately adjacent to it.

[0055] 14(B) and 14(C), examples of division modes include a mode in which the shape and size (size) of the section M change depending on the position of the captured image IM. For example, the size (size) of the section M that constitutes an area in which many rectangular pattern candidates corresponding to the rectangular detection target are detected may be reduced and many sections M may be arranged depending on the actual positional relationship between the rectangular detection target and the imaging device 8. The division unit 49 may divide the captured image IM into multiple sections M so that different division modes are used for each type of processing (for example, for each of the first and second stage processing).

[0056] A plurality of storage processing units 41B are provided corresponding to the plurality of partitions M, respectively. Each of the plurality of storage processing units 41B stores information about the image of the corresponding partition M. The plurality of storage processing units 41B may be composed of a plurality of physically isolated devices or a plurality of logically isolated areas. A method of allocating a plurality of storage processing units 41B to the plurality of partitions M may involve providing an independently accessible port for each partition M, or, as shown in FIGS. 15A and 15B, sharing a single accessible port 51 for the plurality of partitions M. Providing many accessible ports 51 can complicate the wiring and increase costs. However, sharing a single accessible port 51 for the plurality of partitions M simplifies the wiring and reduces costs. Furthermore, when sharing a single accessible port 51 for the plurality of partitions M, it is preferable to share an accessible port for partitions M that are unlikely to be referenced simultaneously and independently, for example.

[0057] The target portion detection device 41 can find a quadrangular target portion in stages. In this embodiment, the device is configured to use a plurality of storage processing units 41B for each of the processes in the following four stages (i.e., vertex candidate extraction process, line segment candidate derivation process, connecting line segment candidate derivation process, and pattern candidate derivation process).

[0058] As shown in FIG. 16B , the extraction unit 43 extracts vertex candidates that can form the vertices of a quadrangular pattern (step S1: vertex candidate extraction processing), derives from the multiple vertex candidates a multiple of line segment candidates each consisting of a combination of vertex candidates that can form one side of the quadrangular pattern (step S2: line segment candidate derivation processing), derives from the multiple derived line segment candidates a multiple of connecting line segment candidates each consisting of a combination of line segment candidates that can form two adjacent sides of the quadrangular pattern (step S3: connecting line segment candidate derivation processing), derives from the multiple derived connecting line segment candidates a combination of connecting line segment candidates that can form four sides of the quadrangular pattern (step S4: pattern candidate derivation processing), and detects a target portion of the quadrangular shape based on the quadrangular pattern candidate formed by the derived combination of connecting line segment candidates and the quadrangular pattern stored in the storage device 41E (step S5).

[0059] An example will be described in which the rectangular target portion obtained by the processing of steps S1 to S5 described above is the marker 60 shown in FIG. 16A. The detection unit 45 reads the vertex candidates VC1, VC2, VC3, and VC4 extracted by the extraction unit 43 (step S1) and stored in the storage processing unit 41B. Next, the detection unit 45 derives multiple line segment candidates SC12, SC13, SC24, and SC34 that can form one side of the rectangular pattern based on the vertex candidates VC1, VC2, VC3, and VC4 read from the storage processing unit 41B (step S2). The detection unit 45 stores the detected line segment candidates SC12, SC13, SC24, and SC34 in the storage processing unit 41B.

[0060] Next, the detection unit 45 reads the line segment candidates SC12, SC13, SC24, and SC34 stored in the storage processing unit 41B and derives a plurality of connecting line segment candidates consisting of combinations of line segment candidates SC12, SC13, SC24, and SC34 that can form two adjacent sides of the quadrangular pattern (step S3). The detection unit 45 detects connecting line segment candidates SC12 & SC13 as a combination of line segment candidates that share the vertex candidate VC1 in common (i.e., connecting line segment candidates). The detection unit 45 also derives connecting line segment candidates SC24 & SC34 as a combination of line segment candidates that share the vertex candidate VC4 in common (i.e., connecting line segment candidates). The detection unit 45 stores the connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34 in the storage processing unit 41B.

[0061] The detection unit 45 reads out the connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34 stored in the storage processing unit 41B, and derives, from the derived plurality of connecting line segment candidates, combinations of connecting line segment candidates that can form the four sides of a quadrangular pattern (i.e., quadrangular pattern candidates) (step S4). The detection unit 45 derives the connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34 as combinations of connecting line segment candidates that share both the vertex candidate VC2 and the vertex candidate VC4. Based on these two connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34, the detection unit 45 derives a quadrangular pattern candidate PAC60 corresponding to the marker 60, as shown in FIG. 16(A).

[0062] The quadrangular pattern candidates derived by the detection unit 45 may be temporarily stored in the storage device 41E. The detection unit 45 may store the quadrangular pattern candidates by storing coordinate information of the vertices that make up the quadrangular pattern candidates. Furthermore, the detection unit 45 may change the storage area of ​​the storage device 41E for storing each vertex candidate, for example, depending on the coordinate range of the vertex, or may store the absolute coordinates of one vertex candidate that makes up the quadrangular pattern candidate and the relative coordinates of the remaining vertex candidates based on the absolute coordinates of the one vertex candidate. Even in this case, the detection unit 45 may calculate the coordinate information of each vertex that makes up the quadrangular pattern candidate by, for example, reading the information stored from the storage device 41E and then performing a specific process.

[0063] The detection unit 45 detects a rectangular target portion based on the rectangular pattern candidates and the rectangular patterns stored in the storage device 41E. Specifically, the detection unit 45 evaluates and selects each of the rectangular pattern candidates stored in the storage device 41E. The detection unit 45 evaluates and selects a rectangular pattern candidate based on, for example, the degree of agreement between coordinate information of vertices constituting the rectangular pattern candidate and coordinate information of vertices constituting the rectangular pattern. The detection unit 45 may also evaluate and select a rectangular pattern candidate based on, for example, the degree of agreement between a rectangular pattern corresponding to another target portion located inside the rectangular pattern candidate and a rectangular pattern corresponding to another target portion located inside the rectangular pattern.

[0064] For example, the detection unit 45 may select a rectangular pattern candidate with the highest degree of match, or may select a rectangular pattern candidate with a degree of match equal to or higher than a predetermined level. The detection unit 45 may evaluate the degree of tilt or distortion of the rectangular shape indicated by the rectangular pattern candidate. The detection unit 45 may also preferentially select a rectangular pattern candidate with less tilt or distortion. This method can be effectively used in cases where the rectangular pattern has a complex shape, such as one formed by combining multiple rectangular patterns.

[0065] The detection unit 45 may also detect a rectangular target portion by performing a matching process (matching) between a rectangular pattern candidate and image information stored in the storage device 41E. In this detection method, the range of the image to be compared is limited, compared to a conventional matching process in which a contour is extracted from differences in brightness on the captured image IM and image information about the extracted contour and the interior of the contour is compared with pre-stored image information. Therefore, the time required to detect a rectangular target portion included in the captured image IM can be reduced.

[0066] In this embodiment, a different calculation processing unit 41C is used for each of the processes in the above four stages (i.e., vertex candidate extraction process, line segment candidate derivation process, connecting line segment candidate derivation process, and pattern candidate derivation process).In other words, the target portion detection device 41 of this embodiment is provided with dedicated calculation processing units 41C for processing each of the vertex candidate extraction process, line segment candidate derivation process, connecting line segment candidate derivation process, and pattern candidate derivation process.

[0067] Up to this point, an example has been described in which all of the multiple vertex candidates that can become the vertices of a quadrilateral shape are extracted from the captured image IM, as shown in Fig. 16(B) , but the number of vertex candidates to be extracted from the captured image IM may be narrowed down below. Below, a method for narrowing down the number of vertex candidates to be extracted from the captured image IM by the above-mentioned creation unit 47 will be described.

[0068] The following describes a processing flow when the target portion detection device 41 detects a rectangular target portion (marker 60) when a captured image IM1 as shown in Fig. 17, a captured image IM2 as shown in Fig. 18, or a captured image IM3 as shown in Fig. 19 is input from the imaging device 8. The sizes of the markers 60 are different in the captured images IM1, IM2, and IM3.

[0069] First, the flow of processing up to detecting a rectangular target portion from a captured image IM1 shown in Fig. 17 will be described. The processes in (1), (2), (4), and (5) shown in Fig. 17 are the same as the processes in (1), (2), (4), and (5) described in Fig. 5 of the above embodiment. That is, in (1) shown in Fig. 17, a captured image (original image) IM is input from the imaging device 8. In (2) shown in Fig. 17, vertex candidates that can become the vertices of a rectangular shape are extracted from the input captured image IM. The vertex candidates are stored as a coordinate list, not as image data.

[0070] In (4) shown in FIG. 17 , a combination of vertex candidates that can become the vertices of a quadrangle when connected, i.e., a quadrangular pattern candidate that can become the four sides of a quadrangle, is derived. More specifically, a plurality of quadrangular pattern candidates that fit within the enclosing area 70A of the defined pattern 70 shown in FIG. 16 (A) are derived. At this time, the plurality of quadrangular pattern candidates are selected and evaluated based on factors such as tilt and aspect ratio. In (5) shown in FIG. 17 , a selection is made from the plurality of quadrangular pattern candidates, and from the remaining plurality of quadrangular pattern candidates, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined. This allows the quadrangular target portion corresponding to the marker 60 to be detected. Note that (5) shown in FIG. 17 shows a state in which the captured image IM1 is overlaid.

[0071] In (1A) shown in FIG. 17, a reduced image RIMa1 is created at a scale of 1 / 2, and in (1B) shown in FIG. 17, a reduced image RIMb1 is created at a scale of 1 / 4. In (2A) shown in FIG. 17, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM1. In (4A) shown in FIG. 17, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4). In (5A) shown in FIG. 17, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from multiple quadrangular pattern candidates. In (5A) shown in FIG. 17, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0072] In (2B) shown in FIG. 17, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM1. In (4B) shown in FIG. 17, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4) and (4A). In (5B) shown in FIG. 17, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from among multiple quadrangular pattern candidates. In (5B) shown in FIG. 17, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0073] Next, the process flow for detecting a rectangular target portion from the captured image IM2 shown in FIG. 18 will be described. In (1) of FIG. 18, the captured image (original image) IM2 is input from the imaging device 8. In (2) of FIG. 18, vertex candidates that can become the vertices of a rectangular shape are extracted from the input captured image IM2. In (4) of FIG. 18, combinations of vertex candidates that can become the vertices of a rectangular shape when connected, i.e., rectangular pattern candidates that can become the four sides of a rectangular shape, are derived. More specifically, multiple rectangular pattern candidates that fit within the enclosing area 70A of the specified pattern 70 shown in FIG. 16A are derived. In (5) of FIG. 18, multiple rectangular pattern candidates are selected and evaluated, and from the remaining multiple rectangular pattern candidates, a rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60 is determined. 18, there is no rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60. As a result, the rectangular target portion corresponding to the marker 60 is not detected.

[0074] In (1A) shown in FIG. 18, a reduced image RIMa2 is created at a scale of 1 / 2, and in (1B) shown in FIG. 18, a reduced image RIMb2 is created at a scale of 1 / 4. In (2A) shown in FIG. 18, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM2. In (4A) shown in FIG. 18, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4). In (5A) shown in FIG. 18, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from multiple quadrangular pattern candidates. This allows the quadrangular target portion corresponding to the marker 60 to be detected.

[0075] In (2B) shown in FIG. 18, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM2. In (4B) shown in FIG. 18, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4) and (4A). In (5B) shown in FIG. 18, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from among multiple quadrangular pattern candidates. In (5B) shown in FIG. 18, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0076] Next, the process flow for detecting a rectangular target portion from the captured image IM3 shown in FIG. 19 will be described. In (1) of FIG. 19, the captured image (original image) IM3 is input from the imaging device 8. In (2) of FIG. 19, vertex candidates that can become the vertices of a rectangular shape are extracted from the input captured image IM3. In (4) of FIG. 19, combinations of vertex candidates that can become the vertices of a rectangular shape when connected, i.e., rectangular pattern candidates that can become the four sides of a rectangular shape, are derived. More specifically, multiple rectangular pattern candidates that fit within the enclosing area 70A of the specified pattern 70 shown in FIG. 16A are derived. In (5) of FIG. 19, multiple rectangular pattern candidates are selected and evaluated, and from the remaining multiple rectangular pattern candidates, a rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60 is determined. 19, there is no rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60. As a result, the rectangular target portion corresponding to the marker 60 is not detected.

[0077] In (1A) shown in FIG. 19, a reduced image RIMa3 is created at a scale of 1 / 2, and in (1B) shown in FIG. 19, a reduced image RIMb3 is created at a scale of 1 / 4. In (2A) shown in FIG. 19, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM3. In (4A) shown in FIG. 19, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4). In (5A) shown in FIG. 19, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from multiple quadrangular pattern candidates. In (5B) shown in FIG. 19, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0078] In (2B) shown in FIG. 19, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM3. In (4B) shown in FIG. 19, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4) and (4A). In (5B) shown in FIG. 19, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from among multiple quadrangular pattern candidates. This allows the quadrangular target portion corresponding to the marker 60 to be detected.

[0079] In this embodiment, a different arithmetic processing unit 41C performs the processing for each of the above five processes (1) to (5) (i.e., captured image acquisition process, vertex candidate extraction process, line segment candidate derivation process, pattern candidate derivation process, and target portion detection process). That is, the target portion detection device 41 of this embodiment is provided with dedicated arithmetic processing units 41C that process each of the captured image acquisition process, vertex candidate extraction process, line segment candidate derivation process, pattern candidate derivation process, and target portion detection process.

[0080] Similarly, in this embodiment, a different storage processing unit 41B stores each piece of information for each of the processes in the above five (1) to (5) (i.e., captured image acquisition process, vertex candidate extraction process, line segment candidate derivation process, pattern candidate derivation process, and target portion detection process). That is, the target portion detection device 41 of this embodiment is provided with dedicated calculation processing units 41C that process each of the captured image acquisition process, vertex candidate extraction process, line segment candidate derivation process, pattern candidate derivation process, and target portion detection process.

[0081] Although the above description has been given using an example in which a dedicated arithmetic processing unit 41C is provided, the present invention is not limited to this. For example, a portion of the processes in the above embodiment as shown in FIG. 20A will be described as an example. Specifically, the arithmetic processing unit 41C extracts vertex candidates (step S11), the storage processing unit 41B stores the extracted vertex candidates (step S12), line segment candidates are derived from the extracted vertex candidates (step S13), the storage processing unit 41B stores the derived line segment candidates (step S14), pattern candidates are derived from the derived vertex candidates (step S15), and the storage processing unit 41B stores the derived pattern candidates (step S16). This description will be given using an example in which a portion of the processes in the above embodiment is explained.

[0082] In the above embodiment, a dedicated calculation processing unit 41C or memory processing unit 41B is provided for each of steps S11 to S16. However, for example, as shown in FIG. 20(A), if there is a memory processing unit 41B for the previous processing (step S12) that has already output the stored information (is no longer needed) in step S16, the area in that memory processing unit 41B may be used (reused).

[0083] As another method, as shown in FIG. 20(B) or 20(C), the subsequent processing may use information stored in the previous processing. It is possible to envision a case in which the subsequent processing does not directly store values ​​but only stores the storage locations referenced by the storage in the previous processing. For example, instead of directly storing the coordinates of each vertex as information on a rectangle candidate, it is possible to envision an operation in which only the storage locations of a list of vertices in a storage device are retained and referenced as needed. This method is effective in reducing storage capacity when the data capacity of the referenced storage locations can be kept smaller than the data capacity of the values.

[0084] Furthermore, not only for the storage processing unit 41B of another processing stage, but also for the storage processing unit 41B of the same processing stage, if there is a storage area from which stored information has already been output (is no longer needed), that area may be reused. For example, when serial processing is included, a storage area for an image range that has already been referenced may be reused.

[0085] The controller 40 detects the selected rectangular pattern candidate as a rectangular target portion. The controller 40 can acquire the following information from the rectangular target portion detected by the target portion detection device 41. That is, the controller 40 can acquire, for example, the relative positional relationship and absolute positional relationship between the rectangular target portion and the imaging device 8, the attitude of the target portion, the display content (signal information) displayed by the target portion, the identifier of the target portion, the graphic pattern of the target portion, and the movement of the target portion (attitude, attitude change, time change in signal information). For example, the controller 40 may control the traveling speed of the traveling vehicle 6 based on the size of the rectangular shape corresponding to the marker 60 detected by the target portion detection device 41. The controller 40 may also control each part of the traveling vehicle 6 based on multiple rectangular patterns detected by the target portion detection device 41.

[0086] The effects of the target portion detection device 41 of the above embodiment will be described. In the target portion detection device 41 of the above embodiment, the captured image IM is divided into a plurality of sections M, and independent image processing is performed on each of the divided sections M. This reduces the load on the calculation processing unit 41C compared to performing image processing on the entire image. As a result, the time required to detect a rectangular target portion included in the captured image IM can be shortened.

[0087] In conventional object portion detection devices, the time required for detection processing when increasing the resolution of an input image increases exponentially with the number of pixels, but the object portion detection device 41 of the above embodiment can use the time required for a relatively small image size divided into a mesh as a reference, thereby suppressing the increase to a linear order. This is effective not only when all divided ranges can be processed in parallel, but also when some of the entire divided ranges or all of the entire divided ranges are processed in series. In the object portion detection device 41 of the above embodiment, by dividing the divided ranges into smaller parts, the maximum amount of information handled in each divided range can be sufficiently reduced, thereby reducing the theoretical maximum processing time.

[0088] Furthermore, if the distribution of information volume on an image is somewhat fixed (e.g., shapes tend to occur more frequently in the center but less frequently in the periphery), as shown in Figures 14(A) to 14(C), by varying the size of the division meshes depending on the position, it is possible to reduce the number of divisions while keeping the maximum amount of information contained in each mesh small. This makes it possible to reduce processing time while also suppressing its variance. As a result, the processing time at each processing stage becomes stable, making it easier to perform detection at a constant cycle.

[0089] Furthermore, by handling information using a separate storage processing unit 41B or reference range for each divided range, it becomes easier to identify the location of the information to be referenced within the storage processing unit 41B. In addition, since the maximum amount of information within the divided image range can be made sufficiently small, processing can be completed in a short time even with a simple reference method such as a brute force search. In addition, since it is possible to use a small-scale implementation method such as SRAM, which has high-speed random access and easy channel allocation, even more complex reference methods can be implemented with low latency. As a result, processing time can be shortened.

[0090] In the target portion detection device 41 of the above embodiment, the detection unit 45 may detect a target portion included in the captured image through at least two processes including a first stage and a second stage, and the division unit 49 may divide the captured image into multiple sections so that the division patterns differ from each other for the first stage and the second stage. As shown in FIG. 21 , the multiple calculation processing units 41C may include multiple first calculation units provided corresponding to each of the multiple sections divided in accordance with the first stage and capable of independently performing image processing on each of the multiple sections, and multiple second calculation units provided corresponding to each of the multiple sections divided in accordance with the second stage and capable of independently performing image processing on each of the multiple sections. This configuration can reduce the time required for each process performed by the target portion detection device 41.

[0091] The target portion detection device 41 of the above embodiment includes a plurality of storage processing units 41B provided corresponding to each of the plurality of sections M, and storing information about the images of the corresponding sections M. This reduces the time required to read and write information about the images of the corresponding sections M.

[0092] 16(B), in the object portion detection device 41 of the above embodiment, the detection unit 45 derives, from among a plurality of vertex candidates, a plurality of line segment candidates each consisting of a combination of vertex candidates that can form one side of a quadrangular pattern, derives from among the derived plurality of line segment candidates a plurality of connecting line segment candidates each consisting of a combination of line segment candidates that can form two adjacent sides of the quadrangular pattern, derives from among the derived plurality of connecting line segment candidates a combination of connecting line segment candidates that can form all four sides of the quadrangular pattern, and detects a quadrangular object portion based on the derived combination of connecting line segment candidates. This configuration makes it possible to detect a quadrangular object portion based on the extracted vertex candidates through simple processing.

[0093] In the target portion detection device 41 according to the above embodiment, when detecting rectangular target portions contained in the original image IM and the reduced image RIM, a combination of vertex candidates that can form the vertices of a rectangular pattern that falls within the range of the specified pattern 70 is derived. This makes it possible to reduce the reference to the captured image IM during the detection process compared to when detecting all rectangular target portions contained in the original image IM and the reduced image RIM, and shorten the time required to detect the rectangular target portions.

[0094] In the target portion detection device 41 according to the above embodiment, the detection unit 45 derives combinations of vertex candidates that can form the vertices of a quadrangular pattern that fits within the enclosing area 70A of the first frame portion 71 and the second frame portion 72. This reduces the number of comparisons of combinations of vertex candidates that can form the vertices of a quadrangular pattern that fits within the range of the specified pattern 70.

[0095] Although one embodiment has been described above, one aspect of the present invention is not limited to the above embodiment. Various modifications are possible without departing from the spirit of the invention. (Variation 1) In the target portion detection device 41 according to Variation 1, the detection unit 45 identifies the direction of the vertex candidates extracted by the extraction unit 43 based on the light and dark patterns in the four segmented regions R1, R2, R3, and R4, and derives combinations of vertex candidates that can form a quadrangular pattern based on information regarding the identified direction. Specific examples of the direction of the vertex candidates extracted by the extraction unit 43 are vertex Pa in the upper left direction, vertex Pb in the upper right direction, vertex Pc in the lower left direction, and vertex Pd in ​​the lower right direction, as shown in FIG. 22(A). The extraction unit 43 can determine whether the vertex is the upper left vertex Pa, the upper right vertex Pb, the lower left vertex Pc, or the lower right vertex Pd based on the light and dark combination pattern of four pixels PX consisting of two rows and two columns, as shown in Figure 8 (A).

[0096] As shown in Fig. 22(B), in the target portion detection device 41 according to the first modification, (1) a captured image IM is input from the imaging device 8, and (2) vertex candidates that can become vertices of a quadrangular shape are extracted from the input captured image IM. At this time, the target portion detection device 41 according to the first modification recognizes in which direction the vertex candidates are vertices. In Fig. 22(B), the vertex Pa in the upper left direction is indicated by a circle, the vertex Pb in the upper right direction by an x, the vertex Pc in the lower left direction by a triangle, and the vertex Pd in ​​the lower right direction by a square.

[0097] The object portion detection device 41 according to the first modification (3) extracts only the combination of the vertex Pa in the upper left direction and the vertex Pc in the lower left direction, and the combination of the vertex Pb in the upper right direction and the vertex Pd in ​​the lower right direction, when detecting line segment candidates (up and down direction) that can become sides of a quadrangle from the extracted vertex candidates.Furthermore, the object portion detection device 41 according to the first modification (3) extracts only the combination of the vertex Pa in the upper left direction and the vertex Pb in the upper right direction, and the combination of the vertex Pc in the lower left direction and the vertex Pd in ​​the lower right direction, when detecting line segment candidates (up and down direction) that can become sides of a quadrangle from the extracted vertex candidates.

[0098] The object portion detection device 41 according to the first modification (4) extracts a combination of an upper-left vertex Pa, an upper-right vertex Pb, a lower-left vertex Pc, and a lower-right vertex Pd when detecting a quadrangular pattern that can become the four sides of a quadrangular shape from the detected line segment candidates. This allows multiple quadrangular pattern candidates to be detected. The object portion detection device 41 according to the first modification (5) determines, from the multiple quadrangular pattern candidates, a quadrangular pattern candidate that matches a quadrangular pattern corresponding to a quadrangular object portion, thereby detecting the quadrangular object portion.

[0099] The target portion detection device 41 according to variant example 1 can reduce the number of combinations of vertex candidates that can form a rectangular pattern, thereby further reducing the time required to detect a rectangular target portion contained in an image.

[0100] In the above-mentioned modified example 1, a different arithmetic processing unit 41C performs the processing for each of the above five processes (1) to (5). Similarly, in the above-mentioned modified example 1, a different storage processing unit 41B stores the information for each of the above-mentioned five processes (1) to (5). Furthermore, in the (2) vertex candidate extraction process, the storage processing unit 41B itself that stores the extracted vertex candidates may be divided into separate units, or one storage processing unit 41B may be divided into multiple storage areas, for each light and dark combination pattern of four pixels PX consisting of two rows and two columns, i.e., vertex Pa in the upper left direction, vertex Pb in the upper right direction, vertex Pc in the lower left direction, and vertex Pd in ​​the lower right direction.

[0101] In the process (1) above, one arithmetic processing unit 41C processes one input image. An example of such a process includes the generation of a derived image that can be used to detect a rectangular pattern. Processes that can be executed when the input image is imported into the arithmetic processing unit 41C include, for example, filter processing. Filter processing can be configured in a pipeline for the stream input of image data, and this can be applied to detect edge image data and candidates for rectangular vertices.

[0102] Since such processing can only be applied to one pixel range at a time, even if divided coordinate ranges are set and processing is performed by multiple arithmetic processing units 41C, there is no effect on parallelization or speed. For this reason, it is appropriate to perform such processing by a single arithmetic processing unit 41C in order to reduce costs. Furthermore, one arithmetic processing unit 41C may only have the function of masking the input image by the divided coordinate range and transmitting it to the corresponding multiple storage processing units 41B or multiple arithmetic processing units 41C in the subsequent stage, without performing the above-mentioned filter processing or the like.

[0103] When one process is executed by multiple arithmetic processing units 41C, the arithmetic processing units 41C refer to multiple storage processing units 41B that have the type of information necessary to configure the shape to be detected at that stage. Here, if multiple types of information are stored in one type of storage processing unit 41B, the multiple arithmetic processing units 41C that execute one process may refer to only that storage processing unit 41B. If a storage processing unit 41B is stored for each type of information, the multiple arithmetic processing units 41C that execute one process may refer to multiple storage processing units 41B.

[0104] The multiple arithmetic processing units 41C that execute one process and reference the multiple storage processing units 41B define one piece of information as a reference when, for example, there are two or more pieces of information that make up the detected shape. For example, when detecting a horizontal line segment, it is the vertex located on the left side, and when detecting a combination of two horizontal and vertical line segments, it is the straight line on the horizontal side. For the information defined as the reference, the multiple storage processing units 41B that include coordinate ranges corresponding to each of the multiple arithmetic processing units 41C are associated as reference destinations.

[0105] For example, as shown in Fig. 23, when a mesh is evenly divided (each mesh is an a x a square), to detect a line segment from two vertices, a region (e.g., hatched region HA) corresponding to information defined as a reference is associated with a plurality of storage processing units 41B including a coordinate range (e.g., hatched region HA1) corresponding to each of a plurality of calculation processing units 41C as reference destinations for the region (e.g., hatched region HA). Note that the size a of one side of the mesh can be adjusted to the maximum possible size of a line segment candidate, i.e., the size of the specified pattern 70 shown in Fig. 9(A).

[0106] For information other than the reference information, a wider range including the periphery of the coordinate range corresponding to each of the multiple arithmetic processing units 41C is used as the reference for the correspondence. Restrictions may be imposed on this correspondence. For example, when detecting a horizontal line segment, if the left vertex is used as the reference, the right vertex will be detected. Therefore, the multiple arithmetic processing units 41C can correspond only to the right of the coordinate range corresponding to each of the multiple arithmetic processing units 41C. Furthermore, it is also possible to limit the coordinate range that information other than the reference information can take on the image based on the actual positional relationship between the rectangular detection target and the imaging device 8, and have the multiple arithmetic processing units 41C refer only to the multiple storage processing units 41B within that range.

[0107] For example, as shown in Figure 23, in the case of equal division (one mesh is an a x a square), if the left vertex V is defined as the reference, then from the position that the left vertex V can take within the hatched area HA1, the right vertex V can be assumed to exist within six ranges of mesh HA2: the hatched area HA1 and the adjacent areas to the right, above and below.

[0108] As described in FIG. 23 , the coordinate range of the multiple storage processing units 41B referenced by the multiple arithmetic processing units 41C may be wider than the coordinate range corresponding to each of the multiple arithmetic processing units 41C. In this case, different multiple arithmetic processing units 41C may refer to the same multiple storage processing units 41B. In such a case, exclusive control (access) may be used. The multiple subsequent storage processing units 41B that store the detection results by the multiple arithmetic processing units 41C may be storage processing units 41B that correspond to the coordinate range in which the information defined as the reference is located.

[0109] The division method of the coordinate ranges corresponding to each of the multiple arithmetic processing units 41C and the multiple storage processing units 41B may differ from the division method at the current stage (the coordinate ranges held by the information defined as the reference), and the results of the multiple arithmetic processing units 41C may be stored and transmitted to multiple storage processing units 41B at a subsequent stage. In this case, there is a possibility that the timing of writing values ​​may overlap. In such a case, priorities may be set for the multiple arithmetic processing units 41C so that only one detection result with overlapping write timing is retained and stored, or a queue may be introduced so that the results are stored with a time lag.

[0110] (Other Modifications) In the above embodiment, an example has been described in which the imaging device 8 is provided on the overhead traveling vehicle 6, but it may also be mounted on a traveling device such as an automated guided vehicle (AGV) or an autonomous mobile transport robot (AMR), or on a fixed-point observation device, etc.

[0111] 4...track, 6...overhead traveling vehicle (traveling cart), 8...imaging device, 35...rear cover, 40...controller, 41...target portion detection device, 41A...input processing unit, 41B...storage processing unit (storage unit), 41C...arithmetic processing unit (arithmetic unit), 41D...output processing unit, 41E...storage device, 43...extraction unit, 45...detection unit, 47...creation unit, 49...division unit, 60...marker, 61...black area, 63...white area, 70...specified pattern, 70A...enclosing area, 71...first frame portion, 72...second frame portion, 160...logo, A1...determination area, IM...original image (captured image), RIM...reduced image, R1, R2, R3, R4...division areas.

Claims

1. A target portion detection device that detects a rectangular target portion included in an image captured by an imaging device, comprising: a division unit that divides the captured image into a plurality of sections; a plurality of calculation units that are provided corresponding to each of the plurality of sections and are capable of independently executing image processing in the corresponding section; and a detection unit that detects the target portion included in the captured image based on the calculation results by the plurality of calculation units.

2. The target portion detection device of claim 1, wherein the detection unit detects the target portion contained in the captured image by performing multiple processes, and the division unit divides the captured image into multiple sections with different division patterns for each of the multiple processes.

3. The target portion detection device according to claim 1, wherein the detection unit detects the target portion included in the captured image by first-stage and second-stage processing, the division unit divides the captured image into a plurality of different sections according to each of the first-stage and second-stage processing, and the plurality of calculation units include a plurality of first calculation units provided corresponding to each of the plurality of sections corresponding to the first stage and capable of performing image processing in each section independently, and a plurality of second calculation units provided corresponding to each of the plurality of sections corresponding to the second stage and capable of performing image processing in each section independently.

4. A target portion detection device as described in claim 1, wherein the multiple calculation units include: the calculation unit that executes a captured image acquisition process to acquire the captured image from the imaging device; the calculation unit that executes a vertex candidate extraction process to extract vertex candidates that can become vertices of the quadrangular shape from the captured image; the calculation unit that executes a line segment candidate derivation process to derive line segment candidates that can become sides of the quadrangular shape from the vertex candidates; the calculation unit that executes a pattern candidate derivation process to derive quadrangular pattern candidates that can become four sides of the quadrangular shape from the line segment candidates; and the calculation unit that executes a target portion detection process to determine, from the multiple quadrangular pattern candidates, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to a target portion of the quadrangular shape.

5. The object portion detection device according to claim 1, further comprising a plurality of storage units provided corresponding to the plurality of sections, each storing information relating to an image of the corresponding section.

6. A target portion detection device as claimed in any one of claims 1 to 5, comprising: a storage device that stores a quadrangular pattern corresponding to the target portion; an extraction unit that extracts a plurality of vertex candidates that can become vertices of the quadrangular shape included in the captured image; a detection unit that derives a combination of the vertex candidates that can form vertices of the quadrangular pattern from the plurality of vertex candidates extracted by the extraction unit, and detects the quadrangular target portion based on the derived combination of vertex candidates and the quadrangular pattern; and a creation unit that reduces the original image that is the captured image to create a reduced image, wherein the detection unit derives a combination of the vertex candidates that can form vertices of the quadrangular pattern that fall within the range of a specified pattern that defines lower and upper limit values ​​of size, from each of the original image and the reduced image.

7. A target portion detection device as described in claim 6, wherein the specified pattern is composed of a first frame portion that specifies the lower limit value and a second frame portion that is arranged to surround the first frame portion and that specifies the upper limit value, and the detection unit derives combinations of vertex candidates that can form vertices of the quadrangular pattern that fit within the enclosed area of ​​the first frame portion and the second frame portion.

8. The target portion detection device according to any one of claims 1 to 5, wherein the detection unit detects the rectangular two-dimensional code.

9. A traveling vehicle comprising: a traveling unit that travels along a predetermined traveling route; the target portion detection device according to any one of claims 1 to 5; and a controller that controls the traveling of the traveling unit based on information obtained from the rectangular target portion detected by the target portion detection device.

Images