Object part detection device and traveling cart

Abstract

An object part detection device (41) is a device for detecting a quadrilateral object part included in a captured image captured by an imaging device. The object part detection device (41) comprises: a storage unit (41E) that stores a quadrilateral pattern corresponding to an object part; an extraction unit (43) that extracts a plurality of vertex candidates that could be quadrilateral vertices included in the captured image; and a detection unit (45) that derives, from among the plurality of vertex candidates extracted by the extraction unit (43), a combination of vertex candidates that can form the vertices of the quadrilateral pattern, and detects the quadrangular object part on the basis of the derived combination of vertex candidates.

Description

Target part detection device and traveling vehicle

[0001] One aspect of the present invention relates to an object portion detection device and a traveling vehicle that detect an object portion included in an image.

[0002] There is known an object portion detection device (rectangle detection device) that detects a rectangular shape from image data captured by an imaging device. For example, Patent Document 1 discloses a detection device that detects a rectangular shape as an object portion contained in an image. The rectangle detection device of Patent Document 1 acquires image data from an image captured by an on-board camera, from which edges have been detected, and performs a partial matching process between multiple pieces of image data at different magnifications and a single square template to determine the degree of partial matching between the image data and each side of the template at the scanning position where the image data is scanned. The object portion detection device then integrates the partial matching degrees stored for each piece of image data at different magnifications to detect a rectangular shape or an image area where a rectangular shape is present from the image data.

[0003] Japanese Patent Application Laid-Open No. 2015-219820

[0004] In the conventional target portion detection device described above, a matching process is performed to compare images to detect the target portion, but such a process generally requires time. Furthermore, in devices such as transport vehicles that incorporate such target portion detection devices, there is a demand for a detection range that is as wide as possible. In order to widen the detection range, it is necessary to increase the resolution of the captured image captured by the imaging device. However, the higher the resolution of the captured image, the greater the number of target portions, and the longer the processing time required to detect the target portions.

[0005] Therefore, an object of one aspect of the present invention is to provide an object portion detection device and a traveling vehicle that can reduce the time required to detect a rectangular object portion included in an image.

[0006] (1) A target portion detection device according to one aspect of the present invention is a target portion detection device that detects a rectangular target portion included in an image captured by an imaging device, and includes: a memory unit that stores a rectangular pattern corresponding to the target portion; an extraction unit that extracts a plurality of vertex candidates that can become vertices of the rectangular shape included in the captured image; and a detection unit that derives a combination of vertex candidates that can form vertices of the rectangular pattern from the plurality of vertex candidates extracted by the extraction unit, and detects the rectangular target portion based on the derived combination of vertex candidates and the rectangular pattern.

[0007] This object portion detection device extracts multiple vertex candidates that can be the vertices of a quadrangular shape included in an image, derives a combination of the extracted vertex candidates that can form the vertices of a quadrangular pattern from the extracted multiple vertex candidates, and detects a quadrangular object portion based on the derived combination of vertex candidates and the quadrangular pattern. This eliminates the need for a matching process, such as extracting a contour from differences in light and dark in the image and comparing the shape of the extracted contour and image information inside the contour with pre-stored image information, as in the conventional method. As a result, the time required to detect a quadrangular object portion included in an image can be reduced.

[0008] (2) In the object portion detection device described in (1) above, when a region consisting of four divided regions separated by two mutually orthogonal virtual straight lines is set as a determination region, the extraction unit may determine whether the determination region is a vertex candidate based on the light and dark patterns in the four divided regions. In this configuration, it is possible to easily extract vertices that can become vertices of a quadrangular shape using a simple method based on the light and dark patterns of the four divided regions separated by the two mutually orthogonal virtual straight lines.

[0009] (3) In the object portion detection device described in (2) above, the extraction unit may detect vertex candidates based on a combination pattern of light and dark in four pixels arranged in two rows and two columns. This configuration makes it possible to detect vertex candidates more easily and quickly.

[0010] (4) In the object portion detection device described in (2) or (3) above, the detection unit may identify the direction of the vertex candidates extracted by the extraction unit based on the light and dark patterns in the four segmented regions, and derive combinations of vertex candidates that can form a quadrangular pattern based on information about the identified direction. This configuration can reduce the number of combinations of vertex candidates that can form a quadrangular pattern, thereby further shortening the time required to detect a quadrangular object portion included in an image.

[0011] (5) In the object portion detection device described in any one of (1) to (4) above, the detection unit may derive, from among a plurality of vertex candidates, a plurality of line segment candidates each consisting of a combination of vertex candidates that can form one side of a quadrangular pattern, derive, from among the derived plurality of line segment candidates, a plurality of connecting line segment candidates each consisting of a combination of line segment candidates that can form two adjacent sides of the quadrangular pattern, derive, from among the derived plurality of connecting line segment candidates, a combination of connecting line segment candidates that can form four sides of the quadrangular pattern, and detect a quadrangular object portion based on the quadrangular pattern candidate each consisting of the derived combination of connecting line segment candidates and the quadrangular pattern stored in the storage unit. This configuration enables detection of a quadrangular object portion based on the extracted vertex candidates through simple processing.

[0012] (6) The object portion detection device according to any one of (1) to (5) above may include a creation unit that creates a reduced image by reducing an original image that is a captured image, and the detection unit may derive, from each of the original image and the reduced image, combinations of vertex candidates that can form vertices of a quadrangular pattern that falls within a range of a specified pattern that defines a lower limit value and an upper limit value of the size. This reduces the number of comparisons of vertex candidate combinations during the detection process compared to detecting all quadrangular object portions included in the original image and the reduced image, and shortens the time required to detect quadrangular object portions.

[0013] (7) In the object portion detection device described in (6) above, the specified pattern may be configured with a first frame portion that defines a lower limit value and a second frame portion that is arranged to surround the first frame portion and defines the upper limit value, and the detection unit may derive combinations of vertex candidates that can form vertices of a quadrangular pattern that fits within the enclosed area of ​​the first frame portion and the second frame portion. This configuration can reduce the number of comparisons of combinations of vertex candidates that can form vertices of a quadrangular pattern that fits within the range of the specified pattern.

[0014] (8) In the object portion detection device according to any one of (1) to (7), the detection unit may detect a rectangular two-dimensional code. This configuration allows for more rapid detection of the two-dimensional code.

[0015] (9) A traveling vehicle according to one aspect of the present invention may include a traveling unit that travels along a predetermined traveling path, the target portion detection device described in any one of (1) to (8) above, and a controller that controls the traveling of the traveling unit based on information obtained from the rectangular target portion detected by the target portion detection device. Since the traveling vehicle can quickly detect the rectangular target portion, it can respond even if it is necessary to control the traveling unit within a limited time based on information acquired by an imaging device while traveling.

[0016] According to one aspect of the present invention, it is possible to reduce the time required to detect a target portion included in an image.

[0017] FIG. 1 is a schematic configuration diagram showing a traveling vehicle system including a traveling vehicle equipped with an object portion detection device according to an embodiment. FIG. 2 is a side view of the traveling vehicle of FIG. 1 as seen from the side. FIG. 3 is a rear view of the main body of the traveling vehicle of FIG. 1 as seen from behind in the traveling direction. FIG. 4 is a block diagram showing the functional configuration of the traveling vehicle. FIG. 5 is a diagram explaining an overview of processing in the object portion detection device of FIG. 1. FIG. 6(A) is a diagram showing a rectangular pattern of the traveling vehicle and markers of FIG. 1. FIG. 6(B) is a diagram showing a logo pattern formed of a plurality of rectangular patterns. FIG. 6(C) is a diagram showing a rectangular pattern formed of a plurality of rectangular patterns. FIG. 6(D) is a diagram showing a rectangular pattern of the traveling vehicle, black areas of the marker, and white areas of the marker. FIG. 6(E) is a diagram showing vertex candidates that may constitute the rectangular pattern of the traveling vehicle and black areas of the marker. FIG. 6(F) is a diagram showing vertex candidates that may constitute the rectangular pattern of the logo. FIG. 7(A) is a diagram showing a determination region including four partitioned regions divided by two virtual straight lines that are perpendicular to each other. FIG. 7(B) is a diagram showing an example of scanning a marker portion. FIG. 7(C) is an enlarged view of a determination region obtained when scanning a marker portion. FIG. 8(A) is a diagram showing a combination pattern of light and dark in four pixels consisting of two rows and two columns. FIG. 8(B) is a diagram showing an edge detected when an edge extraction filter is applied in a direction along the left-right direction. FIG. 8(C) is a diagram showing an edge detected when an edge extraction filter is applied in a direction along the up-down direction. FIG. 9(A) is a diagram showing a quadrangular target portion detected by an object portion detection device. FIG. 9(B) is a flowchart showing the processing flow when the object portion detection device detects a quadrangular target portion. FIG. 10 is a block diagram showing the functional configuration of an object portion detection device according to Modification 1. FIG. 11(A) is a diagram explaining a prescribed pattern for scanning a captured image. FIG. 11(B) is a diagram showing whether or not a prescribed pattern is detected in a captured image and a reduced image at short, medium, and long distances. 12A to 12D are examples of circuit diagrams for generating reduced images, and Fig. 13 is a diagram for explaining the process of correcting the vertices of a quadrangular pattern to the positions of the vertices of a quadrangular pattern corresponding to the markers.Fig. 14 is a diagram illustrating an example of a processing outline in the target portion detection device according to Modification 1. Fig. 15 is a diagram illustrating another example of a processing outline in the target portion detection device according to Modification 1. Fig. 16 is a diagram illustrating yet another example of a processing outline in the target portion detection device according to Modification 1. Fig. 17(A) is a diagram illustrating the direction of the vertex candidate. Fig. 17(B) is a diagram illustrating an example of a processing outline in the target portion detection device according to Modification 2.

[0018] Hereinafter, an overhead traveling vehicle (traveling carriage) 6 (hereinafter also referred to as "traveling vehicle 6") equipped with a target portion detection device 41 according to one embodiment will be described with reference to the drawings. In the description of the drawings, identical elements are given the same reference numerals, and duplicated explanations will be omitted.

[0019] 1 to 3, a mobile vehicle system 1 will be described as an example in which multiple mobile vehicles 6 travel along a one-way track 4 installed on the ceiling or the like of a factory. The track 4 is suspended from, for example, the ceiling. The mobile vehicles 6 transport items 10, such as containers such as FOUPs (Front Opening Unified Pods) that store multiple semiconductor wafers and reticle pods that store glass substrates, as well as general parts.

[0020] The traveling vehicle 6 travels along the track 4 and transports the article 10. The traveling vehicle 6 is configured to be able to transfer the article 10. The traveling vehicle 6 is an overhead traveling automatic guided vehicle. The traveling vehicle 6 has a traveling unit 20, a main body unit 7, an imaging device 8, a marker 60, and a controller 40. The traveling unit 20 is configured to include a motor and the like, and causes the traveling vehicle 6 to travel along the track 4. The main body unit 7 has a main body frame, a lateral feed unit, a θ drive, an elevation drive unit, an elevation unit, a front cover 34, and a rear cover 35.

[0021] The imaging device 8 is provided on the front cover 34 of the main body 7 so that its imaging range is in front of the host traveling vehicle 6. The imaging device 8 is a device that includes a lens and an imaging element that converts light entering through the lens into an electrical signal. Examples of the imaging device 8 include a visible light camera and a near-infrared camera. The imaging device 8 captures an image of the area in front of the host traveling vehicle 6. For example, the imaging device 8 captures an image of the leading traveling vehicle 6A, which is a traveling vehicle 6 located in front of the host traveling vehicle 6, so that a marker 60 provided on the leading traveling vehicle 6A is included in the captured image IM. The captured image IM (see FIG. 5 ) captured by the imaging device 8 is acquired by a controller 40, which will be described in detail later.

[0022] The imaging device 8 may be integrated with or configured separately from a target portion detection device 41 (see FIG. 4 ) as an image processing device including a storage processing unit and an arithmetic processing unit. The storage processing unit may be, for example, a static random access memory (SRAM), a dynamic random access memory (DRAM), or a storage unit configured with a combination of SRAM and DRAM, or may be configured with multiple storage units to enable parallel processing. The arithmetic processing unit may be, for example, a central processing unit (CPU), a field programmable gate array (FPGA), or an arithmetic unit configured with a combination of a CPU and an FPGA, or may be configured with multiple arithmetic units to enable parallel processing. In this embodiment, the target portion detection device 41 is configured as part of the controller 40, i.e., separately from the imaging device 8, as described in detail below.

[0023] The marker 60 is provided on the rear cover 35 so as to be visible from a rear traveling vehicle 6B, which is a traveling vehicle 6 located behind the own traveling vehicle 6. The marker 60 is a two-dimensional code, for example, an AR marker or a QR code (registered trademark). In this embodiment, an example will be described in which the two-dimensional code is an AR marker. The marker 60 is formed to include a black area consisting of a square and a white area arranged in part of the black area. The white area formed inside the black area forms a patterned graphic.

[0024] To mount the markers 60 on the rear cover 35, for example, a panel on which the markers 60 are printed may be attached to the rear cover 35, or the markers 60 may be printed directly on the rear cover 35 or laser marked. The markers 60 may also be realized by at least one of welding or fitting a material having different reflectance in a specific wavelength range, and processing that results in different reflectance in a specific wavelength range, so that the imaging device 8 can recognize the difference between light and dark.

[0025] The controller 40 shown in FIG. 4 is an electronic control unit including a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The controller 40 controls various operations of the traveling vehicle 6. Specifically, the controller 40 controls the traveling unit 20, the traverse unit, the θ drive, the lift drive unit, the lift unit, and the imaging device 8. The controller 40 also communicates with the traveling vehicle controller 12 using a communication line (feeder line) of the track 4 or the like. The controller 40 can be configured as software in which a program stored in a ROM is loaded into a RAM and executed by the CPU. The controller 40 may also be configured as hardware including an electronic circuit or the like. A part of the controller 40 of this embodiment constitutes a target portion detection device 41. The target portion detection device 41 mainly includes an input processing unit 41A, a storage processing unit 41B, an arithmetic processing unit 41C, an output processing unit 41D, and a storage device (storage unit) 41E.

[0026] The input processing unit 41A is a unit that interfaces with external devices. For example, the input processing unit 41A receives the captured image IM from the imaging device 8. The storage processing unit 41B is configured, for example, by a DRAM, an SRAM, etc. The storage processing unit 41B stores the captured image IM processed by the target portion detection device 41, information necessary for the detection process, etc. The calculation processing unit 41C is configured, for example, by a CPU, an FPGA, etc. The calculation processing unit 41C reads the captured image IM, etc. from the storage processing unit 41B, and performs target portion detection processing. The output processing unit 41D is a unit that interfaces with external devices. The output processing unit 41D outputs the target portion detection results obtained by the calculation processing unit 41C to the outside. The storage device 41E is configured, for example, by a hard disk drive (HDD), a solid state drive (SSD), etc. The storage device 41E stores, for example, a rectangular pattern corresponding to a rectangular target portion, which will be described in detail later. The storage device 41E may temporarily store the captured image IM input from the imaging device 8.

[0027] The target portion detection device 41 detects a rectangular target portion from the captured image IM. Examples of rectangular target portions include, for example, a moving object, a fixed object, and portions (areas) contained therein. Examples of rectangular target portions contained in the captured image IM obtained from the imaging device 8 provided on the traveling vehicle 6 include, for example, another traveling vehicle 6 (rear cover 35), an item 10 transported by the traveling vehicle 6, the track 4 on which the traveling vehicle 6 travels, attachments to the track 4, a buffer (not shown) for holding the item 10, a marker 60 (see FIG. 3 ) provided on the traveling vehicle 6 or the track 4, a logo 160 (see FIG. 6B ), a figure, a symbol, etc. The controller 40 can execute various controls based on the rectangular target portion acquired by the target portion detection device 41. Note that the various controls referred to here will be described in detail later.

[0028] 5, the target portion detection device 41 (1) inputs a captured image IM from the imaging device 8, (2) extracts vertex candidates that can become the vertices of a quadrangle from the input captured image IM, (3) detects line segment candidates that can become the sides of the quadrangle from the extracted vertex candidates, (4) detects quadrangle pattern candidates that can become the four sides of the quadrangle from the detected line segment candidates, and (5) determines a quadrangle pattern candidate (solid line) that matches the quadrangle pattern corresponding to the quadrangle target portion from the detected multiple quadrangle pattern candidates (dashed lines), thereby detecting the quadrangle target portion. In the target portion detection device 41, hardware such as a CPU, RAM, and ROM cooperate with software such as a program to form an extraction unit 43 and a detection unit 45 that perform the above processes (1) to (5).

[0029] The storage device 41E stores a rectangular pattern corresponding to a rectangular target portion. As shown in FIG. 6A , the storage device 41E stores, for example, a rectangular pattern PA6 corresponding to the traveling vehicle 6 (rear cover 35) imaged from a predetermined position (each relative distance). If the rectangular target portion is a fixed object, the storage device 41E may store a rectangular pattern corresponding to the rectangular target portion for each image capture position. The storage device 41E also stores, for example, a rectangular pattern PA60 corresponding to a marker 60 provided on the rear cover 35 of the traveling vehicle 6 imaged from a predetermined position. The storage device 41E may store, for example, the rectangular pattern PA6 and the rectangular pattern PA60 individually or integrally.

[0030] For example, if a logo 160 such as that shown in FIG. 6B is depicted on the rear cover 35 of the traveling vehicle 6 captured from a predetermined position, the storage device 41E may store a rectangular pattern PA160 corresponding to the logo. The rectangular pattern PA160 corresponding to the logo may be stored as a combination of multiple rectangular patterns RS1 to RS3. The marker 60 (see FIG. 3) provided on the rear cover 35 is formed by a black area 61 that is the outline of the marker 60 and a white area 63 that indicates the pattern of the marker 60. As shown in FIG. 6D, the storage device 41E may store the rectangular pattern corresponding to the marker 60 as a rectangular pattern PA61 corresponding to the black area 61. The storage device 41E may also store the rectangular pattern corresponding to the marker 60 as a rectangular pattern PA63 corresponding to the white area 63. Furthermore, the storage device 41E may store the rectangular pattern PA61 and the rectangular pattern PA63 integrally.

[0031] The extraction unit 43 extracts all of a plurality of vertex candidates that can be vertices of a quadrilateral shape from the captured image IM. The vertex candidates are managed as information (data), such as position information (e.g., coordinate information) and an ID (identifier). The extraction unit 43 is mainly formed by the calculation processing unit 41C. For example, the vertices of a quadrilateral shape are the four vertices P6 in the rear cover 35 of the traveling vehicle 6 as shown in FIG. 6E, and the four vertices P60 in the black region 61 of the marker 60 affixed to the rear cover 35. Furthermore, for the logo 160, the vertices are the vertices P160 of each of the multiple quadrilateral patterns RS1 to RS3 as shown in FIG. 6F. Note that the vertices of the white region 63 of the marker 60 may also be extracted using a method similar to that for the logo 160.

[0032] The extraction unit 43 may acquire the captured image IM for extracting the vertices P6 and P60 directly from the imaging device 8, or may read it from the storage processing unit 41B or the storage device 41E in which the captured image IM captured by the imaging device 8 is stored. Furthermore, the extraction unit 43 may limit the range in which vertex candidates are detected from the entire captured image IM before extracting the vertex candidates. Furthermore, the extraction unit 43 may extract vertex candidates based on a cropped image in which a range in which vertex candidates are detected is cropped from the captured image IM captured by the imaging device 8. Furthermore, the extraction unit 43 may perform the vertex candidate detection process on an image obtained by performing image correction processing, reduction processing, etc. on the captured image IM captured by the imaging device 8. Furthermore, the extraction unit 43 may store intermediate images for performing image correction processing, reduction processing, etc. on the captured image IM captured by the imaging device 8 in the storage processing unit 41B.

[0033] The extraction unit 43 can extract vertex candidates from, for example, the brightness difference in a determination area A1, which is a small range in the captured image IM. For example, as shown in FIG. 7A , the extraction unit 43 sets four segmented areas R1, R2, R3, and R4 for the determination area A1 on the captured image IM, the segmented areas being divided by two mutually orthogonal imaginary straight lines SL1 and SL2 that pass through the center position CP of the determination area A1. In other words, the extraction unit 43 sets the four segmented areas R1, R2, R3, and R4 so as to include one of the four corners formed by the two mutually orthogonal imaginary straight lines SL1 and SL2. The extraction unit 43 determines whether the determination area A1 is a vertex candidate based on the brightness and darkness patterns in the four segmented areas R1, R2, R3, and R4.

[0034] For example, if the difference between the average brightness value of one of the four segmented regions R1, R2, R3, and R4 (e.g., segmented region R4) and the average brightness values ​​of the remaining three segmented regions (e.g., segmented regions R1, R2, and R3) is equal to or greater than a predetermined threshold, the extraction unit 43 determines that the judgment region A1 is a vertex candidate. Note that, when determining whether the judgment region A1 is a vertex candidate, the extraction unit 43 may take into account deviations and the like in addition to the average brightness value. Alternatively, the extraction unit 43 may perform a binarization process on the captured image IM before determining whether the judgment region A1 is a vertex candidate.

[0035] The extraction unit 43 scans the captured image IM in units of determination regions A1 to search for vertex candidates. For example, the extraction unit 43 extracts vertex candidates from an image of the determination region A1 obtained by scanning the marker 60 portion shown in FIG. 7B, which is a part of the captured image IM captured by the imaging device 8. As shown in FIG. 7C, the extraction unit 43 determines that the determination region A1 is not a vertex candidate based on the fact that the difference between the average luminance value of one segment in the determination region A1 obtained at scanning position SP1 and the average luminance values ​​of the remaining three segment areas is not equal to or greater than a predetermined threshold. Similarly, the extraction unit 43 determines that the determination region A1 obtained at scanning position SP3 is not a vertex candidate. The extraction unit 43 determines that the determination region A1 is not a vertex candidate based on the fact that the difference between the average luminance value of one segment in the determination region A1 obtained at scanning position SP2 and the average luminance values ​​of the remaining three segment areas is equal to or greater than a predetermined threshold. Similarly, the extraction unit 43 determines that the determination area A1 obtained at the scanning position SP4 is a vertex candidate.

[0036] As described above, when using a method for determining whether the determination region A1 is a vertex candidate based on the light and dark patterns in the four segmented regions R1, R2, R3, and R4, it is also possible to determine whether the determination region A1 is a vertex candidate based on the light and dark combination pattern of four pixels PX arranged in two rows and two columns, as shown in FIG. 8A. This is because, when the center of the four pixels PX is taken as the central position, the four pixels PX adjacent to the central position can be considered to be four segmented regions R1, R2, R3, and R4. As a result, when the difference between the average luminance value of one pixel PX and the average luminance values ​​of the remaining three pixels PX is equal to or greater than a predetermined threshold, the extraction unit 43 determines that the determination region A1 formed by the four pixels PX is a vertex candidate.

[0037] The extraction unit 43 may determine whether the determination area A1 is a vertex candidate by a method other than the method of determining whether the determination area A1 is a vertex candidate based on the light and dark patterns in the four divided areas R1, R2, R3, and R4 as described above. For example, the extraction unit 43 may apply an edge extraction filter to the captured image IM to detect edges, and determine the detected edges as vertex candidates.

[0038] Specifically, as shown in Fig. 8(B), the extraction unit 43 applies an edge extraction filter in the left-right direction to detect an edge E1 (dashed circle). Also, as shown in Fig. 8(C), the extraction unit 43 applies an edge extraction filter in the up-down direction to detect an edge E2 (dashed circle). The extraction unit 43 extracts, as vertex candidates, the edges detected by both the edge E1 detected when the edge extraction filter is applied in the left-right direction and the edge E2 detected when the edge extraction filter is applied in the up-down direction.

[0039] The extraction unit 43 may directly input the extracted vertex candidates to the detection unit 45 or may temporarily store them in the storage processing unit 41B. The following describes an example in which the extraction unit 43 temporarily stores the extracted vertex candidates in the storage processing unit 41B. The extraction unit 43 may divide the storage area of ​​the storage processing unit 41B for storing the extracted vertex candidates based on the light-dark pattern in the image of the determination area A1, i.e., the direction of the vertex of the rectangular pattern. The extraction unit 43 may divide the storage area of ​​the storage processing unit 41B for storing the extracted vertex candidates based on the coordinate range of the extracted vertex candidates in the captured image IM. The extraction unit 43 may also calculate and store information other than coordinates that can be used for detecting pattern rectangle candidates in a subsequent stage. The extraction unit 43 may also divide the storage area of ​​the storage processing unit 41B for storing this information, as described above.

[0040] The detection unit 45 derives a combination of vertex candidates that can form a quadrangular pattern based on the multiple vertex candidates extracted by the extraction unit 43. The detection unit 45 detects a quadrangular target portion based on the derived combination of vertex candidates. The detection unit 45 can limit the quadrangular target portion that can be detected based on the tilt, aspect ratio, or size that the quadrangular pattern can assume based on the actual positional relationship between the detection target and the imaging device 8.

[0041] 9B , the extraction unit 43 extracts vertex candidates that can form vertices of a quadrangular pattern (step S1), derives from the plurality of vertex candidates a plurality of line segment candidates each formed by a combination of the vertex candidates that can form one side of the quadrangular pattern (step S2), derives from the derived plurality of line segment candidates a plurality of connecting line segment candidates each formed by a combination of the line segment candidates that can form two adjacent sides of the quadrangular pattern (step S3), derives from the derived plurality of connecting line segment candidates a combination of the connecting line segment candidates that can form four sides of the quadrangular pattern (step S4), and detects the quadrangular target portion based on the quadrangular pattern candidate formed by the derived combination of the connecting line segment candidates and the quadrangular pattern stored in the storage device 41E (step S5).

[0042] An example will be described in which the rectangular target portion obtained by the processing of steps S1 to S5 described above is the marker 60 shown in FIG. 9A. The detection unit 45 reads the vertex candidates VC1, VC2, VC3, and VC4 extracted by the extraction unit 43 (step S1) and stored in the storage processing unit 41B. Next, the detection unit 45 derives multiple line segment candidates SC12, SC13, SC24, and SC34 that can form one side of the rectangular pattern based on the vertex candidates VC1, VC2, VC3, and VC4 read from the storage processing unit 41B (step S2). The detection unit 45 stores the detected line segment candidates SC12, SC13, SC24, and SC34 in the storage processing unit 41B.

[0043] Next, the detection unit 45 reads the line segment candidates SC12, SC13, SC24, and SC34 stored in the storage processing unit 41B and derives a plurality of connecting line segment candidates consisting of combinations of line segment candidates SC12, SC13, SC24, and SC34 that can form two adjacent sides of the quadrangular pattern (step S3). The detection unit 45 detects connecting line segment candidates SC12 & SC13 as a combination of line segment candidates that share the vertex candidate VC1 in common (i.e., connecting line segment candidates). The detection unit 45 also derives connecting line segment candidates SC24 & SC34 as a combination of line segment candidates that share the vertex candidate VC4 in common (i.e., connecting line segment candidates). The detection unit 45 stores the connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34 in the storage processing unit 41B.

[0044] The detection unit 45 reads out the connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34 stored in the storage processing unit 41B, and derives, from the derived plurality of connecting line segment candidates, combinations of connecting line segment candidates that can form the four sides of a quadrangular pattern (i.e., quadrangular pattern candidates) (step S4). The detection unit 45 derives the connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34 as combinations of connecting line segment candidates that share both the vertex candidate VC2 and the vertex candidate VC4. Based on these two connecting line segment candidates SC12 & SC13 and the connecting line segment candidates SC24 & SC34, the detection unit 45 derives a quadrangular pattern candidate PAC60 corresponding to the marker 60, as shown in FIG. 9A.

[0045] The quadrangular pattern candidates derived by the detection unit 45 may be temporarily stored in the storage device 41E. The detection unit 45 may store the quadrangular pattern candidates by storing coordinate information of the vertices that make up the quadrangular pattern candidates. Furthermore, the detection unit 45 may change the storage area of ​​the storage device 41E for storing each vertex candidate, for example, depending on the coordinate range of the vertex, or may store the absolute coordinates of one vertex candidate that makes up the quadrangular pattern candidate and the relative coordinates of the remaining vertex candidates based on the absolute coordinates of the one vertex candidate. Even in this case, the detection unit 45 may calculate the coordinate information of each vertex that makes up the quadrangular pattern candidate by, for example, reading the information stored from the storage device 41E and then performing a specific process.

[0046] The detection unit 45 detects a rectangular target portion based on the rectangular pattern candidates and the rectangular patterns stored in the storage device 41E. Specifically, the detection unit 45 evaluates and selects each of the rectangular pattern candidates stored in the storage device 41E. The detection unit 45 evaluates and selects a rectangular pattern candidate based on, for example, the degree of agreement between coordinate information of vertices constituting the rectangular pattern candidate and coordinate information of vertices constituting the rectangular pattern. The detection unit 45 may also evaluate and select a rectangular pattern candidate based on, for example, the degree of agreement between a rectangular pattern corresponding to another target portion located inside the rectangular pattern candidate and a rectangular pattern corresponding to another target portion located inside the rectangular pattern.

[0047] For example, the detection unit 45 may select a rectangular pattern candidate with the highest degree of match, or may select a rectangular pattern candidate with a degree of match equal to or higher than a predetermined level. The detection unit 45 may evaluate the degree of tilt or distortion of the rectangular shape indicated by the rectangular pattern candidate. The detection unit 45 may also preferentially select a rectangular pattern candidate with less tilt or distortion. This method can be effectively used in cases where the rectangular pattern has a complex shape, such as one formed by combining multiple rectangular patterns.

[0048] The detection unit 45 may also detect a rectangular target portion by performing a matching process (matching) between a rectangular pattern candidate and image information stored in the storage device 41E. In this detection method, the range of the image to be compared is limited, compared to a conventional matching process in which a contour is extracted from differences in brightness on the captured image IM and image information about the extracted contour and the interior of the contour is compared with pre-stored image information. Therefore, the time required to detect a rectangular target portion included in the captured image IM can be reduced.

[0049] The controller 40 detects the selected rectangular pattern candidate as a rectangular target portion. The controller 40 can acquire the following information from the rectangular target portion detected by the target portion detection device 41. That is, the controller 40 can acquire, for example, the relative positional relationship and absolute positional relationship between the rectangular target portion and the imaging device 8, the attitude of the target portion, the display content (signal information) displayed by the target portion, the identifier of the target portion, the graphic pattern of the target portion, and the movement of the target portion (attitude, attitude change, time change in signal information). For example, the controller 40 may control the traveling speed of the traveling vehicle 6 based on the size of the rectangular shape corresponding to the marker 60 detected by the target portion detection device 41. The controller 40 may also control each part of the traveling vehicle 6 based on multiple rectangular patterns detected by the target portion detection device 41.

[0050] The effects of the object portion detection device 41 of the above embodiment will be described. As shown in FIG. 5 , the object portion detection device 41 of the above embodiment extracts multiple vertex candidates that can be vertices of a quadrangular shape included in the captured image IM, derives a combination of vertex candidates that can form the vertices of a quadrangular pattern from the extracted multiple vertex candidates, and detects a quadrangular object portion based on the derived combination of vertex candidates. This eliminates the need to extract a contour from differences in brightness in the captured image IM and perform a matching process, such as comparing the shape of the extracted contour and image information inside the contour with pre-stored image information, as in the conventional method. As a result, the time required to detect a quadrangular object portion included in the captured image IM can be reduced.

[0051] According to the method for detecting a rectangular object portion using the object portion detection device 41 of the above embodiment, when detecting a rectangular object portion, reference to the captured image IM at times other than when data is input from the imaging device 8 is minimized. This prevents a significant increase in processing time due to an increase in the number of pixels when a higher resolution image IM is required. The object portion detection device 41 of the above embodiment uses vertex candidate data (a set of coordinate information) that is smaller in data volume than the captured image IM data itself, thereby enabling the use of a storage processing unit 41B such as an SRAM, which is small but has fast random access and a large number of channels. This reduces latency, which occurs mainly in DRAM when reading data in random order, and the occurrence of processing that cannot be parallelized due to channel limitations. This significantly contributes to increasing the processing speed of the object portion detection device 41. This is particularly useful when implementing the above processing using a non-von Neumann architecture (e.g., FPGA).

[0052] Furthermore, the number of vertex candidates detected in the quadrangular target portion detection method of the above embodiment is proportional to the number of pixels in the captured image IM input from the imaging device 8. Furthermore, the scale factor (proportionality coefficient) corresponds to the fineness of the surrounding environment that may appear in the captured image IM. For example, when detecting quadrangular target portions in a factory, the number of detected objects will be large because there are many small objects. On the other hand, when detecting flying objects such as drones in the air, the number of detected objects will be small because there are few surrounding objects. In this way, by determining the environment in which the target portion detection device 41 is used, the maximum number of detected vertex candidates can be estimated. By limiting the capacity of the memory processing unit 41B accordingly, the theoretical maximum processing time can be calculated. This stabilizes the processing time required to detect quadrangular target portions, making it easier to detect quadrangular target portions at a regular interval compared to conventional image processing.

[0053] 7A , when a determination region A1 is defined as a region consisting of four sectional regions R1, R2, R3, and R4 separated by two mutually orthogonal imaginary straight lines SL1 and SL2, the extraction unit 43 determines whether the determination region A1 is a vertex candidate based on the light and dark patterns in the four sectional regions R1, R2, R3, and R4. With this configuration, it is possible to easily extract vertices that can become vertices of a quadrangle using a simple method based on the light and dark patterns of the four sectional regions R1, R2, R3, and R4 separated by the two mutually orthogonal imaginary straight lines SL1 and SL2.

[0054] In the object portion detection device 41 of the above embodiment, the extraction unit 43 detects vertex candidates based on the combination pattern of light and dark in four pixels PX arranged in two rows and two columns, as shown in FIG. 8A. This allows for easier and faster detection of vertex candidates. Another major advantage of this method of detection is that it can be performed simply by scanning the captured image IM as shown in FIG. 7B, rather than scanning it up, down, left, and right. When an image is input from an imaging device, data is generally input in the scanning order shown in FIG. 7B. Therefore, this data can be sent directly to the calculation processing unit 41C for detection processing, eliminating the need to temporarily store image data in a storage processing unit, thereby reducing processing time.

[0055] 9(B), in the object portion detection device 41 of the above embodiment, the detection unit 45 derives, from among a plurality of vertex candidates, a plurality of line segment candidates each consisting of a combination of vertex candidates that can form one side of a quadrangular pattern, derives from among the derived plurality of line segment candidates a plurality of connecting line segment candidates each consisting of a combination of line segment candidates that can form two adjacent sides of the quadrangular pattern, derives from among the derived plurality of connecting line segment candidates a combination of connecting line segment candidates that can form all four sides of the quadrangular pattern, and detects a quadrangular object portion based on the derived combination of connecting line segment candidates. This configuration makes it possible to detect a quadrangular object portion based on the extracted vertex candidates through simple processing.

[0056] When detecting a quadrangle with four vertices all at once from the beginning without deriving line segment candidates and connecting line segment candidates as in the method for detecting a quadrangular target portion of the above embodiment, the number of comparisons (time required for comparison) is on the order of the fourth power of K, where K is the number of vertices. On the other hand, in the method for detecting a quadrangular target portion of the present embodiment, the number of comparisons required to detect line segments from vertices is on the order of the square of K. The processes for detecting connecting line segments from line segments and quadrangles from connecting line segments can each be completed in time on the order of the square of the number of candidates, which is effective in shortening the processing time.

[0057] Although one embodiment has been described above, one aspect of the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the invention.

[0058] (Variation 1) In the above embodiment, the extraction unit 43 has been described as extracting all of the multiple vertex candidates that can become the vertices of a quadrangular shape from the captured image IM, but the number of vertex candidates extracted from the captured image IM may be narrowed down. Specifically, as shown in Figures 4 and 10, the target portion detection device 41 according to Variation 1 includes a creation unit 47 that creates a reduced image RIM by reducing the original image IM, which is the captured image. The creation unit 47, like the extraction unit 43 and the detection unit 45, is formed by cooperation of hardware such as a CPU, RAM, and ROM with software such as a program.

[0059] Furthermore, the detection unit 45 of the target portion detection device 41 according to the first modification derives, from each of the original image IM and the reduced image RIM, a combination of vertex candidates that can form the vertices of a quadrangular pattern that fits within a specified pattern 70 that defines a lower limit and an upper limit for the size. As shown in FIG. 11A , the specified pattern 70 is composed of a first frame 71 that defines a lower limit and a second frame 72 that surrounds the first frame 71 and defines an upper limit. The detection unit 45 derives a combination of vertex candidates that can form the vertices of a quadrangular pattern that fits within an enclosed area 70A defined by the first frame 71 and the second frame 72.

[0060] A method for creating a reduced image by the creation unit 47 will now be described. As shown in FIGS. 4 and 10 , the creation unit 47 is mainly formed by the calculation processing unit 41C. The original image IM acquired by the creation unit 47 may be input directly from the imaging device 8 or may be input via the storage device 41E. The scale ratio used by the creation unit 47 to create the reduced image RIM can be set arbitrarily. For example, the creation unit 47 generates a reduced image RIM with a scale ratio of 1 / 2 and a reduced image RIM with a scale ratio of 1 / 4. The creation unit 47 sets the scale ratio and the combination of scale ratios as appropriate.

[0061] 12A, for example, one arithmetic processing unit 41C may sequentially create reduced images RIM at various scales based on the original image IM input from the imaging device 8, the storage processing unit 41B, or the storage device 41E. The creation unit 47 creates multiple reduced images RIM through this process. The arithmetic processing unit 41C outputs the created reduced images RIM at the multiple scales to the extraction unit 43 and the detection unit 45.

[0062] 12(B), for example, one arithmetic processing unit 41C may first create a reduced image RIM1 at a relatively large scale from the input original image IM, and then use the same arithmetic processing unit 41C to create a reduced image RIM2 at a smaller scale from the reduced image RIM1. The creation unit 47 may repeat this process multiple times to create multiple reduced images RIM1 and RIM2. The arithmetic processing unit 41C outputs the created reduced images RIM1 at multiple scales to the extraction unit 43 and the detection unit 45.

[0063] 12(C), for example, the creation unit 47 may have a plurality of arithmetic processing units 41C arranged in parallel, each of which creates reduced images RIMa, RIMb, and RIMc at each scale (scale a, scale b, and scale c) from the original image IM. That is, the creation unit 47 may have a plurality of arithmetic processing units 41C simultaneously create images RIMa, RIMb, and RIMc at each scale. The arithmetic processing units 41C output the created reduced images RIMa, RIMb, and RIMc at each scale to the extraction unit 43 and the detection unit 45.

[0064] 12(D), the creation unit 47 may have a first-stage arithmetic processing unit 41C arranged in series to create a reduced image RIMa with a relatively large scale a, a next-stage arithmetic processing unit 41C to create a reduced image RIMb with a scale b smaller than that of the reduced image RIMa, and a next-stage arithmetic processing unit 41C to create a reduced image RIMc with a scale c smaller than that of the reduced image RIMb. The creation unit 47 repeats this process multiple times to create multiple reduced images RIMa, RIMb, and RIMc.

[0065] The creation unit 47 may also be configured by selectively combining the circuit diagrams of Figures 12(A) to 12(D) described above. Each of the reduced images RIM created during the circuit diagrams of Figures 12(A) to 12(D) may be stored in the storage device 41E or the like, and then an even smaller scale image may be generated again by the arithmetic processing unit 41C, or may be used to detect rectangular pattern candidates. In addition to image reduction processing, the creation unit 47 may also perform image correction processing such as a sharpening filter or morphological transformation, or acquire information that can be used to detect rectangular patterns.

[0066] The following description will be given taking as an example a case where the target portion detection device 41 detects a marker 60 as a rectangular target portion. When the distance to the preceding traveling vehicle 6A is short, a medium distance farther than short, or a long distance farther than medium, the captured image IM including the marker 60 captured by the imaging device 8 provided on the traveling vehicle 6 differs, as shown in FIG. 11(B). In the case of the captured image IM shown in FIG. 11(B), the detection unit 45 can obtain the rectangular shape corresponding to the marker 60 only from the long-distance image. That is, when the specified pattern 70 shown in FIG. 11(A) is scanned over the captured image IM, all of the vertices of the rectangular shape corresponding to the marker 60 in the long-distance image fall within the enclosing area 70A. At most, one vertex of the quadrangle corresponding to the marker 60 in the close-up image fits within the surrounding area 70A, and at most, one vertex of the quadrangle corresponding to the marker 60 in the medium-distance image fits within the surrounding area 70A.

[0067] Furthermore, in the case of half-scale reduced images RIM corresponding to the close-distance image, middle-distance image, and long-distance image created by the creation unit 47, the detection unit 45 can acquire the quadrangular shape corresponding to the marker 60 only from the middle-distance image. That is, when the defined pattern 70 shown in Fig. 11A is scanned over the captured image IM, all of the vertices of the quadrangular shape corresponding to the marker 60 in the middle-distance image fit within the surrounding area 70A. At most, one vertex of the quadrangular shape corresponding to the marker 60 in the close-distance image fits within the surrounding area 70A, and at most, three vertices of the quadrangular shape corresponding to the marker 60 in the long-distance image fit within the surrounding area 70A.

[0068] Furthermore, in the case of quarter-scale reduced images RIM corresponding to the close-distance image, the middle-distance image, and the long-distance image created by the creation unit 47, the detection unit 45 can acquire the quadrangle corresponding to the marker 60 only from the long-distance image. That is, when the defined pattern 70 shown in Fig. 11A is scanned over the captured image IM, all of the vertices of the quadrangle corresponding to the marker 60 in the long-distance image fit within the surrounding area 70A. Of the vertices of the quadrangle corresponding to the marker 60 in the middle-distance image and the long-distance image, at most three vertices fit within the surrounding area 70A.

[0069] Furthermore, if the position (range) at which a rectangular pattern corresponding to a rectangular detection target can be captured on a scale drawing at each scale can be limited based on the actual positional relationship of the imaging device 8, the detection range of the rectangular detection target may be limited to the above range for each scale drawing. However, because the variation in the orientation of the rectangular shape corresponding to the rectangular detection target varies depending on the distance (corresponding to the detectable scale) between the imaging device 8 and the rectangular detection target, it is difficult to switch (adjust) the criteria for evaluating the degree of tilt or distortion of the rectangular shape within a single image. For example, when the imaging device 8 and the rectangular detection target are on the same linear trajectory, the tilt of the rectangular detection target (horizontal side) in the image relative to the tilt in the depth direction varies depending on the distance between the two. Specifically, there is almost no tilt when the distance is far, and the tilt becomes greater when the distance is close. Therefore, the larger the range of sizes of the rectangular detection target that can be captured, the more parameters are required for range limitation. Therefore, by limiting the range of size of the rectangular detection object for each scale drawing, it is possible to limit the orientation of the rectangular detection object that may appear, which is expected to make it easier to switch (adjust) the above judgment criteria.

[0070] 13 , when an attempt is made to match the position on the image of a rectangular pattern detected from a reduced image RIM with a small scale (e.g., 1 / 4) with the scale of the original image IM (e.g., 1 / 1), the rectangular pattern PA60 corresponding to the marker 60 and the rectangular pattern candidate PAC60 may be misaligned due to an error (discarded pixels) caused by the reduction process. Therefore, the creation unit 47 may perform an additional search process to correct the vertex P0 of the rectangular pattern candidate PAC60 to the position of the vertex P1 of the rectangular pattern PA60 corresponding to the marker 60.

[0071] The following describes the flow of processing when the target portion detection device 41 of Modification 1 detects a rectangular target portion (marker 60) when a captured image IM1 as shown in Fig. 14, a captured image IM2 as shown in Fig. 15, and a captured image IM3 as shown in Fig. 16 are input from the imaging device 8. The sizes of the markers 60 are different in the captured images IM1, IM2, and IM3.

[0072] First, the flow of processing up to detecting a rectangular target portion from a captured image IM1 shown in Fig. 14 will be described. The processes in (1), (2), (4), and (5) shown in Fig. 14 are the same as the processes in (1), (2), (4), and (5) described in Fig. 5 of the above embodiment. That is, in (1) shown in Fig. 14, a captured image (original image) IM is input from the imaging device 8. In (2) shown in Fig. 14, vertex candidates that can become the vertices of a rectangular shape are extracted from the input captured image IM. The vertex candidates are stored as a coordinate list, not as image data.

[0073] In (4) shown in FIG. 14 , a combination of vertex candidates that can become the vertices of a quadrangle when connected, i.e., a quadrangular pattern candidate that can become the four sides of a quadrangle, is derived. More specifically, a plurality of quadrangular pattern candidates that fit within the enclosing area 70A of the defined pattern 70 shown in FIG. 11A are derived. At this time, the plurality of quadrangular pattern candidates are selected and evaluated based on factors such as tilt and aspect ratio. In (5) shown in FIG. 14 , a plurality of quadrangular pattern candidates are selected and evaluated, and from the remaining plurality of quadrangular pattern candidates, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined. This allows the quadrangular target portion corresponding to the marker 60 to be detected. Note that (5) shown in FIG. 14 shows a state in which the captured image IM1 is overlaid.

[0074] In (1A) shown in FIG. 14, a reduced image RIMa1 is created at a scale of 1 / 2, and in (1B) shown in FIG. 14, a reduced image RIMb1 is created at a scale of 1 / 4. In (2A) shown in FIG. 14, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM1. In (4A) shown in FIG. 14, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4). In (5A) shown in FIG. 14, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from multiple quadrangular pattern candidates. In (5A) shown in FIG. 14, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0075] In (2B) shown in FIG. 14, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM1. In (4B) shown in FIG. 14, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4) and (4A). In (5B) shown in FIG. 14, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from among multiple quadrangular pattern candidates. In (5B) shown in FIG. 14, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0076] Next, the process flow for detecting a rectangular target portion from the captured image IM2 shown in FIG. 15 will be described. In (1) of FIG. 15, the captured image (original image) IM2 is input from the imaging device 8. In (2) of FIG. 15, vertex candidates that can become the vertices of a rectangular shape are extracted from the input captured image IM2. In (4) of FIG. 15, combinations of vertex candidates that can become the vertices of a rectangular shape when connected, i.e., rectangular pattern candidates that can become the four sides of a rectangular shape, are derived. More specifically, multiple rectangular pattern candidates that fit within the enclosing area 70A of the specified pattern 70 shown in FIG. 11A are derived. In (5) of FIG. 15, multiple rectangular pattern candidates are selected and evaluated, and from the remaining multiple rectangular pattern candidates, a rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60 is determined. 15, there is no rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60. As a result, the rectangular target portion corresponding to the marker 60 is not detected.

[0077] In (1A) shown in FIG. 15, a reduced image RIMa2 is created at a scale of 1 / 2, and in (1B) shown in FIG. 15, a reduced image RIMb2 is created at a scale of 1 / 4. In (2A) shown in FIG. 15, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM2. In (4A) shown in FIG. 15, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4). In (5A) shown in FIG. 15, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from multiple quadrangular pattern candidates. This allows the quadrangular target portion corresponding to the marker 60 to be detected.

[0078] In (2B) shown in FIG. 15, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM2. In (4B) shown in FIG. 15, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4) and (4A). In (5B) shown in FIG. 15, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from among multiple quadrangular pattern candidates. In (5B) shown in FIG. 15, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0079] Next, the process flow for detecting a rectangular target portion from the captured image IM3 shown in FIG. 16 will be described. In (1) of FIG. 16, the captured image (original image) IM3 is input from the imaging device 8. In (2) of FIG. 16, vertex candidates that can become the vertices of a rectangular shape are extracted from the input captured image IM3. In (4) of FIG. 16, combinations of vertex candidates that can become the vertices of a rectangular shape when connected, i.e., rectangular pattern candidates that can become the four sides of a rectangular shape, are derived. More specifically, multiple rectangular pattern candidates that fit within the enclosing area 70A of the specified pattern 70 shown in FIG. 11A are derived. In (5) of FIG. 16, the multiple rectangular pattern candidates are selected and evaluated, and from the remaining multiple rectangular pattern candidates, a rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60 is determined. 16, there is no rectangular pattern candidate that matches the rectangular pattern corresponding to the marker 60. As a result, the rectangular target portion corresponding to the marker 60 is not detected.

[0080] In (1A) shown in FIG. 16, a reduced image RIMa3 is created at a scale of 1 / 2, and in (1B) shown in FIG. 16, a reduced image RIMb3 is created at a scale of 1 / 4. In (2A) shown in FIG. 16, similar to (2), vertex candidates that can become the vertices of a quadrangular shape are extracted from the input captured image IM3. In (4A) shown in FIG. 16, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4). In (5A) shown in FIG. 16, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from multiple quadrangular pattern candidates. In (5B) shown in FIG. 16, there is no quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60. As a result, the quadrangular target portion corresponding to the marker 60 is not detected.

[0081] In (2B) shown in FIG. 16, similar to (2), vertex candidates that can become vertices of a quadrangular shape are extracted from the input captured image IM3. In (4B) shown in FIG. 16, similar to (4), quadrangular pattern candidates are derived. The quadrangular pattern candidates derived at this time differ from the quadrangular pattern candidates in (4) and (4A). In (5B) shown in FIG. 16, a quadrangular pattern candidate that matches the quadrangular pattern corresponding to the marker 60 is determined from among multiple quadrangular pattern candidates. This allows the quadrangular target portion corresponding to the marker 60 to be detected.

[0082] In the target portion detection device 41 according to the first modification, when detecting the quadrangular target portions contained in each of the original image IM and the reduced image RIM, combinations of vertex candidates that can form the vertices of a quadrangular pattern that falls within the range of the specified pattern 70 are derived. This makes it possible to reduce the number of comparisons of combinations of vertex candidates during the detection process compared to when detecting all quadrangular target portions contained in the original image IM and the reduced image RIM, and shorten the time required to detect the quadrangular target portions.

[0083] In the target portion detection device 41 according to the first modification, the degree of increase in processing time and data volume of detection target candidates that occurs when increasing the resolution of a captured image can be reduced to an order of proportionality, whereas in the past, the increase was on the order of more than two times the number of pixels in the captured image IM. In the first modification, for example, the number of pixels in the image of the marker 60 shown in FIG. 11(B) is limited to the range of the size of the rectangular target portion, which is sufficiently smaller than the input image size. In the target portion detection device 41 according to the first modification, the effect is greater the smaller the size of the image of the target on which the specified pattern 70 is scanned.

[0084] On the other hand, the creation of reduced images RIM increases the image to be scanned with the specified pattern 70, i.e., the number of pixels to be examined. In particular, when the size range of the rectangular target portion is reduced, a sufficient number of reduced images RIM is required to ensure that rectangular pattern candidates are detected at any scale. However, in this case, the increase in the number of pixels to be examined is limited to a few times. For example, when reducing by 1 / 2 scale (1 / 2, 1 / 4, 1 / 8), the number of pixels increases by 1 / 4, and considering the sum of the geometric series, the increase is limited to 1 (1 - 1 / 4) ≒ 1.33 times or less. Similarly, when reducing by 1 / 2 (1 / 2) scale, the increase is limited to 2 times or less. Considering the above, the increase in the number of pixels required to create reduced images RIM can be compensated for depending on the size range of the rectangular pattern candidates, and the effect can be enhanced by actively implementing both the generation of reduced images RIM and the reduction of the size range of the rectangular pattern.

[0085] Note that restricting the upper limit is effective in reducing processing time delays and increasing the amount of extracted data that accompanies an increase in the number of pixels. On the other hand, restricting the lower limit is effective in excluding small markers that, even if detected, do not guarantee sufficient accuracy in the information obtained. In particular, for images with many fine contours and fine shape patterns, restricting the lower limit can greatly contribute to resolving the problem.

[0086] In the target portion detection device 41 according to the first modification, the detection unit 45 derives combinations of vertex candidates that can form the vertices of a quadrangular pattern that fits within the enclosing area 70A of the first frame portion 71 and the second frame portion 72. This reduces the number of comparisons of combinations of vertex candidates that can form the vertices of a quadrangular pattern that fits within the range of the specified pattern 70.

[0087] (Variation 2) In the target portion detection device 41 according to Variation 2, the detection unit 45 identifies the direction of the vertex candidates extracted by the extraction unit 43 based on the light and dark patterns in the four segmented regions R1, R2, R3, and R4, and derives combinations of vertex candidates that can form a rectangular pattern based on information about the identified direction. Specific examples of the direction of the vertex candidates extracted by the extraction unit 43 are the upper-left vertex Pa, the upper-right vertex Pb, the lower-left vertex Pc, and the lower-right vertex Pd, as shown in FIG. 17A . The extraction unit 43 can identify whether the vertex candidate is the upper-left vertex Pa, the upper-right vertex Pb, the lower-left vertex Pc, or the lower-right vertex Pd based on the light and dark combination patterns of four pixels PX arranged in two rows and two columns, as shown in FIG. 8A .

[0088] As shown in FIG. 17B , the target portion detection device 41 according to the second modification (1) receives a captured image IM from the imaging device 8, and (2) extracts vertex candidates that can become vertices of a rectangular shape from the input captured image IM. At this time, the target portion detection device 41 according to the second modification (2) recognizes the direction of the vertex candidates. In FIG. 17B , the upper-left vertex Pa is indicated by a circle, the upper-right vertex Pb by an x, the lower-left vertex Pc by a triangle, and the lower-right vertex Pd by a square. The extraction unit 43 may separate the memory areas of the memory processing unit 41B that store the extracted vertex candidates for each light-dark combination pattern of four pixels PX consisting of two rows and two columns, i.e., the upper-left vertex Pa, the upper-right vertex Pb, the lower-left vertex Pc, and the lower-right vertex Pd.

[0089] The object portion detection device 41 according to the second modification (3) extracts only the combination of the vertex Pa in the upper left direction and the vertex Pc in the lower left direction, and the combination of the vertex Pb in the upper right direction and the vertex Pd in ​​the lower right direction, when detecting line segment candidates (up and down direction) that can become sides of a quadrangle from the extracted vertex candidates.Furthermore, the object portion detection device 41 according to the second modification (3) extracts only the combination of the vertex Pa in the upper left direction and the vertex Pb in the upper right direction, and the combination of the vertex Pc in the lower left direction and the vertex Pd in ​​the lower right direction, when detecting line segment candidates (up and down direction) that can become sides of a quadrangle from the extracted vertex candidates.

[0090] The object portion detection device 41 according to the second modification (4) extracts a combination of an upper-left vertex Pa, an upper-right vertex Pb, a lower-left vertex Pc, and a lower-right vertex Pd when detecting a quadrangular pattern that can become the four sides of a quadrangular shape from the detected line segment candidates. This detects a plurality of quadrangular pattern candidates. The object portion detection device 41 according to the second modification (5) determines, from the plurality of quadrangular pattern candidates, a quadrangular pattern candidate that matches a quadrangular pattern corresponding to a quadrangular object portion, thereby detecting the quadrangular object portion.

[0091] The target portion detection device 41 according to variant example 2 can reduce the number of combinations of vertex candidates that can form a rectangular pattern, thereby further reducing the time required to detect a rectangular target portion contained in an image.

[0092] (Other Modifications) In the target portion detection device 41 according to Modification 1, an example has been described in which both the original image IM and the reduced image RIM are targeted as images to be scanned with the prescribed pattern 70 shown in FIG. 11A , but it is also possible to target only at least one of the original image IM and the reduced image RIM. When scanning the prescribed pattern 70 only on the original image IM, the target portion detection device 41 does not need to create the reduced image RIM. In other words, the target portion detection device 41 according to this modification does not need to be provided with the creation unit 47.

[0093] In the above embodiment, an example was given in which the imaging device 8 is provided on the overhead traveling vehicle 6, but it may also be mounted on a traveling device such as an automated guided vehicle (AGV) or an autonomous mobile transport robot (AMR), or a fixed-point observation device, etc.

[0094] 4...track, 6...overhead traveling vehicle (traveling cart), 8...imaging device, 35...rear cover, 40...controller, 41...target portion detection device, 41A...input processing unit, 41B...storage processing unit, 41C...arithmetic processing unit, 41D...output processing unit, 41E...storage device (storage unit), 43...extraction unit, 45...detection unit, 47...creation unit, 60...marker, 61...black area, 63...white area, 70...specified pattern, 70A...enclosing area, 71...first frame portion, 72...second frame portion, 160...logo, A1...determination area, IM...original image (captured image), RIM...reduced image, R1, R2, R3, R4...division areas.

Claims

1. A target portion detection device that detects a rectangular target portion included in an image captured by an imaging device, comprising: a memory unit that stores a rectangular pattern corresponding to the target portion; an extraction unit that extracts a plurality of vertex candidates that can become vertices of the rectangular shape included in the captured image; and a detection unit that derives a combination of the vertex candidates that can form vertices of the rectangular pattern from the plurality of vertex candidates extracted by the extraction unit, and detects the rectangular target portion based on the derived combination of vertex candidates and the rectangular pattern.

2. The object portion detection device according to claim 1, wherein when a judgment area is an area consisting of four divided areas separated by two mutually perpendicular virtual straight lines, the extraction unit determines whether the judgment area is a vertex candidate based on the light and dark patterns in the four divided areas.

3. The object portion detection device according to claim 2, wherein said extraction unit detects said vertex candidates based on a combination pattern of light and dark in four pixels arranged in two rows and two columns.

4. A target portion detection device as described in claim 2 or 3, wherein the detection unit identifies the direction of the vertex candidates extracted by the extraction unit based on the light and dark patterns in the four divided areas, and derives combinations of the vertex candidates that can form the rectangular pattern based on information regarding the identified direction.

5. A target portion detection device as described in claim 1 or 2, wherein the detection unit derives, from the plurality of vertex candidates, a plurality of line segment candidates consisting of combinations of the vertex candidates that can form one side of the quadrangular pattern, from the derived plurality of line segment candidates, a plurality of connecting line segment candidates consisting of combinations of the line segment candidates that can form two adjacent sides of the quadrangular pattern, from the derived plurality of connecting line segment candidates, derives combinations of the connecting line segment candidates that can form four sides of the quadrangular pattern, and detects the quadrangular target portion based on the quadrangular pattern candidate consisting of the derived combinations of connecting line segment candidates and the quadrangular pattern stored in the memory unit.

6. A target portion detection device as described in claim 1 or 2, further comprising a creation unit that creates a reduced image by reducing the original image that is the captured image, and wherein the detection unit derives combinations of vertex candidates that can form vertices of the quadrangular pattern that fall within the range of a specified pattern that specifies lower and upper limits of size, from each of the original image and the reduced image.

7. A target portion detection device as described in claim 6, wherein the specified pattern is composed of a first frame portion that specifies the lower limit value and a second frame portion that is arranged to surround the first frame portion and that specifies the upper limit value, and the detection unit derives combinations of vertex candidates that can form vertices of the quadrangular pattern that fit within the enclosed area of ​​the first frame portion and the second frame portion.

8. The target portion detection device according to claim 1 or 2, wherein the detection unit detects the rectangular two-dimensional code.

9. A traveling vehicle comprising: a traveling unit that travels along a predetermined traveling route; the target portion detection device according to claim 1 or 2; and a controller that controls the traveling of the traveling unit based on information obtained from the rectangular target portion detected by the target portion detection device.

Images