Self-position estimation system and parking facility

Abstract

To provide a self-position estimation system that can estimate the self-position of an automobile even in the circumference of parking facilities having a movable part, and the parking facilities.SOLUTION: A self-position estimation system comprises an on-vehicle camera which is mounted on an automobile, and a processing device which extracts a feature point from an image captured by the on-vehicle camera and uses the extracted feature point to estimate the self-position of the automobile. A self-position estimation program of the processing device when recognizing an exclusion mark fitted to the parking facilities in the image extracts a feature point from the remaining region excluding a predetermined exclusion region from the image, and uses the extracted feature point to estimate the self-position of the automobile in the circumference of the parking facilities.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present application relates to a self-location estimation system and a parking facility. [Background technology]

[0002] Patent Document 1 listed below discloses a technology related to Visual SLAM (Simultaneous Localization and Mapping), which estimates the vehicle's position while creating an environmental map based on the camera's position and orientation, and environmental feature points on images captured at each position. Visual SLAM is effective when the vehicle travels autonomously in parking facilities where there is no accurate map. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6699761 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, the location of a vehicle can be estimated by storing a map containing three-dimensional position information of multiple feature points in advance, capturing images of the surrounding area with an onboard camera mounted on the vehicle, calculating the three-dimensional position information of the feature points from two captured frame images, and comparing the calculated three-dimensional position information of the feature points with the three-dimensional position information of the feature points in the stored map. However, if the image captured by the onboard camera includes parking equipment with movable parts such as doors, the appearance of the parking equipment changes depending on the situation, and the feature points that can be extracted from the image also change. As a result, there is a risk that the vehicle's self-location cannot be estimated around parking equipment with movable parts.

[0005] Therefore, an object of the present application is to provide a self-position estimation system and parking facility that can estimate the self-position of a vehicle even in the vicinity of parking facility having movable parts. [Means for solving the problem]

[0006] A self-location estimation system according to a first aspect of the present application comprises an on-board camera mounted on a vehicle, and a processing device that extracts feature points from an image captured by the on-board camera and estimates the vehicle's self-location using the extracted feature points. When the processing device recognizes an exclusion mark attached to a parking facility in the image, it extracts feature points from the remaining area of ​​the image excluding a specified exclusion area, and estimates the vehicle's self-location around the parking facility using the extracted feature points.

[0007] According to this configuration, by attaching the exclusion mark to an appropriate position on the parking facility, even if the parking facility is included in an image captured by an onboard camera, feature points can be extracted from the remaining area of ​​the image excluding the area around the movable parts of the parking facility. In this case, the feature points extracted by the processing device are not affected by the state of the parking facility (the state of the movable parts), so the vehicle's self-position can be estimated even around parking facility with movable parts.

[0008] Furthermore, in the above-described self-location estimation system, when the processing device recognizes a feature point mark attached to a parking facility in an image captured by the onboard camera, in addition to extracting feature points from the remaining area, the processing device may extract the feature point mark as a feature point and estimate the self-location of the vehicle around the parking facility using the extracted feature points.

[0009] With this configuration, even if the number of feature points that can be extracted decreases due to the smaller area of ​​the image from which feature points are extracted, the feature points can be compensated for by placing feature point marks on the parking facilities, thereby improving the accuracy of vehicle self-location estimation around the parking facilities.

[0010] In addition, a self-location estimation system according to a second aspect of the present application comprises a plurality of on-board cameras mounted on a vehicle, and a processing device that extracts feature points from images captured by the plurality of on-board cameras and estimates the self-location of the vehicle using the extracted feature points; when the processing device recognizes an exclusion mark attached to a parking facility in an image captured by some of the plurality of on-board cameras, it extracts feature points from images captured by the remaining on-board cameras other than the some of the plurality of on-board cameras, and estimates the self-location of the vehicle around the parking facility using the extracted feature points.

[0011] According to this configuration, by attaching the exclusion marks to appropriate positions on the parking facility, even if the parking facility is included in the images captured by some of the on-board cameras, feature points can be extracted from the images captured by the remaining on-board cameras. In this case, the feature points extracted by the processing device are not affected by the state of the parking facility (the state of the moving parts), so the vehicle's self-position can be estimated even around parking facilities with moving parts.

[0012] In addition, a self-location estimation system according to a third aspect of the present application comprises a plurality of on-board cameras mounted on a vehicle, and a processing device that extracts feature points from images captured by the plurality of on-board cameras and estimates the self-location of the vehicle using the extracted feature points, wherein the processing device determines the state of the parking facility, selects an exclusion camera from the plurality of on-board cameras based on the determined state of the parking facility, extracts feature points from images captured by the remaining on-board cameras of the plurality of on-board cameras other than the exclusion camera, and estimates the self-location of the vehicle around the parking facility using the extracted feature points.

[0013] With this configuration, feature points can be extracted from images captured by the onboard cameras other than the one capturing the parking facility, depending on the state of the parking facility. In this case, the feature points extracted by the processing device are not affected by the state of the parking facility (the state of the moving parts), so the vehicle's own position can be estimated even around parking facilities with moving parts.

[0014] In addition, a self-location estimation system according to a fourth aspect of the present application includes an onboard camera mounted on a vehicle, a storage device that stores a map including three-dimensional position information of a plurality of feature points for each state of the parking facility, and a processing device that determines the state of the parking facility and compares the three-dimensional position information of feature points calculated from two frame images captured by the onboard camera with the three-dimensional position information of feature points in the map corresponding to the determined state of the parking facility among the maps stored in the storage device, thereby estimating the self-location of the vehicle around the parking facility.

[0015] In this configuration, the vehicle's own position around the parking facility is estimated by comparing the three-dimensional position information of the feature points in the map corresponding to the state of the parking facility with the three-dimensional position information of the feature points calculated from two frame images captured by the on-board camera. Therefore, the vehicle's own position can be estimated regardless of the state of the parking facility (the state of the moving parts). Therefore, the vehicle's own position can be estimated even around parking facilities with moving parts.

[0016] A parking facility according to a first aspect of the present application is a parking facility for vehicles entering and leaving the facility, and is equipped with an on-board camera and a processing device that extracts feature points from images captured by the on-board camera and estimates the vehicle's own position using the extracted feature points. The parking facility is equipped with at least one exclusion mark, which is provided on a fixed part of at least the exterior of the parking facility and is a mark that the processing device can recognize from the image, and when the processing device recognizes the mark, it extracts feature points from the remaining area of ​​the image excluding a specified exclusion area and estimates the vehicle's own position around the parking facility using the extracted feature points.

[0017] With this configuration, even if a parking facility is included in an image captured by an onboard camera, feature points can be extracted from the remaining area of ​​the image, excluding the area around the movable parts of the parking facility. In this case, the feature points extracted by the processing device are not affected by the state of the parking facility, so the vehicle's self-position can be estimated even around the parking facility.

[0018] In addition, the parking facility may be provided with a feature point mark that is provided on a fixed portion of the outer surface of the parking facility and that can be recognized by the processing device from the image, and when the processing device recognizes the mark, in addition to extracting feature points from the remaining area, the mark may be extracted as a feature point and the extracted feature point may be used to estimate the vehicle's own position around the parking facility.

[0019] With this configuration, even if the number of feature points that can be extracted decreases due to the smaller area of ​​the image from which feature points are extracted, the feature points can be compensated for by placing feature point marks on the parking facilities, thereby improving the accuracy of vehicle self-location estimation around the parking facilities.

[0020] In addition, in the above parking facility, the at least one exclusion mark may be a plurality of exclusion marks, and the exclusion area may be an area surrounded by lines connecting adjacent exclusion marks among the plurality of exclusion marks.

[0021] With this configuration, the processing device can easily identify the exclusion area.

[0022] In addition, in the above parking facility, the exclusion area may be an area of ​​a predetermined shape and a predetermined area, with the exclusion mark as its base point.

[0023] This configuration allows the number of exclusion marks to be placed on parking facilities to be reduced.

[0024] In addition, the above parking facility may be equipped with a movable part that can be imaged by the on-board camera when the vehicle drives around the parking facility, a communication device that can communicate wirelessly with the vehicle, and a control device that controls the operation of the movable part, and the control device may be configured to operate the movable part and / or notify the vehicle via the communication device that the movable part has been operated.

[0025] According to this configuration, the processing device can quickly determine the state of the parking facility (the state of the movable part). [Effects of the Invention]

[0026] According to the present application, the vehicle's own position can be estimated even in the vicinity of parking facilities that have movable parts. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a block diagram of a self-localization system. [Figure 2] FIG. 2 is a diagram showing an image of a parking facility with its doors closed and its feature points. [Figure 3] FIG. 3 shows an image of a parking facility with its door open and its feature points. [Figure 4] FIG. 4 is a flow diagram of the self-position estimation program according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating the self-position estimation program according to the first embodiment. [Figure 6] FIG. 6 is a block diagram of a parking facility. [Figure 7] FIG. 7 is a flowchart of the self-position estimation program according to the second embodiment. [Figure 8] FIG. 8 is a flow diagram of a self-position estimation program according to the third embodiment. [Figure 9] FIG. 9 is a flowchart of the self-position estimation program according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0028] (First embodiment) The self-position estimation system according to the first embodiment will be described below.

[0029] <Overall structure> First, the overall configuration of the self-location estimation system will be described. Fig. 1 is a block diagram of a self-location estimation system 10. As shown in Fig. 1, the self-location estimation system 10 includes a plurality of vehicle-mounted cameras 11-14, a storage device 15, and a processing device 16. Each component will be described below in order.

[0030] The on-board cameras 11-14 are cameras mounted on the automobile 20. The on-board cameras 11-14 include a front on-board camera 11 that captures an image in front of the automobile 20, a right on-board camera 12 that captures an image to the right of the automobile 20, a left on-board camera 13 that captures an image to the left of the automobile 20, and a rear on-board camera 14 that captures an image behind the automobile 20. However, the number and imaging directions of the on-board cameras 11-14 provided in the self-location estimation system 10 are not limited to those described above.

[0031] The storage device 15 is a device that stores various information. The storage device 15 of this embodiment is mounted on the automobile 20. However, the storage device 15 may be located on a network. The storage device 15 is, for example, a non-volatile memory. The storage device 15 of this embodiment stores a map including three-dimensional position information of a plurality of feature points. The three-dimensional position information of the plurality of feature points included in the map can be generated using feature points extracted from two frame images captured at multiple locations. The map stored in the storage device 15 also includes three-dimensional position information of the feature points around the parking facility.

[0032] Here, a brief explanation of feature points will be given. Fig. 2(a) is an image of the parking facility 30 with the door 31 closed. Fig. 2(b) is a diagram showing feature points extracted from the image of Fig. 2(a). The black circles in Fig. 2(b) are feature points (the same applies to Figs. 3(b), 5(b), and 5(c)). As shown in Fig. 2(b), in this embodiment, the image is subjected to edge processing or the like to extract corners and vertices of figures appearing in the image, and these are used as feature points. If the same feature point can be extracted from two frame images, the three-dimensional position information of that feature point can be calculated using the principles of triangulation.

[0033] The processing device 16 is a device that performs various types of arithmetic processing. The processing device 16 of this embodiment is mounted on the automobile 20. However, the processing device 16 may also be located on a network. The processing device 16 has, for example, a processor, a volatile memory, a non-volatile memory, an I / O interface, and the like. Various programs including a self-location estimation program are stored in the non-volatile memory of the processing device 16, and the processor performs arithmetic processing using the volatile memory based on each program. The self-location estimation program will be described later.

[0034] The processing device 16 is also communicatively connected to the vehicle-mounted cameras 11-14, and can acquire images captured by each vehicle-mounted camera 11-14 based on signals received from the vehicle-mounted cameras 11-14. The processing device 16 is also communicatively connected to the storage device 15, and can acquire three-dimensional position information of feature points included in the map stored in the storage device 15 based on signals received from the storage device 15.

[0035] <Outline of self-location estimation method> Next, an overview of the self-location estimation method will be described. The self-location estimation system 10 according to this embodiment can estimate the self-location of the automobile 20. The self-location of the automobile 20 is estimated using feature points. As described above, the storage device 15 stores a map including three-dimensional position information of a plurality of feature points. Therefore, by comparing the three-dimensional position information of the feature points calculated from two frame images captured by the on-board camera 11-14 with the three-dimensional position information of the feature points on the map stored in the storage device 15, the position of the automobile 20 equipped with the on-board camera 11-14 can be estimated.

[0036] FIG. 3(a) is an image of the parking facility 30 with the door 31 open, and FIG. 3(b) is a diagram showing feature points extracted from the image of FIG. 3(a). As can be seen by comparing FIG. 2(b) and FIG. 3(b), when the door 31 is open, a larger number of feature points are extracted within the door frame 32 than when the door 31 is closed. Thus, the feature points extracted from an image of the parking facility 30 having a movable part such as the door 31 vary depending on the state of the parking facility 30 (here, whether the door 31 is closed or not). Therefore, for example, if the storage device 15 stores a map generated from feature points with the door 31 closed, while the on-board camera 11-14 captures an image of the parking facility 30 with the door 31 open, it may not be possible to estimate the self-position of the automobile 20.

[0037] Therefore, in this embodiment, when the parking facility 30 is included in the image captured by the in-vehicle cameras 11-14, feature points are extracted from an area excluding movable parts such as the door 31, so that the state of the movable parts does not affect the estimation of the self-position of the automobile 20. Furthermore, even if the number of feature points decreases as a result of the area from which feature points are extracted being reduced, the feature points are compensated for, thereby improving the accuracy of the estimation of the self-position of the automobile 20.

[0038] <Details of self-location estimation method> Next, the self-location estimation method will be described in detail. Below, the self-location estimation method will be described by explaining the flow of the self-location estimation program executed by the processing device 16. FIG. 4 is a flowchart of the self-location estimation program. Also, FIG. 5 is a diagram explaining the self-location estimation program. In this embodiment, a case will be described in which the self-location of the automobile 20 in the vicinity of a parking facility 30 having a door 31 that is a movable part is estimated. However, the movable part may be a part other than the door 31 of the parking facility 30.

[0039] Here, the parking facility 30 of this embodiment will be described. The parking facility 30 of this embodiment is a facility through which the automobile 20 enters and leaves, and is equipped with a door 31, which is a movable part (see FIG. 2(a)). The door 31 is exposed to the outside, and can be photographed by the on-board cameras 11-14 when the automobile 20 travels around the parking facility 30. Furthermore, as shown in FIG. 5(a), the parking facility 30 is equipped with an exclusion mark 33 and a feature point mark 34. The double square in FIG. 5(a) is the exclusion mark 33, and the double circle is the feature point mark 34. In this embodiment, the exclusion mark 33 and the feature point mark 34 are attached to the periphery of the door frame 32, which is a fixed part on the outer surface of the parking facility 30.

[0040] Fig. 6 is a block diagram of the parking facility 30. As shown in Fig. 6, the parking facility 30 includes a communication device 35 capable of wireless communication with the automobile 20, and a control device 36 that controls the operation of the door 31. The control device 36 can transmit (notify) various information to the automobile 20 via the communication device 35. For example, the control device 36 can operate a movable part such as the door 31 and / or notify the automobile 20 via the communication device 35 that the movable part has been operated.

[0041] As shown in Fig. 4, when the self-location estimation program is started, the processing device 16 acquires two adjacent frame images from each of the vehicle-mounted cameras 11-14 (step S1). Note that the vehicle-mounted cameras 11-14 continuously capture images at a predetermined frame rate. The "two adjacent frame images" mentioned above refer to two frame images captured at adjacent times on the time axis.

[0042] Next, the processing device 16 determines whether or not the two frame images acquired in step S1 include the exclusion mark 33 (step S2).

[0043] In step S2, if it is determined that the two frame images do not contain the exclusion mark 33 (NO in step S2), that is, if the processing device 16 cannot recognize the exclusion mark 33 in the image, the processing device 16 extracts feature points from the entire image for each of the two frame images acquired in step S1 (step S3). The method for extracting feature points from an image is as described above. After step S3, the process proceeds to step S6.

[0044] On the other hand, if it is determined in step S2 that the two frame images contain an exclusion mark 33 (YES in step S2), that is, if the processing device 16 recognizes an exclusion mark 33 in the image, the processing device 16 identifies an exclusion area (step S4). In this embodiment, the area surrounded by lines connecting adjacent exclusion marks 33 is identified as the exclusion area. In the example shown in FIG. 5(b), the area including the inside of the door frame 32, indicated by diagonal lines, is identified as the exclusion area. Note that the lines connecting adjacent exclusion marks 33 may be straight or curved.

[0045] After step S4, the processing device 16 extracts feature points from the remaining areas of each of the two frame images acquired in step S1, excluding the exclusion area identified in step S4 (step S5). In the example shown in FIG. 5(b), feature points are extracted from areas other than the area around the door frame 32. This makes it possible to extract the same feature points from images of the same parking facility 30, regardless of whether the door 31 of the parking facility 30 is open or not, that is, regardless of the state of the parking facility 30. After step S5, the process proceeds to step S6.

[0046] In step S6, the processing device 16 determines whether or not the two frame images acquired in step S1 include the feature point mark 34. If it is determined in step S6 that the feature point mark 34 is not included in the two frame images (NO in step S6), that is, if the processing device 16 cannot recognize the feature point mark 34 in the images, the process proceeds to step S8.

[0047] On the other hand, if it is determined in step S6 that the feature point marks 34 are included in the two frame images (YES in step S6), that is, if the processing device 16 recognizes the feature point marks 34 in the images, the processing device 16 extracts the feature point marks 34 as feature points (step S7). Note that in step S7, the feature point marks 34 are extracted as feature points regardless of whether or not the feature point marks 34 are within the aforementioned exclusion area. Even if the number of feature points that can be extracted from the images is reduced by going through step S5, the feature points can be supplemented by going through step S7. After going through step S7, the process proceeds to step S8.

[0048] In step S8, the processing device 16 associates identical feature points from the plurality of feature points extracted from each of the two frame images, and calculates three-dimensional position information for each feature point.

[0049] In step S9, the processing device 16 compares the three-dimensional position information of the feature points in the map stored in the storage device 15 with the three-dimensional position information of the feature points calculated in step S8, and estimates the self-position of the automobile 20. After step S9, the process returns to step S1 and each step is repeated.

[0050] As described above, the storage device 15 stores a map including three-dimensional position information of feature points, but the method of acquiring feature points for generating the map is not limited. In this embodiment, the three-dimensional position information of feature points around the parking facility 30 included in the map stored in the storage device 15 is generated by capturing an image of the area around the parking facility 30 to which the exclusion mark 33 and feature point mark 34 have been attached in advance, and using feature points extracted from the resulting image through processing similar to steps S1 to S7 described above. However, for example, the area around the parking facility 30 without the exclusion mark 33 and feature point mark 34 attached may be captured, and a map generated using feature points extracted from the resulting image may be stored in the storage device 15.

[0051] According to the self-position estimation system 10 of this embodiment, feature points can be extracted from the remaining area of ​​the image captured by the in-vehicle cameras 11-14, excluding the area around the movable parts of the parking facility 30. In this case, the feature points extracted by the processing device 16 are not affected by the state of the parking facility 30 (whether the door 31 is open or closed). Therefore, the self-position of the automobile 20 can be estimated even around the parking facility 30 that has movable parts.

[0052] In step S4 of this embodiment, the area surrounded by the lines connecting adjacent exclusion marks 33 is identified as the exclusion area, but the method for identifying the exclusion area is not limited to this. For example, a range of a predetermined area and a predetermined shape starting from the exclusion mark 33 may be identified as the exclusion area. Also, for example, the exclusion mark 33 may be a two-dimensional code containing information about the exclusion area, and the exclusion area may be identified based on that information about the exclusion area. Furthermore, multiple types of exclusion marks 33 may be attached to the parking facility 30, and the processing device 16 may identify multiple exclusion areas according to the types of exclusion marks 33 in step S4.

[0053] (Second embodiment) Next, a self-location estimation system 10 according to a second embodiment will be described. The overall configuration of the self-location estimation system 10 according to the second embodiment is the same as the overall configuration of the self-location estimation system 10 according to the first embodiment. However, the self-location estimation program according to the second embodiment is different from the self-location estimation program according to the first embodiment. The self-location estimation program according to the second embodiment will be described below.

[0054] 7 is a flow diagram of a self-location estimation program of the second embodiment. In the first embodiment, when an exclusion mark 33 is included in an image captured by the vehicle-mounted camera 11-14, the processing device 16 extracts feature points from the remaining area excluding the exclusion area. In contrast, in the second embodiment, images captured by the vehicle-mounted camera 11-14 that include the exclusion mark 33 are not used to extract feature points. This will be explained in detail below.

[0055] As shown in FIG. 7, when the self-position estimation program of the second embodiment is started, the processing device 16 acquires two adjacent frame images from each of the vehicle-mounted cameras 11-14 (step S11).

[0056] Next, the processing device 16 determines whether or not the exclusion mark 33 is included in the two frame images acquired in step S11 for each of the vehicle-mounted cameras 11-14 (step S12).

[0057] In step S12, if it is determined that the exclusion mark 33 is not included in the two frame images (NO in step S12), that is, if the processing device 16 cannot recognize the exclusion mark 33 in the images, the processing device 16 extracts feature points from the images captured by all of the in-vehicle cameras 11-14 acquired in step S11 (step S13). After step S13, the process proceeds to step S16.

[0058] On the other hand, if it is determined in step S12 that the exclusion mark 33 is included in the two frame images (YES in step S12), that is, if the processing device 16 recognizes the exclusion mark 33 in the images, the processing device 16 identifies the exclusion camera (step S14). In this embodiment, the camera among the vehicle-mounted cameras 11-14 that captured the image including the exclusion mark 33 is identified as the exclusion camera. For example, if the exclusion mark 33 is included in the image captured by the front vehicle-mounted camera 11, the processing device 16 identifies the front vehicle-mounted camera 11 as the exclusion camera.

[0059] After step S14, the processing device 16 extracts feature points from images captured by the remaining in-vehicle cameras 11-14 other than the excluded camera (step S15). For example, if the image captured by the front in-vehicle camera 11 includes the exclusion mark 33, feature points are extracted from images captured by the remaining in-vehicle cameras 11-14 other than the front in-vehicle camera 11, that is, the right in-vehicle camera 12, the left in-vehicle camera 13, and the rear in-vehicle camera 14. After step S15, the processing device 16 proceeds to step S16.

[0060] In step S16, the processing device 16 matches identical feature points from the multiple feature points extracted from each of the two frame images of the right vehicle-mounted camera 12, and calculates three-dimensional position information for each feature point. Similarly, the processing device 16 matches identical feature points from the multiple feature points extracted from each of the two frame images of the left vehicle-mounted camera 13, and calculates three-dimensional position information for each feature point. The processing device 16 also matches identical feature points from the multiple feature points extracted from each of the two frame images of the rear vehicle-mounted camera 14, and calculates three-dimensional position information for each feature point.

[0061] In step S17, the processing device 16 compares the three-dimensional position information of the feature points in the map stored in the storage device 15 with the three-dimensional position information of the feature points calculated in step S16, and estimates the self-position of the automobile 20. After step S17, the process returns to step S11 and each step is repeated.

[0062] According to the self-position estimation system 10 of this embodiment, feature points can be extracted from images captured by the remaining on-board cameras 11-14 other than the on-board cameras capturing images of the area around the movable parts of the parking facility 30. In this case, the feature points extracted by the processing device 16 are not affected by the state of the parking facility 30 (whether the door 31 is open or closed). Therefore, the self-position of the automobile 20 can be estimated even in the area around the parking facility 30 that has movable parts.

[0063] (Third embodiment) Next, a self-location estimation system 10 according to a third embodiment will be described. The overall configuration of the self-location estimation system 10 according to the third embodiment is the same as the overall configuration of the self-location estimation system 10 according to the first embodiment. However, the self-location estimation program according to the third embodiment is different from the self-location estimation program according to the first embodiment. The self-location estimation program according to the third embodiment will be described below.

[0064] 8 is a flow diagram of a self-location estimation program of the third embodiment. In the third embodiment, as in the second embodiment, excluded cameras are identified, and feature points are extracted from images captured by vehicle-mounted cameras other than the excluded cameras. However, in the third embodiment, excluded cameras are identified based on the state of the parking facility 30 without using the exclusion marks 33. This will be explained in detail below.

[0065] As shown in FIG. 8, when the self-position estimation program of the third embodiment is started, the processing device 16 acquires two adjacent frame images from each of the vehicle-mounted cameras 11-14 (step S21).

[0066] Next, the processing device 16 determines the status of the parking facility 30 (step S22). In this embodiment, the processing device 16 determines whether the door 31 of the parking facility 30 is open or closed. As described above, in the parking facility 30 of this embodiment, the control device 36 can operate the door 31 and / or notify the automobile 20 via the communication device 35 that the door 31 has been operated. Therefore, the processing device 16 of this embodiment determines the status of the parking facility 30 based on a signal transmitted from the parking facility 30 to the automobile 20. However, the method for determining the status of the parking facility 30 is not limited. For example, the processing device 16 may determine the status of the parking facility 30 based on an image captured of the parking facility 30.

[0067] Next, the processing device 16 identifies excluded cameras based on the state of the parking facility 30 determined in step S22 (step S23). In this embodiment, among the on-board cameras 11-14, cameras whose captured images are expected to include moving parts of the parking facility 30 are identified as excluded cameras. Excluded cameras may also be identified based on the orientation of the automobile 20. Predetermined cameras may also be identified as excluded cameras. Note that if it is expected that moving parts of the parking facility 30 will not be included in images captured by any of the on-board cameras 11-14, no excluded cameras are identified.

[0068] Next, the processing device 16 extracts feature points from images captured by the remaining on-board cameras 11-14 other than the excluded camera (step S24). For example, if the front on-board camera 11 is identified as the excluded camera in step S22, the processing device 16 extracts feature points from images captured by the remaining on-board cameras 11-14 other than the front on-board camera 11, that is, the right on-board camera 12, the left on-board camera 13, and the rear on-board camera 14.

[0069] Next, in step S25, the processing device 16 matches identical feature points from the multiple feature points extracted from each of the two frame images of the right vehicle-mounted camera 12, and calculates three-dimensional position information for each feature point. Similarly, the processing device 16 matches identical feature points from the multiple feature points extracted from each of the two frame images of the left vehicle-mounted camera 13, and calculates three-dimensional position information for each feature point. The processing device 16 also matches identical feature points from the multiple feature points extracted from each of the two frame images of the rear vehicle-mounted camera 14, and calculates three-dimensional position information for each feature point.

[0070] Next, the processing device 16 compares the three-dimensional position information of the feature points in the map stored in the storage device 15 with the three-dimensional position information of the feature points calculated in step S25, and estimates the self-position of the automobile 20 (step S26). After step S26, the process returns to step S21 and repeats each step.

[0071] According to the self-location estimation system 10 of this embodiment, feature points can be extracted from images captured by the remaining on-board cameras 11-14 other than the on-board cameras expected to capture images of the area around the movable parts of the parking facility 30. In this case, the feature points extracted by the processing device 16 are not affected by the state of the parking facility 30 (whether the door 31 is open or closed). Therefore, the self-location of the automobile 20 can be estimated even in the area around the parking facility 30 that has movable parts.

[0072] (Fourth embodiment) Next, a fourth embodiment will be described. The overall configuration of the self-location estimation system 10 according to the fourth embodiment is the same as the overall configuration of the self-location estimation system 10 according to the first embodiment. However, the self-location estimation program according to the fourth embodiment is different from the self-location estimation program according to the first embodiment. The self-location estimation program according to the fourth embodiment will be described below.

[0073] 9 is a flow diagram of a self-location estimation program of the fourth embodiment. In the fourth embodiment, the state of the parking facility 30 is determined, as in the third embodiment. However, in the fourth embodiment, excluded cameras are not identified based on the state of the parking facility 30, but a map according to the state of the parking facility 30 is acquired from the storage device 15. This will be explained in detail below.

[0074] In this embodiment, when the self-location estimation program is executed, the storage device 15 is assumed to have stored in advance a map including three-dimensional position information of multiple feature points for each state of the parking facility 30. Specifically, the storage device 15 is assumed to have stored therein both a map for when the door 31 of the parking facility 30 is closed and a map for when the door 31 of the parking facility 30 is open.

[0075] As shown in FIG. 9, when the self-position estimation program of the fourth embodiment is started, the processing device 16 acquires two adjacent frame images from each of the vehicle-mounted cameras 11-14 (step S31).

[0076] Next, the processing device 16 extracts feature points from the image acquired in step S31 (step S32).

[0077] Next, the processing device 16 determines the state of the parking facility 30 (step S33). Specifically, the processing device 16 determines whether the door 31 of the parking facility 30 is open or closed.

[0078] Next, the processing device 16 acquires a map corresponding to the state of the parking facility 30 determined in step S33 from among the maps stored in the storage device 15 (step S34). For example, when the processing device 16 determines in step S33 that the door 31 of the parking facility 30 is open, it acquires a map of the feature points stored in the storage device 15 when the door 31 of the parking facility 30 is open.

[0079] Next, in step S35, the processing device 16 matches identical feature points from the multiple feature points extracted from each of the two frame images of the front vehicle-mounted camera 11, and calculates three-dimensional position information of each feature point. The processing device 16 similarly calculates three-dimensional position information of feature points for the images of the right vehicle-mounted camera 12, the left vehicle-mounted camera 13, and the rear vehicle-mounted camera 14.

[0080] Next, the processing device 16 compares the three-dimensional position information of the feature points in the map acquired in step S34 with the three-dimensional position information of the feature points calculated in step S35, and estimates the self-position of the automobile 20 (step S36). After step S36, the process returns to step S31 and repeats each step.

[0081] According to the self-location estimation system 10 of this embodiment, a map corresponding to the state of the parking facility 30 is acquired from the storage device 15, and therefore the self-location of the automobile 20 can be estimated regardless of the state of the parking facility 30 (whether the door 31 is open or closed). Therefore, the self-location of the automobile 20 can be estimated even in the vicinity of the parking facility 30 that has movable parts. [Explanation of symbols]

[0082] 10 Self-location estimation system 11. Front-mounted camera 12 Right-side vehicle camera 13 Left-side vehicle camera 14 Rear-mounted camera 15 Storage device 16 Processing equipment 20. Automobiles 30 Parking facilities 31 Door (movable part) 32 Door Frame 33 Exclusion Mark 34 Minutiae marks 35 Communication equipment 36 Control device

Claims

1. An in-vehicle camera installed in a vehicle, a processing device that extracts feature points from the image captured by the on-board camera and estimates the vehicle's own position using the extracted feature points; A self-position estimation system in which, when the processing device recognizes an exclusion mark attached to a parking facility in the image, it extracts feature points from the remaining area of ​​the image excluding a specified exclusion area, and estimates the vehicle's self-position around the parking facility using the extracted feature points.

2. 2. The self-location estimation system according to claim 1, wherein, when the processing device recognizes a feature point mark attached to a parking facility in the image captured by the onboard camera, in addition to extracting feature points from the remaining area, the processing device extracts the feature point mark as a feature point and estimates the self-location of the vehicle in the vicinity of the parking facility using the extracted feature points.

3. A plurality of in-vehicle cameras mounted on a vehicle; a processing device that extracts feature points from images captured by the plurality of vehicle-mounted cameras and estimates a self-position of the vehicle using the extracted feature points; When the processing device recognizes an exclusion mark attached to a parking facility in an image taken by one of the multiple onboard cameras, the processing device extracts feature points from the images taken by the remaining onboard cameras other than the one of the multiple onboard cameras, and estimates the vehicle's own position around the parking facility using the extracted feature points.

4. A plurality of in-vehicle cameras mounted on a vehicle; a processing device that extracts feature points from images captured by the plurality of vehicle-mounted cameras and estimates a self-position of the vehicle using the extracted feature points; The processing device determines the state of the parking facility, selects an exclusion camera from the plurality of on-board cameras based on the determined state of the parking facility, extracts feature points from images taken by the remaining on-board cameras other than the exclusion camera among the plurality of on-board cameras, and estimates the vehicle's own position in the vicinity of the parking facility using the extracted feature points.

5. An in-vehicle camera installed in a vehicle, a storage device that stores a map including three-dimensional position information of a plurality of feature points for each state of the parking facility; a processing device that determines the state of the parking facility, and compares three-dimensional position information of feature points in a map stored in the storage device that corresponds to the determined state of the parking facility with three-dimensional position information of feature points calculated from two frame images captured by the on-board camera, thereby estimating the vehicle's own position in the vicinity of the parking facility.

6. A parking facility for an automobile to enter and exit the parking facility, the parking facility including an on-board camera and a processing device that extracts feature points from an image captured by the on-board camera and estimates the automobile's own position using the extracted feature points, A parking facility comprising at least one exclusion mark provided on at least an immovable portion of the exterior of the parking facility, the exclusion mark being recognizable from the image by the processing device, and when the processing device recognizes the mark, extracting feature points from the remaining area of ​​the image excluding a specified exclusion area, and using the extracted feature points to estimate the vehicle's own position around the parking facility.

7. The parking facility according to claim 6, further comprising a feature point mark provided on a fixed portion of the outer surface of the parking facility, the feature point mark being recognizable from the image by the processing device, and when the processing device recognizes the mark, in addition to extracting feature points from the remaining area, the mark is extracted as a feature point and the extracted feature points are used to estimate the vehicle's own position around the parking facility.

8. the at least one exclusion mark is a plurality of exclusion marks; The parking facility according to claim 6 or 7, wherein the exclusion area is an area surrounded by lines connecting adjacent exclusion marks among the plurality of exclusion marks.

9. 8. The parking facility according to claim 6, wherein the exclusion area is an area of ​​a predetermined shape and a predetermined area, the exclusion area being based on the exclusion mark.

10. a movable part that can capture images with the on-board camera when the automobile travels around the parking facility; a communication device capable of wirelessly communicating with the automobile; a control device for controlling the operation of the movable part, The parking facility according to claim 6 or 7, wherein the control device is configured to operate the movable part and / or notify the vehicle via the communication device that the movable part has been operated.

Images