Vehicle position detection device

Abstract

To provide a vehicle position detection device capable of easily and correctly estimating a position of a vehicle body (vehicle) from image information and correctly detecting the position of the vehicle on a world coordinate system.SOLUTION: A vehicle position detection device includes an acquisition section, an estimation section, and a processing section. The acquisition section acquires image information from an imaging section for imaging a detection object area. The estimation section estimates a correspondence position of a vehicle bottom surface of an estimation vehicle corresponding to a position of a reference point based on comparison between a learnt model obtained as a result of learning a relation between a learning vehicle included in the image information and at least one reference point indicating a reference position included in a vehicle bottom surface of the learning vehicle, and the estimation vehicle included in the image information acquired by the acquisition section. The processing section converts the correspondence position into a world coordinate system so as to detect the position of the estimation vehicle on the world coordinate system.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle position detection device. [Background technology]

[0002] Conventionally, a technique has been known in which a vehicle (car) captured in an image captured by a fixed camera or the like is detected using a technique such as machine learning, and the position of the detected vehicle is projected onto a world coordinate system. In this technique, first, an area in the image that includes the characteristics of the vehicle (for example, shape, etc.) is identified using a machine learning technique, and a rectangular "frame" of a size corresponding to the area is drawn. Then, the coordinates of the midpoint of, for example, the bottom side of the frame are considered to be the location of the vehicle, and the coordinates are projected onto the world coordinate system to detect the location of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-26281 Summary of the Invention [Problem to be solved by the invention]

[0004] However, because a frame is drawn around the vehicle body in the image, it is often the case that the coordinates assumed to represent the vehicle body's location do not accurately indicate the actual location of the vehicle body. For example, depending on the vehicle body's orientation (e.g., facing diagonally), the midpoint of the base may be located on the road surface, not on the vehicle body. Therefore, when projected onto the world coordinate system, there is a problem in that an error occurs between the actual vehicle body position and the displayed (recognized) position.

[0005] Therefore, one of the objectives of the present disclosure is to provide a vehicle position detection device that can more easily and accurately estimate the position of a vehicle body (vehicle) from image information and accurately detect the position of the vehicle on a world coordinate system. [Means for solving the problem]

[0006] A vehicle position detection device as an example of the present disclosure includes an acquisition unit that acquires image information from an imaging unit that images a detection target area, an estimation unit that estimates a corresponding position on the vehicle bottom of the estimation vehicle that corresponds to the position of the reference point based on a comparison between a trained model that is a result of learning a relationship between a training vehicle included in the image information and at least one reference point that indicates a reference position included on the vehicle bottom of the training vehicle and an estimation vehicle included in the image information acquired by the acquisition unit, and a processing unit that converts the corresponding position into a world coordinate system and detects the position of the estimation vehicle on the world coordinate system. The reference point is at least one of a ground contact point of a wheel of the training vehicle, a first projection point when a part of the front end of the body of the training vehicle is projected onto a road surface, and a second projection point when a part of the rear end of the body of the training vehicle is projected onto a road surface. According to this configuration, for example, the trained model is created including at least one reference point indicating a reference position included in the underside of the training vehicle, and the vehicle position is estimated based on the reference point. As a result, the accuracy of the vehicle position estimated from the image information is improved, enabling more accurate detection of the position in the world coordinate system. Furthermore, for example, regardless of the direction in which the posture of the vehicle captured in the image information is facing, the corresponding position of the vehicle bottom of the vehicle to be estimated can be estimated using the reference points. For example, even if the vehicle is facing straight sideways, straight ahead, straight behind, etc. in the image information, the corresponding position of the vehicle bottom of the vehicle to be estimated can be estimated. As a result, it is possible to detect the position more accurately on the world coordinate system.

[0008] Furthermore, in the above-described vehicle body position detection device, the first projection point may be, for example, a projection point at approximately the center of the front end of the vehicle body of the training vehicle, and the second projection point may be a projection point at approximately the center of the rear end of the vehicle body of the training vehicle. This configuration, for example, makes it easier to identify the reference point. As a result, when creating a trained model or when performing estimation using the trained model, the accuracy of setting the reference point and the accuracy of recognizing the corresponding position of the reference point can be improved.

[0009] Furthermore, the estimation unit of the vehicle body position detection device may perform estimation when, for example, it detects the entry of an estimation vehicle into the detection target area. With this configuration, for example, it is possible to perform accurate vehicle position detection processing at an appropriate time.

[0010] Furthermore, the estimation unit of the vehicle body position detection device may perform estimation, for example, when it receives a control start signal from a parking control device that manages a parking lot that includes the detection target area. With this configuration, for example, the manager of the parking lot can manage whether or not to perform the position estimation process for the estimating vehicle as needed, which can contribute to efficient operation of the parking lot (improved utilization efficiency).

[0011] The vehicle body position detection device may further include a display unit that displays the reference position. This configuration makes it easier to visually recognize, for example, the posture of the vehicle and the positional relationship between the vehicle and surrounding obstacles based on the reference position. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is an exemplary schematic diagram showing an automatic valet parking system that can use a vehicle position detection device according to an embodiment. [Figure 2] FIG. 2 is an exemplary schematic block diagram illustrating the configuration of a vehicle position detection device according to an embodiment. [Figure 3] FIG. 3 is an exemplary schematic top view illustrating positions of reference points in machine learning used in the vehicle position detection device according to the embodiment. [Figure 4] FIG. 4 is an exemplary and schematic perspective view showing the positions of reference points set during learning on a vehicle included in image information, or specific points detected during estimation, in machine learning used in a vehicle position detection device according to an embodiment. [Figure 5] FIG. 5 is an exemplary and schematic perspective view showing other positions of reference points set during learning for a vehicle included in image information, or specific points detected during estimation, in machine learning used in a vehicle position detection device according to an embodiment. [Figure 6] FIG. 6 is an exemplary schematic perspective view illustrating the position of a specific point detected when an estimation mode using machine learning is executed in the vehicle position detection device according to the embodiment. [Figure 7]FIG. 7 is an exemplary schematic diagram showing a case where the position of the vehicle is estimated based on the specific points detected in FIG. 6 and projected onto the world coordinate system. [Figure 8] FIG. 8 is an exemplary flowchart illustrating a process when a learning mode is executed in machine learning used in the vehicle position detection device according to the embodiment. [Figure 9] FIG. 9 is an exemplary flowchart illustrating processing when executing the estimation mode in machine learning used in the vehicle position detection device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments and modifications of the present disclosure will be described with reference to the drawings. The configurations of the embodiments and modifications described below, as well as the actions and effects brought about by the configurations, are merely examples and are not limited to the contents described below.

[0014] FIG. 1 is an exemplary schematic diagram showing an automatic valet parking system 100 that can use a vehicle position detection device 10 according to an embodiment. The vehicle position detection device 10 can be incorporated into, for example, a parking control device 101 that manages and monitors the automatic valet parking system 100. In another embodiment, the vehicle position detection device 10 and the parking control device 101 may be provided independently and may perform control while transmitting and receiving information to and from each other. Therefore, in the following description, it is assumed that the parking lot P of the automatic valet parking system 100, which is managed and monitored by the parking control device 101, includes a detection target area that is the detection target of the vehicle position detection device 10.

[0015] First, we will explain the automatic valet parking system 100. The automatic valet parking system 100 is a system for realizing automatic valet parking, including automatic parking and automatic departure, in a parking lot P having one or more parking areas R (parking spaces) demarcated by predetermined demarcation lines L, such as white lines, placed on the road surface. The parking control device 101 generates parking guidance information based on, for example, detection results from sensors located within the area of ​​the parking lot P. Then, using the generated parking guidance information, the vehicle is smoothly moved to an appropriate parking area R, or the vehicle's departure from the parking area R is controlled.

[0016] As shown in FIG. 1, in automatic valet parking, after occupant X gets off vehicle V in a predetermined drop-off area P1 in parking lot P, automatic parking is executed in which vehicle V automatically moves (drives) from drop-off area P1 to an empty parking area R (e.g., parking area R1) and parks in response to a predetermined instruction (see guided route C1 indicated by a dashed-dotted arrow in FIG. 1). After automatic parking is completed, automatic exit is executed in which vehicle V automatically leaves parking area R (parking area R1) and moves to a predetermined boarding area P2 and stops there in response to a predetermined call (see guided route C2 indicated by a dashed-dotted arrow in FIG. 1). The predetermined instruction and the predetermined call are realized, for example, by occupant X operating terminal device T.

[0017] Furthermore, in the automatic valet parking system 100, the automatic driving of the vehicle V is realized through cooperation between a parking control device 101 provided in the parking lot P and a vehicle control device 102 mounted on the vehicle V. The parking control device 101 and the vehicle control device 102 are configured to be able to communicate with each other via wireless communication.

[0018] Here, the parking control device 101 monitors the situation within the parking lot P by receiving image information obtained from one or more surveillance cameras 103 (imaging units) that capture images of the situation within the parking lot P and data output from various sensors (not shown) installed within the parking lot P. The surveillance cameras 103 are digital cameras incorporating imaging elements such as a charge coupled device (CCD) or a CMOS image sensor (CIS). The surveillance cameras 103 can output video data (image information) at a predetermined frame rate. The parking control device 101 is configured to manage multiple parking areas R based on the monitoring results. Note that FIG. 1 shows an example in which two surveillance cameras 103 are installed on the walls or the like of the parking lot P. In another example, for example, multiple surveillance cameras 103 may be installed on pillars or ceiling surfaces near one or more parking areas R to accurately monitor the usage status and driving lane status of one or more parking areas R, thereby improving monitoring accuracy. In addition to the monitoring camera 103, an infrared sensor, an ultrasonic sensor, or the like may be installed near the parking area R, and the usage status of the parking area R may be managed in the same manner.

[0019] The vehicle V is equipped with, for example, an on-board camera 308 (cameras 308R, 308L, etc. attached to the door mirrors), and the vehicle control device 102 detects the dividing lines L on the left and right sides of the vehicle V and the parking area R, while checking the surrounding conditions and realizing automatic driving.

[0020] On the other hand, the vehicle position detection device 10 detects the position of the vehicle V present in the parking lot P on the world coordinate system based on image data captured by the surveillance camera 103 or a dedicated imaging unit, and provides the position information of the detected vehicle V to the parking control device 101. At this time, the vehicle position detection device 10 can detect (estimate) the positions of all vehicles V present in the parking lot P, including vehicles V under automatic parking control, vehicles V under automatic exit control, vehicles V parked in the parking area R, and vehicles V stopped in positions other than the parking area R for some reason.

[0021] The vehicle position detection device 10 uses a learning method such as machine learning when detecting (estimating) the position of the vehicle V from the image data. The learning method is not limited to machine learning, and various well-known learning methods can be used. In the following explanation, machine learning (deep learning) will be used as the learning method. The vehicle position detection device 10 executes a learning mode in machine learning and an estimation mode that detects (estimates) the position of the vehicle V using a trained model constructed in the learning mode. The learning mode and estimation mode will be described in detail later.

[0022] Based on the position information of each vehicle V provided by the vehicle position detection device 10, the parking control device 101 controls, for example, a vehicle V in automatic driving so that the automatic driving is not hindered by other vehicles V (other vehicles V in automatic driving or parked vehicles V, etc.) or the vehicle does not come into contact with other vehicles V.

[0023] In the embodiment, the number and arrangement of the drop-off area P1, the pick-up area P2, and the parking area R in the parking lot P are not limited to the example shown in Fig. 1. The technology of the embodiment is applicable to parking lots with various configurations different from the parking lot P shown in Fig. 1.

[0024] FIG. 2 is an exemplary schematic block diagram illustrating the configuration of the vehicle position detection device 10. The vehicle position detection device 10 can be implemented by a general-purpose personal computer including hardware such as a processor and memory. The processor reads and executes programs stored in a memory such as a storage unit, thereby implementing each functional module of the vehicle position detection device 10. The vehicle position detection device 10 includes, as functional modules, a mode switching unit 12, an acquisition unit 14, a learning mode execution unit 16, and an estimation mode execution unit 18. The learning mode execution unit 16 includes detailed modules such as a reference point setting unit 16a and a model creation unit 16b. The estimation mode execution unit 18 includes a reference point estimation unit 18a and a processing unit 18b, and the processing unit 18b further includes detailed modules such as a conversion unit 18b1 and a position identification unit 18b2. These modules may be separated or integrated by function, or may be implemented as hardware.

[0025] The vehicle position detection device 10 is also connected to a trained model storage unit 20 that updatable stores a trained model constructed by machine learning, and a display device 22 that displays a vehicle V detected in a parking lot P on a map in a world coordinate system, etc. The trained model storage unit 20 is a non-volatile, rewritable storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The display device 22 includes a display unit 22a configured, for example, by a liquid crystal display (LCD) or an organic electroluminescent display (OLED). The display device 22 may be configured only by the display unit 22a, or may be integrated with an operation unit 22b that performs input operations when creating training data for the trained model or performing mode switching operations via the mode switching unit 12. When the display unit 22a and the operation unit 22b are integrated, the display unit 22a of the display device 22 is covered by a transparent operation unit 22b such as a touch panel. An operator of the vehicle position detection device 10 can view the image displayed on the display unit 22a via the operation unit 22b. The operator can also perform operation input by touching, pressing, or moving the operation unit 22b with a finger, a touch pen, or the like at a position corresponding to the image displayed on the display screen of the display unit 22a.

[0026] As described above, the vehicle position detection device 10 is also connected to the surveillance camera 103, which acquires image information of the parking lot P (detection target area), and the entry detection unit 23, which detects the entry of a moving object into the parking lot P. As described above, the surveillance camera 103 is capable of capturing image information for detecting the demarcation lines L and parking areas R within the parking lot P and monitoring the presence or absence of parked vehicles, and is also capable of capturing image information showing a vehicle V (a moving or stopped vehicle) present within the parking lot P. The entry detection unit 23 is a sensor capable of detecting an object (e.g., a moving object) entering the parking lot P. The type of sensor for the entry detection unit 23 can be appropriately selected, such as an optical sensor, an electromagnetic sensor, or a capacitance sensor, as long as it can detect an object entering the parking lot P. The detection result of the entry detection unit 23 can be used as a signal for starting the position detection process of the vehicle V by the vehicle position detection device 10. In addition, the entry detection unit 23 may be positioned outside the disembarking area P1 and the boarding area P2 (outside the parking lot P) so that the vehicle position detection device 10 can detect the location of the vehicle V in the disembarking area P1 and the boarding area P2.

[0027] The mode switching unit 12 switches between the learning mode execution unit 16 and the estimation mode execution unit 18 of the vehicle position detection device 10. For example, an operator can select between the learning mode and the estimation mode in machine learning by operating the operation unit 22b.

[0028] The acquisition unit 14 acquires image information (video information or still image information) of an area (detection target area) within the parking lot P acquired by the surveillance camera 103. In this case, the image information acquired by the surveillance camera 103 is assumed to include the entire area of ​​the parking lot P. When two surveillance cameras 103 are installed as shown in FIG. 1, it is sufficient that the entire area of ​​the parking lot P is covered by two types of image information acquired by the two cameras. The same applies when three or more surveillance cameras 103 are installed. The acquisition unit 14 may constantly acquire image information acquired by the surveillance cameras 103, or may start acquiring image information from the surveillance cameras 103 only when the entry detection unit 23 detects that an object (i.e., a moving object) has entered the parking lot P. In this case, the acquisition of image information related to the vehicle V, whose position has been estimated by the estimation mode execution unit 18, is confirmed to have completed parking. The same applies when an imaging unit dedicated to the vehicle position detection device 10 is provided separately from the surveillance cameras 103. In another embodiment, the parking control device 101 may control the acquisition of image information by the acquisition unit 14 .

[0029] The learning mode execution unit 16 executes learning mode processing in machine learning to estimate accurate position information of a vehicle V present in a parking lot P. The learning mode execution unit 16 is executed when the learning mode is selected by the mode switching unit 12.

[0030] As described above, the learning mode execution unit 16 includes a reference point setting unit 16a and a model creation unit 16b. The reference point estimation unit 18a sets a correct answer value for training data to construct a trained model used in machine learning to estimate the position of the vehicle V on the world coordinate system. That is, an operator performs annotation (correct answer assignment work) on the vehicle V captured in the image information of the parking lot P acquired by the acquisition unit 14. Note that the image information used at this time may be referred to as a "learning image," and the vehicle V captured in the image information may be referred to as a "learning vehicle." If the image information captured by the surveillance camera 103 is a video, a frame (i.e., a still image) in which a portion that can be an effective reference point for estimating the vehicle V is clearly captured is selected as the learning image from among multiple frames. If the image information captured by the surveillance camera 103 is a still image, the learning image is selected from the multiple still images. These selections may be made automatically by the learning mode execution unit 16, or may be made by an annotator (annotator) using the operation unit 22b. The reference point setting unit 16a sequentially sets positions on the learning image designated by the annotator using the operation unit 22b as reference points. The reference points are also associated with the shapes of the learning vehicles. The shapes of the learning vehicles are classified into, for example, four-door cars, two-door cars, standard cars, light cars, minivans, two-box cars, sports cars, SUVs, recreational vehicles, minivans, etc., and each is tagged with the image information. Associating the vehicle shapes with the reference points can improve the reliability of the training data.

[0031] 3 is a top view of an exemplary training vehicle V0 (vehicle V) showing the positions of reference points 26, 28 in machine learning used by the vehicle position detection device 10. As shown in FIG. 3, in the estimation mode of this embodiment, the reference points 26, 28 used to estimate the position of the vehicle V are set at, for example, six locations on the training vehicle V0 (specifically, six locations on the bottom of the vehicle). As shown in FIG. 3, the reference points 26, 28 can be the ground contact points (four points: ground contact points 26a to 26d) of each wheel 24 of the training vehicle V0, a ​​first projection point 28a when a portion of the front end portion VF of the vehicle body is projected onto the road surface, and a second projection point 28b when a portion of the rear end portion VR of the vehicle body is projected onto the road surface.

[0032] Here, reference points 26, 28 are positions at which the position and posture of training vehicle V0 relative to the road surface can be easily recognized. In FIG. 3, reference point 26 of right front wheel 24FR is ground contact point 26a, and reference point 26 of right rear wheel 24RR is ground contact point 26b. Similarly, reference point 26 of left front wheel 24FL is ground contact point 26c, and reference point 26 of left rear wheel 24RL is ground contact point 26d. As described above, the installation position and angle of view of surveillance camera 103 are set so that it can monitor vehicle V in addition to demarcation lines L and parking area R in parking lot P. Therefore, if training vehicle V0 on the road surface is captured in image information captured by surveillance camera 103, it is highly likely that one of wheels 24 is captured, and it is highly likely that one of ground contact points 26a to 26d can be identified as reference point 26.

[0033] Furthermore, if the learning vehicle V0 is captured in the image information captured by the monitoring camera 103, it is highly likely that either the front end VF or the rear end VR of the learning vehicle V0 is captured. In other words, it is highly likely that either the first projection point 28a, which is obtained when a portion of the front end VF of the vehicle is projected onto the road surface, or the second projection point 28b, which is obtained when a portion of the rear end VR of the vehicle is projected onto the road surface, can be confirmed. It is easy to determine whether the front end or the rear end of the learning vehicle V0 is captured in the captured image. Therefore, the front end VF of the learning vehicle V0 is easy to recognize in either position. Annotators who perform annotation while viewing image information may deviate from the position they consider to be the correct value of the training data depending on their own experience and intuition. However, for example, in the front end portion VF of the vehicle body, the right corner portion FRc of the front bumper, the left corner portion FLc of the front bumper, or the approximate center portion (approximately central position) FC, etc., have high geometrical characteristics, so there is little discrepancy in recognition by the annotator. In particular, since the vehicle V is generally symmetrical, the discrepancy in recognition of the approximate center portion FC of the front end portion VF of the vehicle body can be considered very small. The same is true for the rear end portion VR of the vehicle body, where the right corner portion RRc of the rear bumper, the left corner portion RLc of the rear bumper, or the approximate center portion (approximately central position) RC are easy to recognize, and the discrepancy in recognition of the approximate center portion RC of the rear end portion RF of the vehicle body can be considered very small. In the example of Figure 3, the first projection point 28a is set to the approximate center portion FC of the front bumper, and the second projection point 28b is set to the approximate center portion RC of the rear bumper. Therefore, when annotating the training vehicle V0, by using the ground contact points 26a to 26d of each wheel 24 and the first projection point 28a and the second projection point 28b as the correct values ​​for identifying the position of the vehicle V, more accurate annotation can be easily achieved without being influenced by the level of skill of the annotator, etc.

[0034] For example, Fig. 4 is an exemplary schematic perspective view showing the positions of reference points 26, 28 that are set on a training vehicle V0 included in an actual training image IM0 (image information) during training in machine learning used by the vehicle position detection device 10. Fig. 4 shows, for example, a training image IM0a captured by the surveillance camera 103 when the training vehicle V0 passes through the gate of a parking lot P and travels on an access road Pin.

[0035] The reference point setting unit 16a sequentially sets (registers) reference points 26, 28 at positions on the training vehicle V0 that the annotator specifies by operating the operation unit 22b while checking the training vehicle V0 included in the training image IM0a displayed on the display unit 22a. In the example of FIG. 4, as the reference point 26 for the wheels 24 of the training vehicle V0, the ground contact point 26a of the right front wheel 24FR is set as the correct value for the ground contact point of the right front wheel 24FR of the training vehicle V0, and the ground contact point 26b of the right rear wheel 24RR is set as the correct value for the ground contact point of the right rear wheel 24RR. Also, in FIG. 4, the annotator projects the position of the approximate center FC of the vehicle front end VF of the training vehicle V0 displayed on the display unit 22a onto the road surface G, and sets a first projection point 28a as the correct value for the road surface projection position of the approximate center FC of the vehicle body front end VF of the vehicle V. In this way, the reference point setting unit 16a can accurately and easily set the reference points 26 and 28 as correct values ​​at positions on the learning vehicle V0 that are easy to confirm.

[0036] FIG. 5 is an exemplary schematic perspective view showing the position of a reference point 28 that is set on a training vehicle V0 contained in an actual training image IM0 (image information) during training in machine learning used by the vehicle position detection device 10. FIG. 5 shows, for example, a training image IM0b captured by a monitoring camera 103 while the training vehicle V0 is traveling in a parking lot P. In the example shown in FIG. 5, the training vehicle V0 is facing directly ahead with respect to the monitoring camera 103. In this case, most of the wheels 24 (front wheel FR, left front wheel 24FL) are hidden by the vehicle body, and the contact points are not visible.

[0037] The reference point setting unit 16a sets (registers) a reference point 28 at a position on the learning vehicle V0 that the annotator specifies by operating the operation unit 22b while checking the learning vehicle V0 displayed on the display unit 22a. In the example of FIG. 5, the annotator projects the position of the approximate center FC of the vehicle front end VF of the learning vehicle V0 displayed on the display unit 22a onto the road surface G, and sets the first projection point 28a as the correct value for the road surface projection position of the approximate center FC of the vehicle body front end VF of the vehicle V. When the area directly behind the learning vehicle V0 is shown in the learning image IM0, only the second projection point 28b can be set by the annotator.

[0038] In this way, regardless of the posture of the learning vehicle V0, the reference point setting unit 16a can easily and accurately register at least one of the ground contact points 26a-26d of the wheels 24, the first projection point 28a, and the second projection point 28b as a correct value. The reference point setting unit 16a may set reference points for hidden portions of the learning vehicle V0 through estimation (prediction) by an annotator. For example, in the case of FIG. 4, the positions of the ground contact point 26c of the front left wheel FL, the ground contact point 26d of the rear left wheel RL, and the second projection point 28b, which is the road surface projection position of the approximate center RC of the rear end VR of the vehicle body, may be set through estimation by an annotator. In this case, even if a learning image IM0 in which the left side or rear of the vehicle body of the learning vehicle V0 can be confirmed is not acquired when the learning mode is executed, the estimation mode can be executed using the reference points 26 and 28 estimated during learning. 5, even when the wheel 24 is not visible, the annotator may set the reference points 26a to 26d, etc., through estimation. In this case, the positional accuracy may be lower than that of the reference points 26, 28 that are visible on the learning image IM0, so the reference points 26, 28 set through estimation may be distinguished as reference values.

[0039] The reference point setting unit 16a sets correct values ​​for a plurality of learning images IM0. The more learning images IM0 processed by the reference point setting unit 16a, the more accurately a learned model can be constructed in the model creation unit 16b to realize position estimation of the vehicle V, which is the estimation target, when the estimation mode is executed.

[0040] It should be noted that the learning image IM0 to be annotated by the reference point setting unit 16a may or may not have been captured in a parking lot P where the position of an actual vehicle V is estimated, as shown in FIGS. 4 and 5. In other words, image information captured in another location may be used as the learning image IM0 as long as it is a vehicle type that may enter the parking lot P. It may also be acquired from an existing database that stores vehicle images. Note that, when using learning images captured in a parking lot P where the position of an actual vehicle V is estimated, a trained model that corresponds to the tendency of vehicles using the parking lot P can be constructed at an early stage, enabling efficient construction of the trained model.

[0041] The model creation unit 16b constructs a trained model and sequentially updates the contents of the trained model storage unit 20. Well-known techniques can be used to construct the trained model, and detailed description thereof will be omitted here. For example, the trained model can be created by deep learning, which is a type of machine learning technique. The model creation unit 16b performs training using the training data created by the reference point setting unit 16a. In this case, for example, a function is constructed using parameters, a loss is defined for the correct data, and training is performed by minimizing this loss.

[0042] The estimation mode execution unit 18 executes estimation mode processing using a trained model in order to estimate accurate position information of a vehicle V present in a parking lot P. The estimation mode execution unit 18 is executed when the estimation mode is selected by the mode switching unit 12.

[0043] For example, when acquisition unit 14 acquires a signal indicating that an object (moving body) has entered parking lot P from entry detection unit 23, estimation mode execution unit 18 acquires image information from surveillance camera 103 via acquisition unit 14. In the following explanation, FIGS. 4 and 5 will be referred to as estimation images IM1 and IM2.

[0044] The reference point estimation unit 18a estimates corresponding positions (specific points) on the underside of the estimation vehicle corresponding to the positions of the reference points based on a comparison between the learned model stored in the learned model storage unit 20 and the estimation vehicle V1 included in the estimation image acquired by the acquisition unit 14. That is, the estimation image IM1 captured by the monitoring camera 103 and acquired by the acquisition unit 14 is input to the learned model, an area that can be considered to be the estimation vehicle V1 is extracted, and specific points 26T, 28T corresponding to the reference points 26, 28 are detected. Note that the reference point estimation unit 18a does not need to detect specific points 26T, 28T corresponding to all of the six reference points 26, 28 described above, and it is sufficient to detect at least one of them. As described above, specific point 26Ta corresponding to ground contact point 26a as reference point 26 is the ground contact point of right front wheel 24FR, specific point 26Tb corresponding to ground contact point 26b is the ground contact point of right rear wheel 24RR, specific point 26Tc corresponding to ground contact point 26c is the ground contact point of left front wheel 24FL, and specific point 26Td corresponding to ground contact point 26d is the ground contact point of left rear wheel 24RL. Furthermore, of the reference points 28, specific point 28Ta corresponding to first projection point 28a is a road surface projection point of approximately the center FC of the vehicle front end VF, and specific point 28Tb corresponding to second projection point 28b is a road surface projection point of approximately the center RC of the vehicle rear end VR. Therefore, if at least one of the six specific points 26T and 28T can be detected, the position of the bottom of vehicle V1 can be determined, and the position of vehicle V1 can be identified.

[0045] If two or more of the specific points 26T and 28T can be detected, it becomes possible to further identify the orientation, size, etc. of the estimation vehicle V1 on the world coordinate system. For example, by inputting the estimation image IM1 shown in FIG. 4 into a learned model and comparing it, the estimation vehicle V1 is extracted, and if specific point 26Ta, contact point 26Tb, and specific point 28Ta are detected, the position (pixel position) of the estimation vehicle V1 on the estimation image IM1 (image) is identified. Furthermore, by detecting specific point 28Ta, it is possible to estimate that the front end VF of the estimation vehicle V1 is facing the monitoring camera 103, and the length of the wheelbase can be estimated from the distance (number of pixels) between specific point 26Ta and specific point 26Tb.

[0046] 5 is input to the trained model for comparison, the estimation vehicle V1 is extracted, and when only the specific point 28Ta is detected, the position (pixel position) of the estimation vehicle V1 on the estimation image IM2 (image) is identified. Furthermore, by detecting the specific point 28Ta, it can be detected that the front end VF of the estimation vehicle V1 is facing the monitoring camera 103.

[0047] Next, processing unit 18b converts the corresponding positions (specific points 26T, 28T) corresponding to reference points 26, 28 estimated by reference point estimation unit 18a into a world coordinate system. Well-known techniques can be used for the conversion into the world coordinate system, and details will be omitted. For example, conversion unit 18b1 projects the estimated coordinates of estimation vehicle V1 into the world coordinate system using affine projection technology or the like. Position identification unit 18b2 identifies the position (coordinates) of estimation vehicle V1 on the world coordinate system and outputs the position (coordinates) to display device 22 or parking control device 101, for example.

[0048] For example, Fig. 6 is an exemplary schematic perspective view showing the positions of specific points 26T and 28T corresponding to reference points 26 and 28, detected when an estimation mode using machine learning is executed in the vehicle position detection device 10. In the example of Fig. 6, estimation vehicles V3 and V4 are extracted on the road surface G of a parking lot P by comparing an estimation image IM3 captured by a surveillance camera 103 with a learned model. For the estimation vehicle V3, specific points 26T corresponding to the reference point 26 are detected, including a specific point 26Ta of the right front wheel 24FR, a specific point 26Tb of the right rear wheel 24RR, and a specific point 26Td of the left rear wheel 24RL, and a specific point 28Ta corresponding to a first projection point 28a, which is a road surface projection point at approximately the center FC of the front end VF of the vehicle body. Furthermore, with regard to the estimation vehicle V4, specific points 26Ta of the right front wheel 24FR, 26Tc of the left front wheel 24FL, and 28Ta corresponding to the first projection point 28a, which is a road surface projection point at approximately the center FC of the front end VF of the vehicle body, have been detected.

[0049] Fig. 7 is an exemplary schematic diagram showing the positions of estimation vehicles V3 and V4 projected onto map M in the world coordinate system based on the specific points detected in Fig. 6. In this case, the road surface G of the parking lot P is at the same height, so the height direction of map M in the world coordinate system is omitted. As shown in Fig. 7, estimation vehicles V3 and V4 are indicated by corresponding positions (specific points 26T and 28T) on map M that correspond to reference points 26 and 28 that indicate the positions of the vehicle bottoms, and in the case of Fig. 7, the locations of the estimation vehicles V3 and V4 can be easily and accurately detected from a bird's eye view.

[0050] The estimation mode execution unit 18 may feed back the estimation image and estimation results used in the estimation mode processing to the learning mode execution unit 16 and use them to update the trained model.

[0051] In this way, the vehicle position detection device 10 of this embodiment estimates (identifies) specific points 26T, 28T corresponding to the reference points 26, 28 set on the opposite side of the vehicle to be used for estimating the position of the vehicle V. As a result, the position (coordinates) of the estimation vehicle V1 on the world coordinate system can be detected more accurately and easily using only image information captured by the monitoring camera 103, without using a special sensor (for example, a sensor that identifies the position with high accuracy, such as a radar).

[0052] The operation of the vehicle position detection device 10 configured as above will be described with reference to the flowcharts of FIGS.

[0053] FIG. 8 is an exemplary flowchart showing a process performed when the learning mode in the machine learning used in the vehicle position detection device 10 is executed.

[0054] The vehicle position detection device 10 first switches between learning mode and estimation mode via the mode switching unit 12 in response to the operation of the operation unit 22b by the annotator (S100). If the learning mode is not selected (No in S100), this flow is temporarily terminated. On the other hand, if the learning mode is selected (Yes in S100), learning images (information) captured by the monitoring camera 103 are acquired via the acquisition unit 14 (S102). Then, the reference point setting unit 16a of the learning mode execution unit 16 creates training data in response to the operation of the operation unit 22b by the annotator (S104). That is, for multiple learning images IM0, reference points 26 and 28 for the learning vehicle V0 are set and associated with the learning vehicle V0. Then, the model creation unit 16b executes a learning process to create a trained model using well-known machine learning (e.g., deep learning) techniques (S106). Then, the model creation unit 16b provides the created trained model to the trained model storage unit 20, and constructs (updates) the stored trained model (S108).

[0055] The vehicle position detection device 10 checks whether the learning termination condition is met based on the state of the mode switching unit 12 and the operation state of the operation unit 22b (S110). If the learning termination condition is not met (No in S110), the process proceeds to S100 and checks the current mode state. If the learning mode continues, the next multiple learning images IM0 are acquired, and the subsequent processes are repeatedly executed to continue building the trained model. If the learning termination condition is met in S110 (Yes in S110), for example, if a command to end learning is input from the operation unit 22b or if imaging by the monitoring camera 103 stops, the flow is temporarily terminated.

[0056] FIG. 9 is an exemplary flowchart showing processing when the estimation mode in machine learning used in the vehicle position detection device 10 is executed.

[0057] The vehicle position detection device 10 first switches between the learning mode and the estimation mode via the mode switching unit 12 in response to an operation of the operation unit 22b by the annotator (S200). If the estimation mode is not selected (No in S200), this flow is temporarily terminated. On the other hand, if the estimation mode is selected (Yes in S200), the acquisition unit 14 causes the entry detection unit 23 to check whether an object (moving object) has entered the parking lot P (S202). If the entry of an object cannot be confirmed (No in S202), that is, if there is no entry of the estimation vehicle V1 or the like, the vehicle position detection device 10 temporarily terminates this flow. As a result, if there is no entry of the estimation vehicle V1 or the like, the processing is paused, which makes it possible to reduce the processing load or to use the pause period to execute the learning mode processing, thereby contributing to efficient operation of the vehicle position detection device 10.

[0058] If the intrusion of an object is confirmed in S202 (Yes in S202), that is, if the intrusion of the estimation vehicle V1 or the like is detected, the reference point estimation unit 18a acquires the estimation image (information) captured by the monitoring camera 103 via the acquisition unit 14 (S204). The reference point estimation unit 18a also acquires a trained model from the trained model storage unit 20 (S206). Then, the reference point estimation unit 18a inputs the estimation image to the training model, extracts the estimation vehicle V1 from the estimation image IM2 or the like, and executes a process of identifying specific points 26T and 28T, which are corresponding positions corresponding to the reference points 26 and 28 (S208). If the reference point estimation unit 18a determines that a specific point 26T or a specific point 28T exists in the estimation image IM2 or the like (Yes in S210), the conversion unit 18b1 converts the coordinates of the specific point 26T or the specific point 28T into the world coordinate system and projects them onto the map M of the world coordinate system (S212). Next, the position identification unit 18b2 identifies the position of the estimation vehicle on the map M of the world coordinate system (S214), and the processing unit 18b executes a result notification process (S216) such as outputting the map M of the world coordinate system indicating the position of the estimation vehicle V1 to the display device 22 or the like and providing the coordinates to the parking control device 101.

[0059] When the processing unit 18b outputs the position (coordinates) of the estimation vehicle V1 on the world coordinate system, the reference point estimation unit 18a may display the reference point 26 (reference point 28) and / or the corresponding specific point 26T (specific point 28T) on the display device 22 (display unit 22a). Displaying the reference point 26 and the specific point 26T makes it easier to visually grasp the attitude of the estimation vehicle V1 and the positional relationship between the estimation vehicle V1 and surrounding objects (obstacles such as walls and other vehicles). Furthermore, the parking control device 101 may be provided with a display device capable of displaying the position (coordinates) of the estimation vehicle V1 on the world coordinate system, the reference point 26 (reference point 28) and / or the corresponding specific point 26T (specific point 28T). In this case, it becomes easier for the administrator of the automatic valet parking system 100 to visually grasp the posture of the estimation vehicle V1 and the positional relationship between the estimation vehicle V1 and surrounding objects (obstacles such as walls and other vehicles), making it easier to operate the automatic valet parking system 100 more efficiently.

[0060] Then, the vehicle position detection device 10 checks whether the estimation process termination condition is met based on the state of the mode switching unit 12 and the operation state of the operation unit 22b (S218), and if the estimation process termination condition is not met (No in S180), the process proceeds to S204, where the next estimation image is acquired, and the subsequent processes are repeated to continue the estimation process.If the estimation process termination condition is met in S218 (Yes in S218), for example, if an estimation termination operation is input from the operation unit 22b or imaging by the monitoring camera 103 stops, this flow is temporarily terminated.

[0061] In addition, in S210, if reference point estimation unit 18a cannot detect specific point 26T or specific point 28T in estimation image IM2 or the like (S210 No), the process proceeds to S218, where a determination process is performed as to whether or not an estimation process termination condition is satisfied. If the estimation process termination condition is not satisfied, the estimation process is continued on a new estimation image.

[0062] In this way, according to the vehicle position detection device 10 of the embodiment, the trained model is created in a state that includes at least one reference point that indicates a reference position that is included on the underside of the training vehicle and is easy to identify as a correct value, and the vehicle position is estimated based on the reference point. As a result, the accuracy of the vehicle position estimated from image information can be improved, and more accurate position detection on the world coordinate system becomes possible.

[0063] In the above example, the estimation mode execution unit 18 acquires image information from the surveillance camera 103 via the acquisition unit 14 and executes the estimation process when the acquisition unit 14 acquires a signal from the entry detection unit 23 indicating that an object (moving body) has entered the parking lot P. In another embodiment, regardless of the entry of the estimation vehicle V1 or the like, the estimation mode execution unit 18 may execute the learning mode process when a control start signal is transmitted from the parking control device 101 (for example, when a request operation by the administrator is executed). That is, the parking control device 101 may transmit, to the vehicle position detection device 10, a control start signal requesting a process to detect (estimate) the position of the vehicle V from the image data or a control end signal to stop the process, thereby controlling (setting) whether or not the vehicle position detection device 10 executes the estimation process. In this case, the parking control device 101 (or the administrator managing the parking lot P) can acquire accurate position information, attitude information, etc. of the vehicle V (estimation vehicle V1) as an estimation result by the vehicle position detection device 10, as necessary. As a result, it is possible to contribute to efficient operation of the automated valet parking system 100 (improvement of utilization efficiency).

[0064] In the above-described embodiment, the vehicle position detection device 10 is applied to the automatic valet parking system 100. However, the vehicle position detection device 10 can be applied to other technologies and provide similar benefits as long as the vehicle position contained in the image information can be effectively utilized by identifying the vehicle position on a world coordinate system. For example, the vehicle position detection device 10 can also be used to estimate the vehicle position in a regular parking lot, monitor the parking status of vehicles on roads, traffic conditions, and whether or not a vehicle is entering a specific area. This can contribute to vehicle monitoring and control based on the accurate vehicle position.

[0065] In addition, the programs for the learning mode processing and estimation mode processing executed by the processor that realizes the vehicle position detection device 10 of this embodiment may be configured to be provided by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0066] Furthermore, the program for executing the processing of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed in this embodiment may be provided or distributed via a network such as the Internet.

[0067] Although the embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0068] 10...vehicle position detection device, 12...mode switching unit, 14...acquisition unit, 16...learning mode execution unit, 16a...reference point setting unit, 16b...model creation unit, 18...estimation mode execution unit, 18a...reference point estimation unit, 18b...processing unit, 18b1...conversion unit, 18b2...position identification unit, 20...learned model memory unit, 22...display device, 22a...display unit, 22b...operation unit, 23...entry detection unit, 24...wheel, 26, 28...reference point, 26a, 26b, 26c, 26d...grounding point, 28a...first projection point, 28b...second projection point, 26T, 28T...identified point.

Claims

1. an acquisition unit that acquires image information from an imaging unit that images the detection target area; an estimation unit that estimates a corresponding position on the vehicle underside of the estimation vehicle corresponding to the position of the reference point based on a comparison between a trained model obtained by learning the relationship between the training vehicle included in the image information and at least one reference point indicating a reference position included in the vehicle underside of the training vehicle and the estimation vehicle included in the image information acquired by the acquisition unit; a processing unit that converts the corresponding position into a world coordinate system and detects the position of the estimation vehicle on the world coordinate system; Including, A vehicle position detection device, wherein the reference point is at least one of the following: the contact point of the wheels of the training vehicle; a first projection point when a portion of the front end of the training vehicle's body is projected onto the road surface; and a second projection point when a portion of the rear end of the training vehicle's body is projected onto the road surface.

2. 2. The vehicle position detection device according to claim 1, wherein the first projection point is a projection point at approximately the center of the front end of the body of the training vehicle, and the second projection point is a projection point at approximately the center of the rear end of the body of the training vehicle.

3. 3. The vehicle position detection device according to claim 1, wherein the estimation unit executes the estimation when it detects an entry of an estimation vehicle into the detection target area.

4. 3. The vehicle position detection device according to claim 1, wherein the estimation unit performs the estimation when a control start signal is received from a parking control device that manages a parking lot that includes the detection target area.

5. 5. The vehicle position detection device according to claim 1, further comprising a display unit that displays the reference position.

Images