Vehicle position estimation system, trained model generation method

Abstract

To provide a vehicle position estimation system capable of precisely estimating the position of a vehicle shown in an image captured by a camera.SOLUTION: The vehicle position estimation system extracts general feature points that indicate feature points that are not dependent on the vehicle model and multiple unique feature points that indicate feature points corresponding to each of the multiple applicable vehicle models from picked-up images using pre-trained models generated by machine learning, select the target unique feature points from multiple types of unique feature points depending on the type of the vehicle, and estimate the position of the target vehicle based on the image coordinates of general feature points and target unique feature points.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to techniques for estimating a position of a vehicle. [Background technology]

[0002] Various techniques have been proposed to improve the accuracy of estimating the position of a vehicle. For example, Patent Document 1 discloses a technique for estimating the relative positions and attitudes of multiple vehicles with high accuracy even when it is difficult to use GPS satellite signals. Another document that shows the technical level of this technical field is the following Patent Document 2. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-189910 [Patent Document 2] JP 2016-184316 A Summary of the Invention [Problem to be solved by the invention]

[0004] Regarding technology for estimating the position of a vehicle, there is a need for technology for estimating the position of a vehicle captured in an image captured by a camera. For example, the estimated position of a vehicle captured in an image captured by a camera can be used in functions such as an Auto Valet Parking (AVP) system, an autonomous driving system, and connected services. However, conventionally, it has not been possible to obtain an estimated position with sufficient accuracy.

[0005] One object of the present disclosure is to provide a technology that can improve the estimation accuracy for estimating the position of a vehicle captured in an image captured by a camera in consideration of the above-mentioned problems. [Means for solving the problem]

[0006] A first aspect of the present disclosure relates to a vehicle position estimation system. The vehicle position estimation system according to the first aspect includes a camera and one or more processors that estimate the position of a target vehicle captured in an image captured by the camera. The one or more processors are configured to execute the following processes: extracting generic feature points indicating feature points independent of the vehicle type from the captured image and multiple types of specific feature points indicating feature points corresponding to each of multiple types of applicable vehicle types, using a trained model generated in advance by machine learning; acquiring information on the vehicle type of the target vehicle; selecting a target specific feature point from the multiple types of specific feature points according to the vehicle type of the target vehicle; and estimating the position of the target vehicle based on the image coordinates of the generic feature points and the target specific feature point.

[0007] A second aspect of the present disclosure relates to a method for generating a trained model for causing a computer to function to extract feature points of a vehicle shown in a target image. The trained model is composed of an upper layer that receives the target image as an input, a generic feature point extraction layer that receives an output from the upper layer as an input and outputs feature points of the vehicle, and a plurality of unique feature point extraction layers that correspond to a plurality of applicable vehicle models and each receive an output from the upper layer as an input and output feature points of the vehicle. The generation method according to the second aspect includes training the upper layer and the generic feature point extraction layer with training data composed of a plurality of images showing various vehicles of unspecified vehicle models, and training each of the plurality of unique feature point extraction layers with training data composed of a plurality of images showing vehicles of the corresponding applicable vehicle models. Effect of the Invention

[0008] According to the present disclosure, the position of a target vehicle can be estimated using image coordinates of target-specific feature points optimized for the vehicle model of the target vehicle in addition to generic feature points, thereby improving estimation accuracy. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an overview of a vehicle position estimation system according to an embodiment. [Diagram 2]FIG. 2 is a diagram illustrating an example of the configuration of a vehicle position estimation function according to the first embodiment. [Diagram 3] FIG. 11 is a diagram showing an example of extracted feature points. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing unit according to the embodiment. [Diagram 5] FIG. 4 is a diagram showing a process executed by an information processing unit according to the first embodiment. [Figure 6] FIG. 1 is a diagram for explaining a trained model according to an embodiment. [Figure 7] FIG. 1 is a diagram illustrating a method for generating a trained model according to an embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of the configuration of a vehicle position estimation function according to the second embodiment. [Figure 9] FIG. 11 is a diagram showing a process executed by an information processing unit according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] 1. First embodiment 1.1 Vehicle position estimation system The vehicle position estimation system according to this embodiment provides a vehicle position estimation function that estimates the position of a vehicle shown in an image captured by a camera. Fig. 1 is a diagram showing an outline of a vehicle position estimation system 10 according to this embodiment. The vehicle position estimation system 10 includes an information processing unit 100 and a camera 200.

[0011] The camera 200 is installed so as to capture an image of the vehicle 1 that is the subject of estimation. The captured image 2 captured by the camera 200 is transmitted to the information processing unit 100.

[0012] The information processing unit 100 is a computer that executes processing related to a vehicle position estimation function. The information processing unit 100 acquires a captured image 2, executes processing, and outputs an estimated position of a vehicle 1 (hereinafter also referred to as a "target vehicle 1") that appears in the captured image 2. The estimated position indicates the position of the target vehicle 1 estimated in a predetermined world coordinate system. The world coordinate system may be appropriately provided depending on the environment in which the vehicle position estimation system 10 is applied.

[0013] The vehicle position estimation system 10 may function as a part of various systems. For example, the vehicle position estimation system 10 may function as a part of an AVP system. In this case, for example, the target vehicle 1 is a vehicle that performs parking by AVP, and the estimated position is output to identify the position of the target vehicle 1 in the parking lot. Also, the camera 200 is, for example, a camera installed in the parking lot where the AVP is implemented. Also, the world coordinate system is, for example, a coordinate system that gives a position in the parking lot.

[0014] Furthermore, for example, the vehicle position estimation system 10 may function as a part of an automatic driving system. In this case, for example, the target vehicle 1 is an automatic driving vehicle, and the estimated position is output for self-position estimation of the target vehicle 1. Furthermore, the camera 200 is, for example, an infrastructure camera installed to monitor the vehicle. Furthermore, the world coordinate system is, for example, a coordinate system that gives a position on a map.

[0015] 1.2 Vehicle position estimation function configuration 2 is a diagram showing an example of the configuration of a vehicle position estimation function according to this embodiment. The vehicle position estimation function is composed of a feature point extraction unit P10, an object specific feature point selection unit P20, and a vehicle position estimation unit P30. Each of the feature point extraction unit P10, the object specific feature point selection unit P20, and the vehicle position estimation unit P30 is realized by the information processing unit 100 executing processing.

[0016] The feature point extraction unit P10 acquires a captured image 2 and extracts feature points of the target vehicle 1 from the captured image 2. The extracted feature points are represented by image coordinates in the captured image 2. In particular, the feature points extracted by the feature point extraction unit P10 include generic feature points indicating feature points independent of vehicle type, and multiple types of specific feature points indicating feature points corresponding to each of multiple applicable vehicle types. The feature points can also be referred to as "key points." Each of the generic feature points and the specific feature points may include multiple feature points.

[0017] Generic feature points are feature points that can be uniformly interpreted for various vehicles. For example, generic feature points are the four corners of a rectangular area that indicates the road surface occupied by the target vehicle 1 or the tire contact surface. Unique feature points are feature points that are optimized for the corresponding applicable vehicle model. For example, unique feature points are feature points related to shapes or parts that are unique to the corresponding applicable vehicle model.

[0018] The applicable vehicle type is a type of vehicle supported by the vehicle position estimation system 10, and may be appropriately set according to the environment in which the vehicle position estimation system 10 is applied. For example, each of the multiple applicable vehicle types may be classified according to the vehicle body type (SUV, minivan, sedan, etc.) or the vehicle product name. Also, for example, each of the multiple applicable vehicle types may be classified according to a predetermined classification number appropriately assigned according to the model, specifications, etc.

[0019] Fig. 3 is a diagram showing an example of feature points of a target vehicle 1 extracted by the feature point extraction unit P10 when there are two applicable vehicle models, vehicle model A and vehicle model B. Fig. 3 shows an example of feature points extracted for a captured image 2 in which the target vehicle 1 is vehicle model A, and an example of feature points extracted for a captured image 2 in which the target vehicle 1 is vehicle model B.

[0020] 3, for both captured images 2 in which the target vehicle 1 is vehicle type A and captured images 2 in which the target vehicle 1 is vehicle type B, generic feature points, specific feature points relating to vehicle type A (specific feature points relating to vehicle type A), and specific feature points relating to vehicle type B (specific feature points relating to vehicle type B) are extracted. The specific feature points relating to vehicle type A are optimized for vehicle type A, and therefore provide more accurate feature points for the captured image 2 in which the target vehicle 1 is vehicle type A. On the other hand, the specific feature points relating to vehicle type B are optimized for vehicle type B, and therefore provide more accurate feature points for the captured image 2 in which the target vehicle 1 is vehicle type B.

[0021] In this embodiment, the feature point extraction unit P10 is configured with a trained model that is generated in advance by machine learning. That is, the feature point extraction unit P10 uses the trained model to extract generic feature points and multiple types of unique feature points from the captured image 2. The trained model is generated so as to receive the captured image 2 as input and output the generic feature points and multiple types of unique feature points in the input captured image 2. The configuration and generation method of the trained model will be described later.

[0022] Referring again to FIG. 2, the target unique feature point selection unit P20 acquires the multiple types of extracted unique feature points. Then, the target unique feature point selection unit P20 selects unique feature points suitable for the target vehicle 1 from the multiple types of unique feature points (hereinafter referred to as "target unique feature points"). In the first embodiment, the target unique feature point selection unit P20 acquires vehicle model information of the target vehicle 1, and selects unique feature points according to the vehicle model of the target vehicle 1 as the target unique feature points. For example, when the target vehicle 1 is vehicle model A, the target unique feature point selection unit P20 selects unique feature points related to vehicle model A from the multiple types of unique feature points as the target unique feature points.

[0023] The vehicle position estimation unit P30 acquires the extracted generic feature points and the selected target specific feature points. The vehicle position estimation unit P30 also accesses the applicable vehicle model database D10. The applicable vehicle model database D10 is a database that manages data related to each of a plurality of applicable vehicle models. In this embodiment, the applicable vehicle model database D10 includes data that can identify the coordinates of each feature point in a vehicle coordinate system based on the vehicle for each applicable vehicle model. For example, the applicable vehicle model database D10 manages coordinate data in the vehicle coordinate system of each feature point calculated in advance from CAD data of the vehicle body for each applicable vehicle model.

[0024] The vehicle position estimation unit P30 estimates the position of the target vehicle 1 based on the image coordinates of the generic feature points and the target specific feature points. The vehicle position estimation unit P30 can estimate the position of the target vehicle 1, for example, as follows.

[0025] The vehicle position estimation unit P30 can obtain coordinates in the vehicle coordinate system for each of the generic feature points and the object specific feature points by accessing the applicable vehicle model database D10. For example, the vehicle position estimation unit P30 obtains coordinate data corresponding to the vehicle model of the target vehicle 1 from the applicable vehicle model database D10 and obtains coordinates by matching feature points. Here, when the position of the target vehicle 1 in the world coordinate system is given, the coordinates in the world coordinate system of the generic feature points and the object specific feature points can be determined from the coordinates in the vehicle coordinate system. Furthermore, the coordinates in the world coordinate system can be converted into coordinates in the image coordinate system by a known coordinate conversion using the camera parameters of the camera 200 (focal length, distortion correction parameters, camera position, camera attitude, etc.). In other words, if the position of the target vehicle 1 is assumed, the vehicle position estimation unit P30 can calculate the coordinates in the image coordinate system of the generic feature points and the object specific feature points. In this case, the vehicle position estimation unit P30 may calculate the coordinates in the image coordinate system by adding a constraint condition (for example, height=0 as a constraint condition for the road surface position).

[0026] Therefore, the vehicle position estimation unit P30 can estimate the position of the target vehicle 1 by solving a task of minimizing the difference between the image coordinates of the acquired generic feature points and target specific feature points and the image coordinates of the generic feature points and target feature points calculated when the position of the target vehicle 1 is assumed, with the position of the target vehicle 1 being treated as an unknown.

[0027] For example, consider a case where the position of the target vehicle 1 is estimated in three dimensions (X, Y, θ). In this case, the vehicle position estimation unit P30 can estimate the position of the target vehicle 1 by solving a task expressed by the following equation (1). Here, (u i ,v i ) is the image coordinate of the acquired generic feature point or target specific feature point, (u i ',v i ')| (X,Y,θ) is the image coordinate of the generic feature point or the target specific feature point calculated when the position of the target vehicle 1 is (X, Y, θ), and w i is the weight for each feature point. i For example, a small value is set for a generic feature point, and a large value is set for a specific feature point. Note that the following formula (1) can also be applied when estimating the position of the target vehicle 1 in higher or lower dimensions. For example, a task can be configured similarly when estimating the position of the target vehicle 1 in six dimensions of (X, Y, Z, yaw, picture, roll).

number

[0028] As described above, the vehicle position estimation function according to this embodiment is configured. The information processing unit 100 outputs the estimated position of the target vehicle 1 estimated by the vehicle position estimation unit P30.

[0029] 1.3 Information Processing Section 4 is a diagram showing an example of a hardware configuration of the information processing unit 100. The information processing unit 100 includes one or more processors 110 (hereinafter simply referred to as processors 110), one or more storage devices (hereinafter simply referred to as storage devices 120), and a communication I / F 130.

[0030] The processor 110 executes various processes and may be configured with, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.

[0031] The storage device 120 stores various information necessary for the execution of processes by the processor 110. The storage device 120 can be configured with a recording medium such as a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), or a solid state drive (SSD).

[0032] The storage device 120 stores a computer program 121, a trained model 122, and an applicable vehicle model database D10.

[0033] The computer program 121 is executed by the processor 110. Various processes by the information processing unit 100 may be realized by cooperation between the processor 110 executing the computer program 121 and the storage device 120. In particular, the feature point extraction unit P10, the object specific feature point selection unit P20, and the vehicle position estimation unit P30 may be realized. The feature point extraction unit P10, the object specific feature point selection unit P20, and the vehicle position estimation unit P30 may be realized by a single processor 110 or by separate processors 110. The computer program 121 may be recorded in a computer-readable recording medium. The processor 110 may configure the feature point extraction unit P10 by reading out the trained model 122 from the storage device 120 and using it.

[0034] The communication I / F 130 is an interface for connecting to and communicating with the outside of the information processing unit 100. Examples of the communication I / F 130 include a device for connecting to the Internet, a device for connecting to a mobile communication network, etc. The information processing unit 100 communicates with the camera 200 and the target vehicle 1 via the communication I / F 130.

[0035] 5 is a flowchart showing an example of processing executed by the information processing unit 100 based on the configuration described in "1.2 Vehicle position estimation function", more specifically, processing executed by the processor 110. The processing according to the flowchart shown in FIG. 5 is started, for example, when the information processing unit 100 acquires the captured image 2 from the camera 200.

[0036] In step S110, the processor 110 extracts generic feature points and multiple types of unique feature points from the captured image 2 using the trained model 122.

[0037] Next, in step S120, the processor 110 acquires vehicle type information of the target vehicle 1. The processor 110 may acquire the vehicle type information directly by communicating with the target vehicle 1, or may acquire the vehicle type information by performing image recognition on the captured image 2. Alternatively, the processor 110 may acquire identification information of the target vehicle 1 by communicating with the target vehicle 1, and acquire the vehicle type information by referring to the applicable vehicle type database D10 using the identification information.

[0038] Next, in step S130, the processor 110 selects a target unique feature point from the extracted multiple unique feature points according to the vehicle type of the target vehicle 1.

[0039] Next, in step S140, the processor 110 estimates the position of the target vehicle 1 based on the image coordinates of the extracted generic feature points and the selected target specific feature points. For example, the processor 110 estimates the position of the target vehicle 1 by solving the task represented by the above equation (1) using an extended Kalman filter, Lagrange's undetermined multiplier method, iterative processing by a solver, or the like. By executing the processing related to step S140, an estimated position of the target vehicle 1 is obtained. After step S140, the processing ends.

[0040] The vehicle position estimation function according to the present embodiment is realized by the processor 110 executing the processes in this manner. Furthermore, the vehicle position estimation program according to the present embodiment is realized by the computer program 121 that causes the processor 110 to execute the processes in this manner.

[0041] 1.4 Pre-trained models The following describes the trained model 122 for extracting feature points of the target vehicle 1 from the captured image 2. FIG.

[0042] The trained model 122 is composed of an upper layer 20, a generic feature point extraction layer 31, and a plurality of specific feature point extraction layers corresponding to each of a plurality of applicable vehicle models.

[0043] The upper layer 20 receives the captured image 2 as input and outputs the feature amount of the captured image 2. The upper layer 20 may employ a CNN (Convolutional Neural Network), a Transformer model, or the like. In the trained model 122, the captured image 2 may be considered as a "target image" from which feature points are extracted.

[0044] The generic feature point extraction layer 31 receives the output of the upper layer 20 and outputs generic feature points. i} iindicates the generic feature points to be output. The generic feature point extraction layer 31 is composed of, for example, a feature amount extraction unit for extracting features suitable for extracting generic feature points from the output of the upper layer 20, and an output unit for outputting image coordinates of generic feature points from the output of the feature amount extraction unit. In this case, the generic feature point extraction layer 31 may employ a CNN, a Transformer model, or the like as the feature amount extraction unit. Also, an Affine layer, a Softmax layer, or the like may be employed as the output unit.

[0045] Each of the multiple unique feature point extraction layers receives the output of the upper layer 20 as input, and outputs unique feature points according to the corresponding applicable vehicle model. In Fig. 6, a unique feature point extraction layer 32A and a unique feature point extraction layer 32B are shown. For example, the unique feature point extraction layer 32A corresponds to vehicle model A, and extracts unique feature points {kpA i} i The unique feature point extraction layer 32B corresponds to the vehicle model B and outputs the unique feature points {kpB i} i Each of the plurality of unique feature point extraction layers is composed of, for example, a feature amount extraction unit for extracting features suitable for extracting unique feature points according to the corresponding applicable vehicle model from the output of the upper layer 20, and an output unit for outputting image coordinates of the unique feature points according to the corresponding applicable vehicle model from the output of the feature amount extraction unit. In particular, each unique feature point extraction layer may have an equivalent configuration to each other.

[0046] 6 shows two unique feature point extraction layers 32A and 32B, the multiple unique feature point extraction layers may include three or more unique feature point extraction layers. As understood from the above description, even when three or more unique feature point extraction layers are included, it is possible to configure a similar trained model 122.

[0047] As described above, the trained model 122 according to this embodiment is configured. The trained model 122 is generated in advance by machine learning. Fig. 7 is a flowchart showing a method for generating the trained model 122. The process according to the flowchart shown in Fig. 7 is executed by the processor 110 during training, for example.

[0048] First, in step S210, the upper layer 20 and the generic feature point extraction layer 31 are trained. The upper layer 20 and the generic feature point extraction layer 31 are trained using training data (hereinafter also referred to as "generic training data") consisting of a plurality of images showing various vehicles without specifying the model. The training data may include information on correct feature points. The generic training data may be, for example, a large-scale data set. The upper layer 20 and the generic feature point extraction layer 31 may be trained using, for example, the backpropagation method. At this time, the upper layer 20 and the generic feature point extraction layer 31 may be trained integrally.

[0049] By learning using generic training data, the upper layer 20 and the generic feature point extraction layer 30 are expected to output results with high generalization performance for various vehicles. In particular, it is possible to prepare a large amount of training data as generic training data, and high-level learning can be expected.

[0050] Next, in step S220, each of the multiple unique feature point extraction layers is trained. Each unique feature point extraction layer is trained using training data made up of multiple images showing a vehicle of the corresponding applicable model. For example, the unique feature point extraction layer for model A is trained using training data made up of multiple images showing a vehicle of model A. Each unique feature point extraction layer can be trained using the backpropagation method in the same way. At this time, training may be performed in a state where the upper layer 20 and the unique feature point extraction layer to be trained are connected. However, in order to maintain the learning results of the upper layer 20 using generic training data, it is desirable to freeze the parameters related to the upper layer 20 during training of each unique feature point extraction layer.

[0051] By training each of the multiple unique feature point extraction layers in this way, it is expected that each unique feature point extraction layer will output results optimized for the corresponding applicable vehicle model. In particular, considering that the upper layer 20 is trained to a high level of proficiency using generic training data, each unique feature point extraction layer can train to a high level of proficiency even with a relatively small amount of training data. As a result, highly efficient learning is possible.

[0052] After step S220, the process ends. By executing the process in this manner, the method for generating the trained model 122 according to this embodiment is realized.

[0053] According to the trained model 122 of this embodiment, it is easy to expand the applicable vehicle models after the fact. For example, consider a case where there are two applicable vehicle models, vehicle model A and vehicle model B. Then, consider a case where vehicle model C is newly added as an applicable vehicle model. In this case, a unique feature point extraction layer for vehicle model C can be newly connected to the upper layer 20 to configure the trained model 122. The unique feature point extraction layer for vehicle model C may be the generic feature point extraction layer 31 or another unique feature point extraction layer. Then, the trained model 122 is trained only on the unique feature point extraction layer for vehicle model C. In this way, by using the trained model 122 of this embodiment, it is possible to realize a vehicle position estimation system 10 in which it is easy to expand the applicable vehicle models after the fact.

[0054] 1.5 Effects As described above, according to this embodiment, position estimation is performed based on the image coordinates of target specific feature points optimized for the vehicle type of the target vehicle 1 in addition to generic feature points, thereby realizing a vehicle position estimation function with high estimation accuracy.

[0055] 2. Second embodiment The vehicle position estimation system 10 according to the second embodiment will be described below. The vehicle position estimation system 10 according to the second embodiment provides a vehicle position estimation function, similar to the first embodiment. In the following description, differences from the first embodiment will be mainly described, and contents common to the first embodiment will be omitted as appropriate.

[0056] 2.1 Configuration of vehicle position estimation function Fig. 8 is a diagram showing an example of the configuration of a vehicle position estimation function according to the second embodiment. The vehicle position estimation function according to the second embodiment is composed of a feature point extraction unit P11, an object specific feature point selection unit P21, a vehicle position estimation unit P30, and a posture estimation unit P40. In Fig. 8, elements common to the first embodiment are denoted by the same reference numerals.

[0057] The feature point extraction unit P11 extracts generic feature points and multiple types of unique feature points from the captured image 2. In particular, in the second embodiment, each of the multiple types of unique feature points extracted by the feature point extraction unit P11 further includes multiple types of posture-specific feature points classified according to the posture of the vehicle. Here, the "vehicle posture" refers to the posture of the vehicle as seen from the camera 200, and indicates whether the vehicle is facing forward or backward in the captured image 2, for example. The posture-specific feature points may include multiple feature points.

[0058] The posture-specific feature points are characteristic feature points that are further optimized for the posture of the corresponding vehicle among the inherent feature points. For example, if posture X indicates that the vehicle faces forward in the captured image 2, the posture-specific feature points for posture X are feature points related to shapes and parts that are often captured when the vehicle faces forward.

[0059] The feature point extraction unit P11 is configured by the trained model 122, as in the first embodiment. Therefore, in the trained model 122, each of the multiple unique feature point extraction layers is configured to output unique feature points including multiple types of posture-specific feature points. For example, the unique feature point extraction layer for vehicle type A outputs posture-specific feature points {kpAX i} i , and the posture-specific feature points {kpAYi} i The apparatus is configured to output:

[0060] The attitude estimation unit P40 estimates the attitude of the target vehicle 1. For example, the attitude estimation unit P40 acquires the extracted generic feature points and estimates the attitude of the target vehicle 1 based on the image coordinates of the generic feature points. When the generic feature points are the four corners of a rectangular area, the attitude estimation unit P40 estimates the attitude of the target vehicle 1 from, for example, the shape of the rectangular area represented by the generic feature points. The attitude estimation unit P40 may estimate the attitude of the target vehicle 1 by image recognition of the captured image 2. Also, for example, the attitude estimation unit P40 may estimate the attitude of the target vehicle 1 by tracking the target vehicle 1.

[0061] The target unique feature point selection unit P21 further acquires posture information of the target vehicle 1 from the posture estimation unit P40. In the second embodiment, the target unique feature point selection unit P21 selects unique feature points corresponding to the vehicle type of the target vehicle 1 from among the multiple types of unique feature points, and selects, as the target unique feature point, a posture-specific feature point corresponding to the posture of the target vehicle 1 from among the multiple types of posture-specific feature points included in the selected unique feature points. That is, in the second embodiment, the target unique feature point is a feature point that is suitable for both the vehicle type and posture of the target vehicle 1.

[0062] 2.2 Information Processing Section FIG. 9 is a flowchart showing an example of the processing executed by the information processing unit 100, more specifically, the processing executed by the processor 110, based on the configuration described in "2.1 Vehicle position estimation function".

[0063] Steps S310 and S320 are equivalent to steps S110 and S120, respectively, described with reference to Fig. 5. After step S320, the process proceeds to step S330.

[0064] In step S330, the processor 110 estimates the attitude of the target vehicle 1.

[0065] Next, in step S340, the processor 110 selects a target unique feature point from the extracted multiple unique feature points according to the vehicle type and posture of the target vehicle 1.

[0066] Next, in step S350, the processor 110 estimates the position of the target vehicle 1 based on the image coordinates of the extracted generic feature points and the selected target specific feature points. By executing the process related to step S350, an estimated position of the target vehicle 1 is obtained. After step S350, the process ends.

[0067] 2.3 Effects As described above, according to the second embodiment, position estimation is performed based on image coordinates of target unique feature points that are further optimized for the posture of the target vehicle 1 compared to the first embodiment, thereby realizing a vehicle position estimation function with even higher estimation accuracy.

[0068] 2.4 Variations The vehicle position estimation system 10 according to the second embodiment may employ the following modified aspects.

[0069] In a modified example, the feature point extraction unit P11 is configured so that each of the multiple types of unique feature points to be extracted includes multiple feature points to which reliability that changes according to the vehicle's posture is assigned, rather than multiple types of posture-specific feature points. For example, when five feature points kpA1, kpA2, kpA3, kpA4, and kpA5 are extracted as unique feature points related to vehicle type A, the following reliability is assigned to each feature point. The reliability may be assigned to the class of each feature point by a map prepared in advance. In this case, the map may be generated by past position estimation results, compatibility tests, etc. [Table 1]

[0070] In the modified example, the target unique feature point selection unit P21 is configured to select unique feature points corresponding to the vehicle type of the target vehicle 1 from among multiple types of unique feature points as target unique feature points, and to exclude feature points whose reliability is less than a threshold value from the target unique feature points based on the posture of the target vehicle 1. For example, assume that the target vehicle 1 is vehicle type A, and that the target unique feature point selection unit P21 has selected the unique feature points related to the vehicle type A as the target unique feature points. Also assume that the threshold value is 40%. In this case, the target unique feature point selection unit P21 executes processing to exclude kpA1 and kpA2 from the target unique feature points.

[0071] This enables the vehicle position estimation unit P30 to estimate the position of the target vehicle 1 based on the feature points that have sufficient reliability for the attitude of the target vehicle 1. Furthermore, by adopting such a modified embodiment, it is possible to achieve the same effects as those described above. [Explanation of symbols]

[0072] 1 target vehicle, 2 captured image, 10 vehicle position estimation system, 100 information processing unit, 110 processors, 200 cameras

Claims

1. A camera and one or more processors for estimating a position of a target vehicle shown in an image captured by the camera; The one or more processors: A process of extracting generic feature points indicating feature points independent of vehicle type from the captured image and multiple types of specific feature points indicating feature points corresponding to each of multiple applicable vehicle types, using a trained model generated in advance by machine learning; A process of acquiring information on the vehicle type of the target vehicle; A process of selecting a target unique feature point from the plurality of types of unique feature points according to a vehicle type of the target vehicle; A process of estimating a position of the target vehicle based on image coordinates of the generic feature points and the target specific feature points; It is configured to run Vehicle position estimation system.

2. 2. The vehicle position estimation system according to claim 1, The trained model is An upper layer that receives the captured image as an input; a generic feature point extraction layer that receives the output of the upper layer as input and outputs the generic feature points; a plurality of unique feature point extraction layers corresponding to the plurality of applicable vehicle models, each of which receives an output from the upper layer as an input and outputs unique feature points according to the corresponding applicable vehicle model; It is composed of The upper layer and the generic feature point extraction layer are trained using training data consisting of a plurality of images showing various vehicles without specifying the vehicle type, Each of the plurality of characteristic feature extraction layers is trained using training data consisting of a plurality of images showing the corresponding applicable vehicle model. Characterized by Vehicle position estimation system.

3. 3. A vehicle position estimation system according to claim 1, further comprising: each of the plurality of types of unique feature points includes a plurality of types of posture-specific feature points classified according to a posture of a vehicle; The one or more processors are further configured to perform a process of acquiring information about a posture of the target vehicle; The process of selecting the object characteristic feature points includes: selecting a unique feature point corresponding to a vehicle type of the target vehicle from among the plurality of types of unique feature points; selecting, as the target unique feature point, a pose-specific feature point corresponding to a pose of the target vehicle from among the plurality of types of pose-specific feature points included in the selected unique feature point; Includes Characterized by Vehicle position estimation system.

4. 3. A vehicle position estimation system according to claim 1, further comprising: each of the plurality of types of unique feature points includes a plurality of feature points to which a reliability that varies depending on a posture of the vehicle is assigned; The one or more processors are further configured to perform a process of acquiring information about a posture of the target vehicle; The step of selecting the object characteristic feature points comprises: selecting, as the target unique feature point, a unique feature point corresponding to a vehicle type of the target vehicle from among the plurality of types of unique feature points; removing feature points whose reliability is less than a threshold value from among the target unique feature points based on the posture of the target vehicle; Includes Characterized by Vehicle position estimation system.

5. A method for generating a trained model for causing a computer to function to extract feature points of a vehicle shown in a target image, The trained model is An upper layer that receives the target image as an input; a generic feature point extraction layer that receives the output of the upper layer as input and outputs feature points of the vehicle; a plurality of characteristic feature extraction layers corresponding to a plurality of applicable vehicle models, each of which receives an output from the upper layer as an input and outputs characteristic points of the vehicle; It is composed of training the upper layer and the generic feature point extraction layer using training data consisting of a plurality of images showing various vehicles of unspecified models; training each of the plurality of unique feature point extraction layers using training data consisting of a plurality of images showing a corresponding vehicle of an applicable model; Includes Generation method.

Images