Vehicle position estimation system, method for generating a trained model

Abstract

To provide a vehicle position estimation system capable of precisely estimating the position of a vehicle shown in an image captured by a camera.SOLUTION: The vehicle position estimation system extracts general feature points that indicate feature points that are not dependent on the vehicle model and multiple unique feature points that indicate feature points corresponding to each of the multiple applicable vehicle models from picked-up images using pre-trained models generated by machine learning, select the target unique feature points from multiple types of unique feature points depending on the type of the vehicle, and estimate the position of the target vehicle based on the image coordinates of general feature points and target unique feature points.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] This disclosure relates to a technology for estimating the position of a vehicle. [Background technology]

[0002] Various techniques have been proposed to improve the accuracy of vehicle position estimation. For example, Patent Document 1 discloses a technique for accurately estimating the relative position and orientation between multiple vehicles, even when it is difficult to use GPS satellite signals. In addition, Patent Document 2 below is a document that demonstrates the level of technology in this field. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-189910 [Patent Document 2] Japanese Patent Publication No. 2016-184316 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] Regarding vehicle position estimation technology, there is a need for technology that estimates the position of vehicles captured in images taken by cameras. For example, the estimated position of a vehicle captured in an image taken by a camera can be used in functions such as AVP (Auto Valet Parking) systems, autonomous driving systems, and connected services. However, conventionally, it has not been possible to obtain sufficiently accurate estimated positions.

[0005] One of the purposes of this disclosure is, in light of the above-mentioned problems, to provide a technology that can improve the estimation accuracy of a technology for estimating the position of a vehicle captured in an image taken by a camera. [Means for solving the problem]

[0006] The first aspect of this disclosure relates to a vehicle position estimation system. The vehicle position estimation system according to the first aspect comprises a camera and one or more processors that estimate the position of a target vehicle captured in an image taken by the camera. The one or more processors are configured to perform the following processes using a pre-trained model generated by machine learning: extracting general feature points that indicate feature points independent of vehicle type and multiple types of specific feature points that indicate feature points corresponding to each of multiple types of applicable vehicle types from the captured image; acquiring information on the vehicle type of the target vehicle; selecting target specific feature points from the multiple types of specific feature points according to the vehicle type of the target vehicle; and estimating the position of the target vehicle based on the image coordinates of the general feature points and the target specific feature points.

[0007] The second aspect of this disclosure relates to a method for generating a trained model that causes a computer to function to extract feature points of vehicles appearing in a target image. The trained model consists of an upper layer that takes the target image as input, a general-purpose feature point extraction layer that takes the output of the upper layer as input and outputs the feature points of the vehicle, and a plurality of unique feature point extraction layers, each corresponding to a plurality of applicable vehicle types, which each outputs the feature points of the vehicle taking the output of the upper layer as input. The generation method relating to the second aspect includes training the upper layer and the general-purpose feature point extraction layer with training data consisting of a plurality of images showing various vehicles without specifying a vehicle type, and training each of the plurality of unique feature point extraction layers with training data consisting of a plurality of images showing vehicles of the corresponding applicable vehicle type. [Effects of the Invention]

[0008] According to this disclosure, the position of the target vehicle can be estimated using image coordinates of specific feature points optimized for the vehicle type, in addition to general feature points, thereby improving estimation accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows a schematic diagram of the vehicle position estimation system according to the embodiment. [Figure 2]It is a diagram showing an example of the configuration of the vehicle position estimation function according to the first embodiment. [Figure 3] It is a diagram showing an example of feature points to be extracted. [Figure 4] It is a diagram showing an example of the hardware configuration of the information processing unit according to the embodiment. [Figure 5] It is a diagram showing the processing executed by the information processing unit according to the first embodiment. [Figure 6] It is a diagram for explaining the learned model according to the embodiment. [Figure 7] It is a diagram showing a method for generating the learned model according to the embodiment. [Figure 8] It is a diagram showing an example of the configuration of the vehicle position estimation function according to the second embodiment. [Figure 9] It is a diagram showing the processing executed by the information processing unit according to the second embodiment.

Embodiments for Carrying Out the Invention

[0010] 1 First Embodiment 1.1 Vehicle Position Estimation System The vehicle position estimation system according to the present embodiment provides a vehicle position estimation function for estimating the position of a vehicle reflected in a captured image captured by a camera. FIG. 1 is a diagram showing an overview of the vehicle position estimation system 10 according to the present embodiment. The vehicle position estimation system 10 includes an information processing unit 100 and a camera 200.

[0011] The camera 200 is installed so as to capture the vehicle 1 to be estimated. The captured image 2 captured by the camera 200 is transmitted to the information processing unit 100.

[0012] The information processing unit 100 is a computer that executes processing related to the vehicle position estimation function. The information processing unit 100 acquires the captured image 2, executes processing, and outputs the estimated position of the vehicle 1 (hereinafter also referred to as the "target vehicle 1") reflected in the captured image 2. The estimated position indicates the position of the target vehicle 1 estimated in a predetermined world coordinate system. The world coordinate system may be suitably given according to the environment to which the vehicle position estimation system 10 is applied.

[0013] The vehicle position estimation system 10 may function as part of various systems. For example, the vehicle position estimation system 10 may function as part of an AVP system. In this case, for example, the target vehicle 1 is a vehicle that parks by AVP, and the estimated position is output to identify the position of the target vehicle 1 within the parking lot. Also, the camera 200 is, for example, a camera installed within the parking lot where AVP is implemented. Also, the world coordinate system is, for example, a coordinate system that gives positions in the parking lot.

[0014] Also, for example, the vehicle position estimation system 10 may function as part of an automatic driving system. In this case, for example, the target vehicle 1 is a vehicle that performs automatic driving, and the estimated position is output for self-position estimation of the target vehicle 1. Also, the camera 200 is, for example, an infrastructure camera installed to monitor the vehicle. Also, the world coordinate system is, for example, a coordinate system that gives positions on a map.

[0015] 1.2 Configuration of Vehicle Position Estimation Function FIG. 2 is a diagram showing an example of the configuration of the vehicle position estimation function according to the present embodiment. The vehicle position estimation function is composed of a feature point extraction unit P10, a target specific feature point selection unit P20, and a vehicle position estimation unit P30. Each of the feature point extraction unit P10, the target specific feature point selection unit P20, and the vehicle position estimation unit P30 is realized by the information processing unit 100 executing processing.

[0016] The feature point extraction unit P10 acquires the captured image 2 and extracts the feature points of the target vehicle 1 from the captured image 2. The extracted feature points are represented by image coordinates in the captured image 2. In particular, the feature points extracted by the feature point extraction unit P10 include general feature points that indicate feature points independent of the vehicle type, and multiple types of specific feature points that indicate feature points corresponding to each of the multiple applicable vehicle types. Feature points can also be referred to as "key points." Both the general feature points and specific feature points may contain multiple feature points.

[0017] General feature points are feature points that can be interpreted uniformly for various vehicles. For example, general feature points include the four corners of the rectangular area representing the road surface occupied by the target vehicle 1, and the contact surface of the tires. Specific feature points are feature points optimized for the corresponding applicable vehicle. For example, specific feature points are feature points related to shapes and parts specific to the corresponding applicable vehicle.

[0018] The applicable vehicle types are the types of vehicles supported by the vehicle position estimation system 10, and may be suitably set according to the environment in which the vehicle position estimation system 10 is applied. For example, each of the multiple applicable vehicle types may be classified by the vehicle's body type (SUV, minivan, sedan, etc.) or by the vehicle's product name. Alternatively, each of the multiple applicable vehicle types may be classified by a predetermined classification number appropriately assigned according to the model, specifications, etc.

[0019] Figure 3 shows an example of feature points extracted by the feature point extraction unit P10 when there are two applicable vehicle types, vehicle type A and vehicle type B. In Figure 3, examples of feature points extracted from an image 2 where the target vehicle 1 is vehicle type A and examples of feature points extracted from an image 2 where the target vehicle 1 is vehicle type B are shown, respectively.

[0020] As shown in Figure 3, for both the image 2 where the target vehicle 1 is of type A and the image 2 where the target vehicle 1 is of type B, general feature points, specific feature points related to type A (type A specific feature points), and specific feature points related to type B (type B specific feature points) are extracted. Since the type A specific feature points are optimized for type A, they provide more accurate feature points for the image 2 where the target vehicle 1 is of type A. On the other hand, since the type B specific feature points are optimized for type B, they provide more accurate feature points for the image 2 where the target vehicle 1 is of type B.

[0021] In this embodiment, the feature point extraction unit P10 is composed of a pre-trained model generated by machine learning. In other words, the feature point extraction unit P10 uses the pre-trained model to extract general feature points and multiple types of unique feature points from the captured image 2. The pre-trained model is generated to take the captured image 2 as input and output general feature points and multiple types of unique feature points from the input captured image 2. The configuration and generation method of the pre-trained model will be described later.

[0022] Refer to Figure 2 again. The target unique feature point selection unit P20 acquires multiple types of extracted unique feature points. Then, the target unique feature point selection unit P20 selects a unique feature point suitable for the target vehicle 1 from among the multiple types of unique feature points (hereinafter referred to as the "target unique feature point"). In the first embodiment, the target unique feature point selection unit P20 acquires the vehicle type information of the target vehicle 1 and selects a unique feature point corresponding to the vehicle type of the target vehicle 1 as the target unique feature point. For example, when the target vehicle 1 is vehicle type A, the target unique feature point selection unit P20 selects a unique feature point related to vehicle type A from among the multiple types of unique feature points as the target unique feature point.

[0023] The vehicle position estimation unit P30 acquires the extracted general-purpose feature points and the selected target-specific feature points. The vehicle position estimation unit P30 also accesses the applicable vehicle database D10. The applicable vehicle database D10 is a database that manages data related to each of multiple applicable vehicle types. In this embodiment, the applicable vehicle database D10 includes data that allows for the identification of the coordinates of each feature point in the vehicle-based coordinate system for each applicable vehicle type. For example, the applicable vehicle database D10 manages coordinate data in the vehicle coordinate system for each feature point that has been pre-calculated from the CAD data of the vehicle body for each applicable vehicle type.

[0024] The vehicle position estimation unit P30 estimates the position of the target vehicle 1 based on the image coordinates of general-purpose feature points and target-specific feature points. The vehicle position estimation unit P30 can estimate the position of the target vehicle 1 as follows, for example.

[0025] The vehicle position estimation unit P30 can obtain the coordinates in the vehicle coordinate system for both general-purpose feature points and target-specific feature points by accessing the applicable vehicle database D10. For example, the vehicle position estimation unit P30 obtains coordinate data corresponding to the vehicle type of target vehicle 1 from the applicable vehicle database D10 and obtains the coordinates by matching the feature points. Here, given the position of target vehicle 1 in the world coordinate system, the coordinates in the world coordinate system of the general-purpose feature points and target-specific feature points can be determined from the coordinates in the vehicle coordinate system. Furthermore, the coordinates in the world coordinate system can be converted to coordinates in the image coordinate system by known coordinate transformations using the camera parameters of camera 200 (focal length, distortion correction parameters, camera position, camera orientation, etc.). In other words, the vehicle position estimation unit P30 can calculate the coordinates in the image coordinate system of the general-purpose feature points and target-specific feature points by assuming the position of target vehicle 1. In this case, the vehicle position estimation unit P30 may also calculate the coordinates in the image coordinate system by adding constraints (for example, setting the height to 0 as a constraint for the road surface position).

[0026] Therefore, the vehicle position estimation unit P30 can estimate the position of the target vehicle 1 by solving the task of minimizing the difference between the image coordinates of the acquired general-purpose feature points and target-specific feature points, and the image coordinates of the general-purpose feature points and target feature points calculated when the position of the target vehicle 1 is assumed, with the position of the target vehicle 1 as an unknown.

[0027] For example, consider the case where the position of target vehicle 1 is estimated in three dimensions (X, Y, θ). In this case, the vehicle position estimation unit P30 can estimate the position of target vehicle 1 by solving the task represented by the following equation (1). Here, (u i ,v i ) is the image coordinate of the acquired general feature point or the target specific feature point, (u i ',v i ')| (X,Y,θ) w is the image coordinate of a general feature point or a specific feature point calculated when the position of the target vehicle 1 is (X, Y, θ), w i w represents the weight for each feature point. i For example, small values ​​are assigned to general feature points, and large values ​​are assigned to eigenspecific feature points. Note that equation (1) below can also be applied when estimating the position of vehicle 1 in higher or lower dimensions. For example, the task can be similarly structured when estimating the position of vehicle 1 in six dimensions (X, Y, Z, yaw, pitch, roll).

number

[0028] As described above, the vehicle position estimation function according to this embodiment is configured. The information processing unit 100 outputs the estimated position of the target vehicle 1 estimated by the vehicle position estimation unit P30.

[0029] 1.3 Information Processing Section Figure 4 shows an example of the hardware configuration of the information processing unit 100. The information processing unit 100 includes one or more processors 110 (hereinafter simply referred to as processor 110), one or more storage devices (hereinafter simply referred to as storage devices 120), and a communication I / F 130.

[0030] The processor 110 performs various processes. The processor 110 can be composed of, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.

[0031] The storage device 120 stores various information necessary for the processor 110 to execute processing. The storage device 120 can be composed of recording media such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0032] The storage device 120 stores the computer program 121, the learned model 122, and the applicable vehicle database D10.

[0033] The computer program 121 is executed by the processor 110. Various processes by the information processing unit 100 may be realized through the cooperation of the processor 110, which executes the computer program 121, and the storage device 120. In particular, the feature point extraction unit P10, the target-specific feature point selection unit P20, and the vehicle position estimation unit P30 may be realized. The feature point extraction unit P10, the target-specific feature point selection unit P20, and the vehicle position estimation unit P30 may be realized by a single processor 110 or by separate processors 110. The computer program 121 may be recorded on a computer-readable recording medium. The processor 110 may configure the feature point extraction unit P10 by reading and using the trained model 122 from the storage device 120.

[0034] The communication interface 130 is an interface for connecting and communicating with the outside world of the information processing unit 100. Examples of communication interfaces 130 include devices for connecting to the internet and devices for connecting to mobile communication networks. The information processing unit 100 communicates with the camera 200 and the target vehicle 1 via the communication interface 130.

[0035] Figure 5 is a flowchart showing an example of a process performed by the information processing unit 100, more specifically, by the processor 110, based on the configuration described in "1.2 Vehicle Position Estimation Function". The process shown in the flowchart in Figure 5 starts, for example, when the information processing unit 100 acquires an image 2 from the camera 200.

[0036] In step S110, the processor 110 uses the trained model 122 to extract general feature points and multiple types of unique feature points from the captured image 2.

[0037] Next, in step S120, the processor 110 acquires vehicle type information for the target vehicle 1. The processor 110 may acquire vehicle type information directly by communicating with the target vehicle 1, or it may acquire vehicle type information by performing image recognition on the captured image 2. Alternatively, the processor 110 may acquire identification information for the target vehicle 1 by communicating with the target vehicle 1, and then acquire vehicle type information by referring to the applicable vehicle database D10 using the identification information.

[0038] Next, in step S130, the processor 110 selects a target unique feature point from among the extracted unique feature points, depending on the type of vehicle 1.

[0039] Next, in step S140, the processor 110 estimates the position of the target vehicle 1 based on the image coordinates of the extracted general feature points and the selected target-specific feature points. For example, the processor 110 estimates the position of the target vehicle 1 by solving the task represented by the above formula (1) using an extended Kalman filter, the method of Lagrange multipliers, iterative processing by a solver, etc. By executing the process according to step S140, the estimated position of the target vehicle 1 is obtained. After step S140, the process ends.

[0040] In this way, when the processor 110 executes the process, the vehicle position estimation function according to this embodiment is realized. Also, the vehicle position estimation program according to this embodiment is realized by the computer program 121 that causes the processor 110 to execute the process.

[0041] 1.4 Learned Model Hereinafter, the learned model 122 for extracting the feature points of the target vehicle 1 from the captured image 2 will be described. FIG. 6 is a diagram for explaining the configuration of the learned model 122.

[0042] The learned model 122 is composed of an upper layer 20, a general feature point extraction layer 31, and a plurality of specific feature point extraction layers corresponding to each of a plurality of applicable vehicle types.

[0043] The upper layer 20 takes the captured image 2 as input and outputs the feature amount of the captured image 2. The upper layer 20 can employ a CNN (Convolutional Neural Network), a Transformer model, or the like. In the learned model 122, the captured image 2 can also be considered as the "target image" for extracting feature points.

[0044] The general feature point extraction layer 31 takes the output of the upper layer 20 as input and outputs general feature points. In FIG. 6, {kp i} iThe output general-purpose feature points are shown. The general-purpose feature point extraction layer 31 is composed of, for example, a feature extraction unit for extracting features suitable for general-purpose feature point extraction from the output of the upper layer 20, and an output unit for outputting the image coordinates of general-purpose feature points from the output of the feature extraction unit. In this case, the general-purpose feature point extraction layer 31 can employ a CNN or Transformer model as the feature extraction unit. It can also employ an Affine layer or Softmax layer as the output unit.

[0045] Each of the multiple feature point extraction layers takes the output of the upper layer 20 as input and outputs a feature point corresponding to the applicable vehicle model. Figure 6 shows feature point extraction layer 32A and feature point extraction layer 32B. For example, feature point extraction layer 32A corresponds to vehicle model A and outputs a feature point {kpA i} i The output is generated. Additionally, the unique feature point extraction layer 32B corresponds to vehicle type B and outputs unique feature points related to vehicle type B {kpB i} i The output is as follows. Each of the multiple feature point extraction layers is composed of, for example, a feature extraction unit for extracting features suitable for extracting feature points corresponding to the applicable vehicle from the output of the upper layer 20, and an output unit for outputting the image coordinates of the feature points corresponding to the applicable vehicle from the output of the feature extraction unit. In particular, each feature point extraction layer may have equivalent configurations to one another.

[0046] Figure 6 shows two feature extraction layers 32A and 32B, but the number of feature extraction layers may include three or more. As can be understood from the above explanation, it is possible to construct a similar trained model 122 even when including three or more feature extraction layers.

[0047] As described above, the trained model 122 according to this embodiment is configured. The trained model 122 is generated in advance by machine learning. Figure 7 is a flowchart showing the method for generating the trained model 122. The processes related to the flowchart shown in Figure 7 are executed by the processor 110, for example, during training.

[0048] First, in step S210, the upper layer 20 and the general-purpose feature point extraction layer 31 are trained. The training of the upper layer 20 and the general-purpose feature point extraction layer 31 is performed using training data (hereinafter also referred to as "general-purpose training data") consisting of multiple images showing various vehicles without specifying a vehicle type. The training data may contain information on the correct feature points. For example, a large-scale dataset can be used as the general-purpose training data. The training of the upper layer 20 and the general-purpose feature point extraction layer 31 can be performed, for example, by backpropagation. At this time, the training of the upper layer 20 and the general-purpose feature point extraction layer 31 may be performed integrally.

[0049] By performing training using general-purpose training data, the upper layer 20 and the general-purpose feature point extraction layer 30 are expected to output results with high generalization performance for various vehicles. In particular, it is possible to prepare a large amount of training data as general-purpose training data, and highly proficiency-based learning can be expected.

[0050] Next, in step S220, each of the multiple eigenpoint extraction layers is trained. Each eigenpoint extraction layer is trained using training data consisting of multiple images showing the corresponding vehicle type. For example, the eigenpoint extraction layer for vehicle type A is trained using training data consisting of multiple images showing vehicle type A. Each eigenpoint extraction layer is similarly trained using backpropagation. At this time, training may be performed with the upper layer 20 and the eigenpoint extraction layer to be trained connected. However, in order to maintain the training results of the upper layer 20 using general-purpose training data, it is desirable to freeze the parameters related to the upper layer 20 during the training of each eigenpoint extraction layer.

[0051] As each of the multiple feature extraction layers undergoes training in this manner, it is expected that each feature extraction layer will output results optimized for the corresponding vehicle model. In particular, considering that the upper layer 20 is trained to a high degree of proficiency using general-purpose training data, each feature extraction layer can learn to a high degree of proficiency even with relatively little training data. Consequently, highly efficient learning is possible.

[0052] After step S220, the process ends. By executing the process in this manner, the method for generating the trained model 122 according to this embodiment is realized.

[0053] According to the pre-trained model 122 of this embodiment, it is easy to retrospectively expand the range of applicable vehicle types. For example, consider a case where there are currently two applicable vehicle types, vehicle type A and vehicle type B. Now consider the case where vehicle type C is added as an applicable vehicle type. In this case, the pre-trained model 122 can be configured by connecting a new feature point extraction layer related to vehicle type C to the upper layer 20. The feature point extraction layer related to vehicle type C may be a general-purpose feature point extraction layer 31 or another feature point extraction layer. Furthermore, the training of the pre-trained model 122 only needs to be performed on the feature point extraction layer related to vehicle type C. In this way, by using the pre-trained model 122 of this embodiment, a vehicle position estimation system 10 can be realized in which the range of applicable vehicle types can be easily expanded retrospectively.

[0054] 1.5 Effects As described above, according to this embodiment, since the position is estimated based on the image coordinates of specific feature points optimized for the vehicle type of the target vehicle 1, in addition to general feature points, a vehicle position estimation function with high estimation accuracy can be realized.

[0055] 2. Second Embodiment The vehicle position estimation system 10 according to the second embodiment will be described below. The vehicle position estimation system 10 according to the second embodiment provides a vehicle position estimation function, similar to the first embodiment. In the following description, the differences from the first embodiment will be explained in detail, and content common to the first embodiment will be omitted as appropriate.

[0056] 2.1 Configuration of the vehicle position estimation function Figure 8 shows an example of the configuration of the vehicle position estimation function according to the second embodiment. The vehicle position estimation function according to the second embodiment consists of a feature point extraction unit P11, a target-specific feature point selection unit P21, a vehicle position estimation unit P30, and a posture estimation unit P40. In Figure 8, elements common to the first embodiment are denoted by the same reference numerals.

[0057] The feature point extraction unit P11 extracts general-purpose feature points and multiple types of unique feature points from the captured image 2. In particular, in the second embodiment, each of the multiple types of unique feature points extracted by the feature point extraction unit P11 further includes multiple types of posture-specific feature points classified according to the vehicle's posture. Here, "vehicle posture" refers to the vehicle's posture as seen from the camera 200, indicating whether the vehicle is facing forward, backward, etc., in the captured image 2. The posture-specific feature points may include multiple feature points.

[0058] Attitude-specific feature points are feature points that are further optimized for the corresponding vehicle attitude from among the intrinsic feature points. For example, if attitude X indicates that the vehicle is facing forward in captured image 2, then the attitude-specific feature points related to attitude X are feature points related to shapes and parts that are clearly visible when the vehicle is facing forward.

[0059] The feature point extraction unit P11 is composed of a trained model 122, similar to the first embodiment. Therefore, in the trained model 122, each of the multiple unique feature point extraction layers is configured to output unique feature points that include multiple types of attitude-specific feature points. For example, the unique feature point extraction layer for vehicle type A outputs attitude-specific feature points {kpAX} related to attitude X. i} i , and posture-specific feature points related to posture Y {kpAYi} i It was configured to output [this].

[0060] The posture estimation unit P40 estimates the posture of the target vehicle 1. For example, the posture estimation unit P40 acquires extracted general-purpose feature points and estimates the posture of the target vehicle 1 based on the image coordinates of the general-purpose feature points. If the general-purpose feature points are the four corners of a rectangular area, the posture estimation unit P40 estimates the posture of the target vehicle 1 from the shape of the rectangular area represented by the general-purpose feature points, for example. The posture estimation unit P40 may also estimate the posture of the target vehicle 1 by image recognition of the captured image 2. Alternatively, for example, the posture estimation unit P40 may estimate the posture of the target vehicle 1 by tracking the target vehicle 1.

[0061] The target unique feature point selection unit P21 further acquires attitude information of the target vehicle 1 from the attitude estimation unit P40. In the second embodiment, the target unique feature point selection unit P21 selects a unique feature point corresponding to the vehicle type of the target vehicle 1 from among multiple types of unique feature points, and selects the attitude-specific feature point corresponding to the attitude of the target vehicle 1 from among multiple types of attitude-specific feature points included in the selected unique feature point as the target unique feature point. In other words, in the second embodiment, the target unique feature point is a feature point that matches both the vehicle type and attitude of the target vehicle 1.

[0062] 2.2 Information Processing Unit Figure 9 is a flowchart showing an example of a process performed by the information processing unit 100, or more specifically, by the processor 110, based on the configuration described in "2.1 Vehicle Position Estimation Function".

[0063] Steps S310 and S320 are equivalent to steps S110 and S120, respectively, as described in Figure 5. After step S320, the process proceeds to step S330.

[0064] In step S330, the processor 110 estimates the attitude of the target vehicle 1.

[0065] Next, in step S340, the processor 110 selects a target unique feature point from among the extracted unique feature points according to the vehicle type and posture of the target vehicle 1.

[0066] Next, in step S350, the processor 110 estimates the position of the target vehicle 1 based on the image coordinates of the extracted general feature points and the selected target-specific feature points. The estimated position of the target vehicle 1 is obtained by executing the process in step S350. After step S350, the process ends.

[0067] 2.3 Effects As described above, according to the second embodiment, the position is estimated based on the image coordinates of the unique feature points of the target vehicle 1, which are further optimized for the attitude of the target vehicle 1 compared to the first embodiment. Therefore, a vehicle position estimation function with even higher estimation accuracy can be realized.

[0068] 2.4 Variations The vehicle position estimation system 10 according to the second embodiment may adopt the following modified form.

[0069] In the modified example, the feature point extraction unit P11 is configured such that each of the extracted unique feature points of multiple types includes multiple feature points to which a confidence level that changes according to the vehicle's attitude is assigned, rather than multiple types of attitude-specific feature points. For example, when five feature points kpA1, kpA2, kpA3, kpA4, and kpA5 are extracted as unique feature points related to vehicle type A, a confidence level is assigned to each feature point as follows. The confidence level may be assigned to the class of each feature point using a pre-prepared map. In this case, the map may be generated from past position estimation results or compatibility tests, etc. [Table 1]

[0070] In the modified example, the target unique feature point selection unit P21 is configured to select unique feature points corresponding to the vehicle type of the target vehicle 1 from among multiple types of unique feature points, and further exclude feature points from the target unique feature points whose confidence level is below a threshold, based on the attitude of the target vehicle 1. For example, suppose the target vehicle 1 is vehicle type A, and the target unique feature point selection unit P21 selects the unique feature points related to vehicle type A as the target unique feature points. Also, suppose the threshold is 40%. In this case, the target unique feature point selection unit P21 executes a process to exclude kpA1 and kpA2 from the target unique feature points.

[0071] This enables the vehicle position estimation unit P30 to estimate the position of the target vehicle 1 based on feature points that have sufficient reliability regarding the attitude of the target vehicle 1. Furthermore, by adopting this modified configuration, it is possible to achieve the same effects as described above. [Explanation of Symbols]

[0072] 1 Target vehicle, 2 Captured image, 10 Vehicle position estimation system, 100 Information processing unit, 110 processors, 200 cameras

Claims

1. Camera and, The system includes one or more processors that estimate the position of a target vehicle as captured in an image taken by the camera. The one or more processors described above are: The process involves using a pre-trained model generated by machine learning to extract from the captured image general feature points that are independent of the vehicle type and multiple types of specific feature points that correspond to each of the multiple applicable vehicle types, A process to obtain information on the type of vehicle of the aforementioned target vehicle, The process of selecting a target unique feature point from among the multiple types of unique feature points according to the type of vehicle, A process for estimating the position of the target vehicle based on the image coordinates of the general feature points and the target-specific feature points, It is configured to execute Vehicle position estimation system.

2. A vehicle position estimation system according to claim 1, The aforementioned trained model is An upper layer that takes the aforementioned captured image as input, A general-purpose feature point extraction layer that takes the output of the upper layer as input and outputs the general-purpose feature points, Multiple unique feature point extraction layers, each corresponding to one of the aforementioned multiple applicable vehicle models, and each receiving the output of the upper layer as input, output unique feature points corresponding to the applicable vehicle model, Composed of, The aforementioned upper layer and the general-purpose feature point extraction layer are trained using training data consisting of multiple images showing various vehicles without specifying a vehicle type. Each of the aforementioned multiple feature point extraction layers is trained using training data consisting of multiple images showing the corresponding vehicle of the adapted type. Characterized by Vehicle position estimation system.

3. A vehicle position estimation system according to claim 1 or 2, Each of the aforementioned multiple types of unique feature points includes multiple types of posture-specific feature points classified according to the vehicle's posture. The one or more processors are configured to further perform processing to acquire information about the attitude of the target vehicle. The process of selecting the aforementioned unique feature points of the target is as follows: Selecting a characteristic feature point from among the aforementioned multiple types of characteristic feature points that corresponds to the type of vehicle in question, Selecting the attitude-specific feature points corresponding to the attitude of the target vehicle from among the multiple types of attitude-specific feature points included in the selected unique feature points, including Characterized by Vehicle position estimation system.

4. A vehicle position estimation system according to claim 1 or 2, Each of the aforementioned multiple types of unique feature points includes multiple feature points to which a reliability level that changes according to the vehicle's attitude is given. The one or more processors are configured to further perform processing to acquire information about the attitude of the target vehicle. Selecting the aforementioned unique feature points of the target means Selecting from among the multiple types of characteristic features that correspond to the type of vehicle in question as the target characteristic feature, Based on the attitude of the vehicle in question, feature points whose reliability is below a threshold are removed from the characteristic feature points of the vehicle in question. including Characterized by Vehicle position estimation system.

5. A method for generating a trained model that enables a computer to function in order to extract feature points of a vehicle shown in a target image, The aforementioned trained model is An upper layer that takes the aforementioned target image as input, A general-purpose feature point extraction layer that takes the output of the upper layer as input and outputs the feature points of the vehicle, Multiple unique feature point extraction layers, each corresponding to a different vehicle model, and each layer taking the output of the upper layer as input to output the characteristic points of the vehicle, It consists of, The upper layer and the general-purpose feature point extraction layer are trained using training data consisting of multiple images showing various vehicles without specifying a particular type. The process involves training each of the multiple unique feature point extraction layers using training data consisting of multiple images showing vehicles of the corresponding applicable vehicle type, including Generation method.

Images