Detection device, position calculation system, and detection method

Abstract

To provide a technique for accurately detecting a vehicle included in a captured image.SOLUTION: A detection device that detects a vehicle included in a captured image includes: an image acquisition unit that acquires the captured image; a type acquisition unit that acquires type information indicating one body type of the vehicle included in the captured image, the one body type being classified according to an external shape of the vehicle; a model acquisition unit that acquires, from among a plurality of first detection models each of which is a machine learning model prepared for each of the one body type, a first detection model selected according to the one body type identified by the type information acquired by the type acquisition unit; and a detection unit that detects the vehicle included in the captured image by inputting the captured image to the first detection model acquired by the model acquisition unit and identifying a target region indicating the vehicle in the captured image.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a detection device, a position calculation system, and a detection method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, vehicles that run autonomously under remote control are known (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to make a vehicle automatically travel by remote control, an image of the vehicle may be captured from outside. However, the inventors of the present application have found that since the external shape of the vehicle differs depending on the body type, there is a risk that the detection accuracy of the vehicle included in the captured image may decrease. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first embodiment of the present disclosure, a detection device is provided. The detection device detects a vehicle included in a captured image, and includes an image acquisition unit that acquires the captured image, a type acquisition unit that acquires type information indicating one body type of the vehicle included in the captured image, the type information being one body type of the vehicle classified according to the external shape of the vehicle, a model acquisition unit that acquires the first detection model selected from a plurality of first detection models that are machine learning models prepared for each of the body types according to the one body type identified by the type information acquired by the type acquisition unit, and a detection unit that inputs the captured image to the first detection model acquired by the model acquisition unit and identifies a target area indicating the vehicle in the captured image, thereby detecting the vehicle included in the captured image. According to this embodiment, the type information indicating one body type of the vehicle included in the captured image can be acquired, and the first detection model selected from a plurality of first detection models prepared for each of the body types according to the one body type identified by the acquired type information can be acquired. Then, the captured image can be input to the first detection model corresponding to the body type of the vehicle included in the captured image, thereby accurately detecting the vehicle included in the captured image. In other words, by using the first detection model suitable for detecting vehicles classified into one body type, each area constituting the captured image can be accurately classified into a target area and an area other than the target area. As a result, it is possible to prevent the detection accuracy of the vehicle included in the captured image from decreasing depending on the body type of the vehicle. (2) In the above embodiment, each of the plurality of first detection models may be trained in advance to identify the target region by inputting a plurality of training images including M (M is an integer of 2 or more) first training images each including the vehicle classified into one of the body types, and N (N is an integer of 0 or more and less than M) second training images each including the vehicle classified into another of the body types different from the one body type. According to this embodiment, by training more first training images each including the vehicle classified into one body type than second training images each including the vehicle classified into another of the body types different from the one body type, a first detection model suitable for detecting vehicles classified into one body type can be prepared for each of the one body types. (3) According to a second embodiment of the present disclosure, a detection device is provided. The detection device detects a vehicle included in a captured image, and includes an image acquisition unit that acquires the captured image, a type acquisition unit that acquires type information indicating one body type of the vehicle included in the captured image, the type information being one body type of the vehicle classified according to the external shape of the vehicle, and a detection unit that inputs the captured image and the one body type identified by the type information acquired by the type acquisition unit into a second detection model that is a machine learning model, and detects the vehicle included in the captured image by identifying a target area indicating the vehicle in the captured image. According to this embodiment, the captured image associated with the one body type is input into the second detection model, so that the vehicle included in the captured image can be detected with high accuracy. In other words, each area constituting the captured image can be classified with high accuracy into a target area and an area other than the target area. As a result, it is possible to suppress a decrease in detection accuracy of the vehicle included in the captured image according to the body type of the vehicle. (4) In the above embodiment, the second detection model may be trained in advance to identify the target region by inputting a plurality of training images including the vehicle and a type correct label associated with each of the plurality of training images, the type correct label indicating the body type of the vehicle included in the training image. According to this embodiment, it is possible to prepare a second detection model trained in association with a plurality of training images including a vehicle and a type correct label indicating the body type of the vehicle included in each training image. (5) In the above embodiment, a region answer label indicating whether each region in the training image is the target region or a non-target region indicating an area other than the vehicle may be associated with each region in the training image. According to this embodiment, each region constituting the captured image can be accurately classified into a target region and a non-target region. This can improve the detection accuracy of vehicles included in the captured image. (6) In the above embodiment, the body type may indicate a type of the vehicle. According to this embodiment, a vehicle included in a captured image can be detected using either a first detection model or a second detection model that has learned differences in the external shapes of vehicles that differ depending on the type of vehicle. This can improve the detection accuracy of a vehicle included in a captured image. (7) In the above aspect, the vehicle may travel by receiving from outside the vehicle travel conditions that define the travel operation of the vehicle, and the type acquisition unit may acquire the type information by identifying the one body type of the vehicle included in the captured image using schedule information that is associated with each of the imaging ranges of a plurality of imaging devices that acquire the captured images and indicates a scheduled time when the vehicle classified into the one body type is scheduled to travel through the imaging range. According to this aspect, the type information about the vehicle included in the captured image can be acquired by identifying the one body type of the vehicle included in the captured image using the schedule information. (8) In the above aspect, the type acquisition unit may acquire the type information by inputting the captured image to a specific model that is a machine learning model trained to output the type information when the captured image is input. According to this aspect, the type information about the vehicle included in the captured image can be acquired by inputting the captured image to a specific model trained to output type information about the vehicle included in the captured image. (9) In the above aspect, the present invention may further include a rotation processing unit that rotates the captured image so that the moving direction of the vehicle faces a predetermined direction. According to this aspect, the captured image can be rotated so that the direction of a vector indicating the moving direction of the vehicle faces a predetermined direction. In this way, it is possible to detect a vehicle included in the captured image with the direction of the vector indicating the moving direction of the vehicle being unified. This makes it possible to improve the detection accuracy of the vehicle included in the captured image. (10) In the above aspect, a distortion correction unit may be further provided that corrects distortion of the captured image. According to this aspect, it is possible to correct distortion of the captured image, thereby improving detection accuracy of a vehicle included in the captured image. (11) According to a third aspect of the present disclosure, there is provided a position calculation system. The position calculation system for calculating a position of a vehicle included in a captured image includes the detection device according to the above aspect, and a position calculation device for calculating a position of the vehicle, the detection device further generates a first mask image by masking a target area, which is an area indicating the vehicle, in the captured image and adding a mask area to the target area, the position calculation device includes a perspective transformation unit that performs perspective transformation on the first mask image to generate a second mask image, and a coordinate point calculation unit that sets a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point and a coordinate point calculation unit that calculates a position of the second mask image by performing perspective transformation on the first mask image. The present invention includes a coordinate point calculation unit that calculates an image coordinate point indicating the position of the vehicle in the image coordinate system by setting a vertex indicating the same position as the first coordinate point as a second coordinate point among the second circumscribing rectangle set in the mask area in the mask image as a second coordinate point and correcting the first coordinate point using the second coordinate point, and a position conversion unit that converts the image coordinate point into a vehicle coordinate point indicating the position of the vehicle in the global coordinate system using a distance from a reference point of the imaging device calculated based on the position of the imaging device in the global coordinate system and a distance from the reference point of a predetermined positioning point of the vehicle. According to this embodiment, it is possible to generate a first mask image in which a target area indicating the vehicle is masked among the areas constituting the captured image, and a second mask image obtained by perspectively converting the first mask image. In this way, it is possible to calculate the image coordinate point by extracting the first coordinate point from the first mask image and the second coordinate point from the second mask image. This makes it possible to calculate the image coordinate point more accurately. Therefore, it is possible to improve the calculation accuracy of the vehicle position. (12) According to a fourth aspect of the present disclosure, a detection method is provided. The detection method for detecting a vehicle included in a captured image includes an image acquisition step of acquiring the captured image, a type acquisition step of identifying the body type of the vehicle included in the captured image, the body type of the vehicle being classified according to the external shape of the vehicle, a model acquisition step of acquiring the first detection model selected according to the body type identified in the type acquisition step from among a plurality of first detection models which are machine learning models prepared for one of the body types, and a detection step of detecting the vehicle included in the captured image by inputting the captured image to the first detection model acquired in the model acquisition step and identifying a target area indicating the vehicle in the captured image. According to this aspect, type information indicating one body type of the vehicle included in the captured image is acquired, and a first detection model selected according to one body type identified by the acquired type information can be acquired from a plurality of first detection models prepared for one body type. Then, the vehicle included in the captured image can be accurately detected by inputting the captured image to the first detection model corresponding to the body type of the vehicle included in the captured image. In other words, by using the first detection model suitable for detecting vehicles classified into one body type, each area constituting the captured image can be accurately classified into a target area and an area other than the target area. As a result, it is possible to prevent the detection accuracy of the vehicle included in the captured image from decreasing depending on the body type of the vehicle. (13) According to a fifth embodiment of the present disclosure, a detection method is provided. The detection method for detecting a vehicle included in a captured image includes an image acquisition step of acquiring the captured image, a type acquisition step of acquiring type information indicating one body type of the vehicle included in the captured image, the type information being one body type of the vehicle classified according to the external shape of the vehicle, and a detection step of inputting the captured image and the one body type identified by the type information acquired in the type acquisition step into a second detection model that is a machine learning model, and detecting the vehicle included in the captured image by identifying a target area indicating the vehicle in the captured image. According to this embodiment, the vehicle included in the captured image can be detected with high accuracy by inputting the captured image associated with one body type into the second detection model. That is, each area constituting the captured image can be classified with high accuracy into a target area and an area other than the target area. As a result, it is possible to suppress a decrease in detection accuracy of the vehicle included in the captured image according to the body type of the vehicle. The present disclosure can be realized in various forms other than the above-mentioned detection device, position calculation system, and detection method, for example, in the form of a manufacturing method of the detection device and the position calculation system, a position calculation method, a control method of the detection device and the position calculation system, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a position calculation system. [Diagram 2] FIG. 1 is a diagram showing a schematic configuration of a detection device according to a first embodiment. [Diagram 3] FIG. 2 is a diagram showing details of a CPU mounted on the detection device in the first embodiment. [Figure 4] FIG. 1 is a diagram showing a schematic configuration of a position calculation device. [Diagram 5] FIG. 2 is a diagram showing a schematic configuration of a remote control device. [Figure 6] 5 is a flowchart showing a detection method and a position calculation method in the first embodiment. [Figure 7] Schematic diagrams showing examples of various images. [Figure 8] FIG. 11 is a diagram for explaining details of a coordinate point calculation step. [Figure 9] FIG. 4 is a diagram for explaining a method of acquiring a base coordinate point. [Figure 10] FIG. 1 is a diagram for explaining details of a position conversion process. [Figure 11] FIG. 2 for explaining the details of the position conversion process. [Figure 12] 4 is a flowchart showing a vehicle operation control method. [Figure 13] FIG. 13 is a diagram showing a schematic configuration of a detection device according to a second embodiment. [Figure 14] FIG. 11 is a diagram showing details of a CPU mounted on a detection device according to a second embodiment. [Figure 15] 10 is a flowchart showing a detection method and a position calculation method in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: A-1. Configuration of position calculation system: FIG. 1 is a diagram showing a schematic configuration of a position calculation system 1. The position calculation system 1 is a system that detects a vehicle 10 included in a captured image and calculates the position of the vehicle 10. In the present disclosure, the captured image is at least one of an original image as raw data acquired by an imaging device 9 as an external device provided at a location different from the vehicle 10, a corrected image obtained by processing the original image, a rotated image, and a processed image. The corrected image, the rotated image, and the processed image will be described in detail later. The position calculation system 1 includes one or more vehicles 10, a detection device 5 that detects the vehicle 10 included in the captured image, and a position calculation device 6 that calculates the position of the vehicle 10 using the detection result by the detection device 5.

[0009] The imaging device 9 captures an image of an imaging range RG including the vehicle 10 to be subjected to position calculation from outside the vehicle 10 to obtain an original image. In this embodiment, the imaging device 9 transmits the obtained original image to the detection device 5 together with camera identification information and the acquisition time of the original image. The camera identification information is information such as a camera ID for identifying a plurality of imaging devices 9. The original image is a two-dimensional image in which pixels arranged on the XcYc plane of the camera coordinate system are collected. The camera coordinate system is a coordinate system having the focal point of the imaging device 9 as the origin and coordinate axes indicated by the Xc axis and the Yc axis perpendicular to the Xc axis. The original image includes at least two-dimensional data of the vehicle 10 to be subjected to position calculation. The original image is preferably a color image, but may be a gray image. The imaging device 9 is, for example, a camera having an imaging element such as a CCD image sensor or a CMOS image sensor and an optical system.

[0010] In this embodiment, the imaging device 9 acquires an original image of the lane 2 and the vehicle 10 traveling on the lane 2, viewed from above. In order to capture the entire lane 2 by one or more imaging devices 9, the installation positions and the number of imaging devices 9 are determined in consideration of the imaging range RG (angle of view) of the imaging device 9. Specifically, each imaging device 9 is installed so that the first imaging range RG1, which is the imaging range RG of the adjacent first imaging device 901, and the second imaging range RG2, which is the imaging range RG of the second imaging device 902, overlap. Furthermore, each imaging device 9 is installed at a position where it can capture an image of a positioning point 10e that is set in advance for a specific part of the vehicle 10 traveling on the lane 2. In this embodiment, the positioning point 10e is the rear end of the left side of the vehicle 10 (hereinafter, the left rear end). Note that the positioning point 10e may be other than the left rear end of the vehicle 10. Further, the imaging device 9 is not limited to acquiring information from above the vehicle 10, but may acquire information from the front, rear, sides, etc. of the vehicle 10.

[0011] The vehicle 10 has a manned driving mode and a remote unmanned driving mode. In the manned driving mode, the driver in the vehicle 10 operates input devices such as a steering wheel and an accelerator provided on the vehicle 10 to generate driving conditions for the vehicle 10. As a result, the vehicle 10 runs according to the generated driving conditions. The driving conditions are conditions that specify the driving operation of the vehicle 10. The driving conditions include, for example, the driving route, position, driving speed, acceleration, and steering angle of the wheels of the vehicle 10. The remote unmanned driving mode has a remote manual driving mode and a remote automatic driving mode. In the remote manual driving mode, the operator operates an operator input device provided at a location different from the vehicle 10 to generate driving conditions for the vehicle 10. As a result, the vehicle 10 receives the driving conditions generated by the operator input device and runs according to the received driving conditions. In the remote automatic driving mode, a remote control device 7 such as a server provided at a location different from the vehicle 10 creates control values ​​that specify the driving operation of the vehicle 10 and transmits them to the vehicle 10. As a result, the vehicle 10 receives the control value and performs automatic driving according to the received control value.

[0012] The vehicle 10 travels in a remote automatic driving mode, for example, in a factory where the vehicle 10 is manufactured by executing a plurality of manufacturing processes. The factory is not limited to being located in one building, or at one site or address, but may be located across multiple buildings, multiple sites, multiple addresses, etc. In this case, the vehicle 10 may travel on public roads, not limited to private roads. The vehicle 10 may also travel outside the factory in the remote unmanned driving mode.

[0013] The vehicle 10 may be, for example, an electric vehicle, a hybrid vehicle, a fuel cell vehicle, a gasoline vehicle, or a diesel vehicle. The vehicle 10 may be a private vehicle such as a passenger car, or a commercial vehicle such as a truck, a bus, or a construction vehicle. The vehicle 10 may be at least one of a finished product, a semi-finished product, and a work-in-progress product.

[0014] The vehicle 10 includes a driving device 110 that accelerates the vehicle 10, a steering device 120 that changes the traveling direction of the vehicle 10, and a braking device 130 that decelerates the vehicle 10. The vehicle 10 further includes a vehicle communication unit 140 for communicating with external devices using wireless communication or the like, a vehicle control device 150, and an in-vehicle sensor group 160. The external devices are devices other than the vehicle 10, such as the detection device 5, the position calculation device 6, the remote control device 7, and the imaging device 9, and other vehicles 10. The vehicle communication unit 140 is, for example, a wireless communication device. The vehicle communication unit 140 communicates with external devices connected to the network Nt, for example, via an access point in a factory. The vehicle control device 150 includes a CPU, a storage unit, and an input / output interface. In the vehicle control device 150, the CPU, the storage unit, and the input / output interface are connected to each other, for example, via an internal bus and an interface circuit. The input / output interface communicates with internal devices such as the driving device 110 mounted on the vehicle 10. The input / output interface is communicatively connected to the vehicle communication unit 140. The on-board sensor group 160 includes, for example, an on-board camera, an on-board radar, and an on-board lidar as sensors that acquire surrounding information indicating the state of the surrounding area of ​​the vehicle 10. The on-board camera captures images of the state of the surrounding area of ​​the vehicle 10. The on-board radar and the on-board lidar detect objects present in the surrounding area of ​​the vehicle 10. Note that the configuration of the vehicle 10 is not limited to the above.

[0015] 2 is a diagram showing a schematic configuration of the detection device 5 in the first embodiment. The detection device 5 detects at least the outer shape (outline) of the vehicle 10 included in the captured image by identifying a target area, which is an area indicating the vehicle 10, among the areas in the captured image. The detection device 5 includes a communication unit 51, a storage unit 53, and a CPU 52. In the detection device 5, the communication unit 51, the storage unit 53, and the CPU 52 are connected to each other via, for example, an internal bus and an interface circuit.

[0016] The communication unit 51 of the detection device 5 communicably connects the detection device 5 to other devices such as the vehicle control device 150, the position calculation device 6, the remote control device 7, and the imaging device 9. The communication unit 51 of the detection device 5 is, for example, a wireless communication device.

[0017] The storage unit 53 of the detection device 5 stores various information including various programs for controlling the operation of the detection device 5, a first detection model Md1 as a detection model, a specific model Md3, and a distortion correction parameter Pa1. The storage unit 53 of the detection device 5 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0018] The detection model is a trained machine learning model used to detect the vehicle 10 included in the captured image. In this embodiment, the storage unit 53 of the detection device 5 stores, as the detection model, a plurality of first detection models Md1 prepared for each body type. In this embodiment, the body type is a group of vehicle types when a plurality of types of the vehicle 10 are classified together according to the external shape of the vehicle 10. In this case, the vehicle 10 is classified into one body type according to, for example, the vehicle rank (also called "vehicle body" or "body size") determined by the overall length, width, and height of the vehicle 10, and the shape of the vehicle 10. In this embodiment, the body types are, for example, "SUV", "sedan", "station wagon", "minivan", "one box", "compact car", and "light car". The types of body types are not limited to those described above.

[0019] The first detection model Md1 is a machine learning model trained by inputting a first teacher data set. The first teacher data set is prepared for each body type. The first teacher data set has a plurality of training images including the vehicle 10, and a region correct answer label associated with each of a plurality of regions constituting each training image. The region correct answer label is a correct answer label indicating whether each region in the training image is a target region indicating the vehicle 10 or a non-target region indicating other than the vehicle 10. In this embodiment, each region constituting the training image is one pixel constituting the training image. Note that each region constituting the training image may be a plurality of pixels constituting the training image. Each region constituting the training image is classified into either a target region or a non-target region according to a calculation result when, for example, RGB values ​​of adjacent regions constituting the training image are compared. Each region constituting the training image may be classified into either a target region or a non-target region according to the calculated probability by calculating the probability that it is a target region. When a region correct answer label is attached to each region constituting the training image, information indicating the position in the training image of the region to which the region correct answer label is attached is associated. When each region constituting the training image is one pixel, the information indicating the position in the training image is, for example, pixel coordinates indicating the pixel position in the training image.

[0020] The first teacher data set includes, as a plurality of training images, M (M is an integer of 2 or more) first training images and N (N is an integer of 0 or more and less than M) second training images. The first training images are images including vehicles 10 classified into one body type. The second training images are images including vehicles 10 classified into other body types different from the one body type. The number M of first training images included in the first teacher data set may be, for example, 1.5 times or more the number N of second training images, or may be 2 times or more. In addition, when there are a plurality of other body types different from the one body type, the number M of first training images included in the first teacher data set is, for example, greater than the total number of second training images including vehicles 10 classified into a plurality of other body types different from the one body type. In other words, the first detection model Md1 is a machine learning model that preferentially learns the feature amount of the vehicle 10 classified into one body type by learning more first training images than second training images. FIG. 2 illustrates the following three first detection models Md1a to Md1c as examples of the first detection model Md1. The first first detection model Md1a is a first detection model Md1 that has been trained on more vehicles 10 classified as "SUV" than on vehicles 10 classified as body types other than "SUV". The second first detection model Md1b is a first detection model Md1 that has been trained on more vehicles 10 classified as "sedan" than on vehicles 10 classified as body types other than "sedan". The third first detection model Md1c is a first detection model Md1 that has been trained on more vehicles 10 classified as "station wagon" than on vehicles 10 classified as body types other than "station wagon". The number and types of the first detection models Md1 are not limited to those described above.

[0021] In this embodiment, when a captured image is input, the first detection model Md1 identifies a target area among the areas constituting the input captured image. Then, the first detection model Md1 masks the target area to generate a first mask image in which a mask area is added to the target area.

[0022] The algorithm of the first detection model Md1 is, for example, a deep neural network (hereinafter, DNN) having a structure of a convolutional neural network (hereinafter, CNN) that realizes semantic segmentation and instance segmentation. Note that the configuration of the first detection model Md1 is not limited to the above. The first detection model Md1 may be, for example, a trained machine learning model that uses an algorithm other than a neural network.

[0023] The specific model Md3 is a trained machine learning model used to identify a body type of the vehicle 10 included in the captured image and acquire type information indicating the identified body type. Specifically, the specific model Md3 is a machine learning model trained to output type information about the vehicle 10 included in the captured image when the captured image is input. The specific model Md3 learns feature amounts according to the body type in order to identify a body type of the vehicle 10. The feature amounts according to the body type are, for example, the shape and the class of the vehicle 10. For example, CNN is used as the algorithm of the specific model Md3. Note that the configuration of the specific model Md3 is not limited to the above. The specific model Md3 may be a trained machine learning model using an algorithm other than a neural network, for example.

[0024] The distortion correction parameter Pa1 is a parameter used when correcting the distortion of a captured image. The distortion correction parameter Pa1 will be described in detail later.

[0025] 3 is a diagram showing details of the CPU 52 mounted on the detection device 5 in the first embodiment. The CPU 52 of the detection device 5 deploys various programs stored in the storage unit 53 of the detection device 5, thereby functioning as an image acquisition unit 521, a distortion correction unit 522, a rotation processing unit 523, a crop processing unit 524, and a type acquisition unit 525. Furthermore, the CPU 52 of the detection device 5 deploys various programs stored in the storage unit 53 of the detection device 5, thereby functioning as a model acquisition unit 526, a detection unit 527, and a first transmission unit 528.

[0026] The image acquisition unit 521 acquires an original image from the imaging device 9 serving as an external device.

[0027] The distortion correction unit 522 corrects distortion of the captured image. In this embodiment, the distortion correction unit 522 generates a corrected image by correcting the distortion of the original image.

[0028] The rotation processing unit 523 rotates the captured image so that the direction of a vector indicating the moving direction of the vehicle 10 (hereinafter, the moving vector) faces a predetermined direction. In this embodiment, the rotation processing unit 523 generates a rotated image by rotating the corrected image.

[0029] The crop processing unit 524 deletes from the captured image, among the regions constituting the captured image, regions other than the region consisting of the vehicle 10 and the region surrounding the vehicle 10 (hereinafter, referred to as the necessary region) (hereinafter, referred to as the unnecessary region). In this way, the crop processing unit 524 cuts out the necessary region from the captured image. In this embodiment, when the vehicle 10 has moved a distance exceeding a predetermined threshold, the crop processing unit 524 deletes from the rotated image the moved region which corresponds to the distance traveled by the vehicle 10 and which is an unnecessary region. In this way, the crop processing unit 524 generates a processed image in which the unmoved region including the vehicle 10 and which is a necessary region has been cut out from the rotated image.

[0030] The type acquisition unit 525 acquires type information indicating one body type of the vehicle 10 included in the captured image. In this embodiment, the type acquisition unit 525 acquires type information indicating one body type of the vehicle 10 included in the captured image by inputting a processed image for the specific model Md3. Note that the method of acquiring the type information is not limited to the above.

[0031] The model acquisition unit 526 acquires the first detection model Md1 selected according to one body type identified by the type information acquired by the type acquisition unit 525. In this embodiment, the model acquisition unit 526 selects and acquires the first detection model Md1 according to one body type identified by the type information acquired by the type acquisition unit 525 from among a plurality of first detection models Md1 prepared for each body type stored in the storage unit 53 of the detection device 5. In other words, the model acquisition unit 526 acquires the first detection model Md1 that has been trained to learn more features of the vehicle 10 classified into one body type identified by the type information acquired by the type acquisition unit 525 than features of the vehicle 10 classified into other body types different from the one body type. For example, when the body type of the vehicle 10 included in the captured image is identified as “SUV” based on the type information acquired by the type acquisition unit 525, the model acquisition unit 526 acquires a first detection model Md1a that has been trained more on vehicles 10 classified as “SUV” than on vehicles 10 classified as other body types.

[0032] The detection unit 527 detects the vehicle 10 included in the captured image by inputting the captured image to the first detection model Md1 acquired by the model acquisition unit 526 and identifying a target area among the areas constituting the captured image. In this embodiment, the detection unit 527 inputs the processed image to the first detection model Md1 acquired by the model acquisition unit 526. As a result, the detection unit 527 generates a first mask image from the processed image by identifying a target area among the areas constituting the processed image.

[0033] The first transmission unit 528 transmits various information to devices other than the detection device 5. The first transmission unit 528 transmits, for example, a detection result by the detection unit 527 to the position calculation device 6. In this embodiment, the detection result by the detection unit 527 is a first mask image. Note that at least a part of the functions of the detection device 5 may be realized as one function of any of the vehicle control device 150, the position calculation device 6, the remote control device 7, and the imaging device 9.

[0034] FIG. 4 is a diagram showing a schematic configuration of the position calculation device 6. The position calculation device 6 calculates the position of the vehicle 10 included in the captured image using the detection result by the detection device 5. The position calculation device 6 adopts the positioning point 10e of the vehicle 10 as the position of the vehicle 10. In this embodiment, the position calculation device 6 calculates the position of the vehicle 10 included in the captured image using the first mask image. The position calculation device 6 includes a communication unit 61, a storage unit 63, and a CPU 62. In the position calculation device 6, the communication unit 61, the storage unit 63, and the CPU 62 are connected to each other via, for example, an internal bus and an interface circuit.

[0035] The communication unit 61 of the position calculation device 6 communicably connects the vehicle control device 150, the detection device 5, the remote control device 7, and the imaging device 9 to the position calculation device 6. The communication unit 61 of the position calculation device 6 is, for example, a wireless communication device.

[0036] The storage unit 63 of the position calculation device 6 stores various information including various programs for controlling the operation of the position calculation device 6, the perspective transformation parameters Pa2, and the camera database Db. The storage unit 63 of the position calculation device 6 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0037] The perspective transformation parameter Pa2 is a parameter used when performing perspective transformation on the first mask image. The details of the perspective transformation parameter Pa2 will be described later.

[0038] The camera database Db is a database indicating, for each imaging device 9, imaging parameters calculated based on the installation position of the imaging device 9 in a global coordinate system. The imaging parameters are parameters related to the distance of the imaging device 9 from a predetermined reference point. In this embodiment, the imaging parameter is the height H of the imaging device 9 from the road surface 20 (FIG. 10 described later), that is, the height H with respect to the road surface 20 on which the vehicle 10 is located. The camera database Db is a database that associates, for example, camera identification information with imaging parameters for the imaging device 9 identified by the camera identification information. Note that the configuration of the camera database Db is not limited to the above. The imaging range RG, installation position, installation angle, etc. of the imaging device 9 may be associated with the camera identification information.

[0039] The CPU 62 of the position calculation device 6 functions as a data acquisition unit 621, a perspective transformation unit 622, a coordinate point calculation unit 623, a position transformation unit 624, and a second transmission unit 625 by deploying various programs stored in the memory unit 63 of the position calculation device 6.

[0040] The data acquisition unit 621 acquires various information. For example, the data acquisition unit 621 acquires a detection result by the detection device 5 from the detection device 5. In this embodiment, the data acquisition unit 621 acquires a first mask image from the detection device 5. In addition, the data acquisition unit 621 refers to the camera database Db stored in the storage unit 63 of the position calculation device 6 to acquire imaging parameters for the imaging device 9 that is the source of the original image to be analyzed.

[0041] The perspective transformation unit 622 generates a second mask image by performing perspective transformation on the first mask image. The coordinate point calculation unit 623 corrects the first coordinate point using the second coordinate point to calculate an image coordinate point. The first coordinate point is a coordinate point in the image coordinate system of a specified vertex of a first circumscribing rectangle set in a mask area in the first mask image. The second coordinate point is a coordinate point in the image coordinate system of a vertex indicating the same position as the first coordinate point, among the vertices of a second circumscribing rectangle set in a mask area in the second mask image. The image coordinate point is a coordinate point indicating the position of the vehicle 10 in the image coordinate system. The position transformation unit 624 transforms the image coordinate point into a vehicle coordinate point using the imaging parameters acquired by the data acquisition unit 621 and the image coordinate point calculated by the coordinate point calculation unit 623. The vehicle coordinate point is a coordinate point indicating the position of the vehicle 10 in the global coordinate system.

[0042] The second transmission unit 625 transmits various information to devices other than the position calculation device 6. For example, the second transmission unit 625 transmits a vehicle coordinate point to the remote control device 7 as information indicating the position of the vehicle 10. Note that at least a part of the function of the position calculation device 6 may be realized as one function of any of the vehicle control device 150, the detection device 5, the remote control device 7, and the imaging device 9.

[0043] FIG. 5 is a diagram showing a schematic configuration of the remote control device 7. The remote control device 7 uses the position information of the vehicle 10 and the like to create a control value that specifies the traveling operation of the vehicle 10 and transmits it to the vehicle 10. In this way, the remote control device 7 remotely controls the operation of the vehicle 10. The remote control device 7 includes a communication unit 71, a storage unit 73, and a CPU 72. In the remote control device 7, the communication unit 71, the storage unit 73, and the CPU 72 are connected to each other, for example, via an internal bus and an interface circuit. The communication unit 71 of the remote control device 7 connects the vehicle control device 150, the detection device 5, the position calculation device 6, and the imaging device 9 to the remote control device 7 so that they can communicate with each other. The communication unit 71 of the remote control device 7 is, for example, a wireless communication device. The storage unit 73 of the remote control device 7 stores various programs that control the operation of the remote control device 7. The storage unit 53 of the remote control device 7 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0044] The CPU 72 of the remote control device 7 deploys various programs stored in the storage unit 73 of the remote control device 7, thereby functioning as an information acquisition unit 721, a control value creation unit 722, and a third transmission unit 723.

[0045] The information acquisition unit 721 acquires various information. For example, the information acquisition unit 721 acquires information related to the driving conditions of the vehicle 10 (hereinafter, driving information). The driving information includes, for example, vehicle coordinate points transmitted from the position calculation device 6 as information indicating the position of the vehicle 10, the driving speed and actual steering angle of the vehicle 10 transmitted from the vehicle control device 150, and driving route information stored in advance in the storage unit 73 of the remote control device 7. The driving route information is information indicating a target driving route of the vehicle 10 traveling in the remote automatic driving mode. Note that the types of information included in the driving information are not limited to those described above.

[0046] The control value creation unit 722 creates a control value that specifies the driving operation of the vehicle 10 by using the driving information acquired by the information acquisition unit 721. Specifically, the control value creation unit 722 creates, for example, a reference control value and a correction control value. The reference control value is a control value for driving the vehicle 10 along a target driving route. The correction control value is a control value that corrects the relative position of the vehicle 10 with respect to the target driving route. The reference control value and the correction control value each include, for example, an acceleration control value that specifies the acceleration in the forward direction of the vehicle 10 and a steering angle control value that specifies the steering angle of the vehicle 10. Note that, instead of the acceleration control value and the steering angle control value, the reference control value and the correction control value each may be a control value that includes either a trajectory control value or a destination control value. The trajectory control value is a control value that specifies the driving trajectory of the vehicle 10 by arranging the target driving positions of the vehicle 10 for each predetermined time in chronological order. The destination control value is a control value that indicates the target arrival time of the vehicle 10 to the target arrival location.

[0047] The third transmission unit 723 transmits various information to devices other than the remote control device 7. For example, the third transmission unit 723 transmits the control value created by the control value creation unit 722 to the vehicle 10 to be controlled. Note that at least a part of the functions of the remote control device 7 may be realized as one function of any of the vehicle control device 150, the detection device 5, the position calculation device 6, and the imaging device 9.

[0048] A-2. Vehicle detection and position calculation methods: Fig. 6 is a flowchart showing a method for detecting a vehicle 10 and a method for calculating a position of the vehicle 10 in the first embodiment. The method shown in Fig. 6 is repeatedly executed at predetermined time intervals after the vehicle 10 to be detected starts traveling in the remote autonomous driving mode, for example.

[0049] FIG. 7 is a schematic diagram showing examples of various images when the detection method and the position calculation method shown in FIG. 6 are executed. In FIG. 7, step numbers corresponding to the steps in FIG. 6 are assigned. In this embodiment, a case will be described in which the vehicle 10 runs (moves) along the traveling direction on a road surface 20 on which mesh-like lattice lines 21 are drawn along the Xg axis parallel to the traveling direction of the vehicle 10 and the Yg axis perpendicular to the Xg axis. The Xg axis and the Yg axis are each a coordinate axis in a global coordinate system. In other embodiments, the lattice lines 21 may be omitted.

[0050] As shown in Fig. 6, first, an image acquiring step (step S1) is executed. As shown in Fig. 7, the image acquiring step is a step of acquiring an original image Im1 including the vehicle 10. In this embodiment, in the image acquiring step, the image acquiring unit 521 acquires the original image Im1 from the imaging device 9.

[0051] As shown in FIG. 6, following the image acquisition step, a distortion correction step (step S2) is executed. The distortion correction step is a step of correcting the distortion of the captured image. In this embodiment, as shown in FIG. 7, in the distortion correction step, the distortion correction unit 522 corrects the distortion of the original image Im1 to generate a corrected image Im2. Specifically, the distortion correction unit 522 corrects the distortion of the original image Im1 using, for example, a distortion correction parameter Pa1 stored in advance in the storage unit 53 of the detection device 5. The distortion correction parameter Pa1 is, for example, a parameter related to position information of the grid lines 21 obtained by calibration. Note that the distortion correction method is not limited to the above. The distortion correction parameter Pa1 may be any parameter other than those described above.

[0052] As shown in FIG. 6, following the distortion correction step, a rotation processing step (step S3) is executed. The rotation processing step is a step of rotating the captured image so that the direction of the movement vector of the vehicle 10 included in the captured image faces a predetermined direction. In this embodiment, as shown in FIG. 7, in the rotation processing step, the rotation processing unit 523 rotates the corrected image Im2 so that the direction of the movement vector for the vehicle 10 included in the corrected image Im2 faces a predetermined direction. In this way, the rotation processing unit 523 generates a rotated image Im3. Specifically, the rotation processing unit 523 rotates the corrected image Im2 with the center of gravity of the vehicle 10 in the corrected image Im2 as the center of rotation so that the direction of the movement vector of the vehicle 10 faces an upward direction on the screen of a display device that displays the corrected image Im2. The movement of a feature point (for example, the center of gravity) of the vehicle 10 can be expressed as the direction of the movement vector by, for example, an optical flow method. The amount and direction of the movement vector of the vehicle 10 are estimated based on, for example, a change in the position of a feature point between image frames that is appropriately set on the corrected image Im2. Note that the rotation processing method is not limited to the above.

[0053] As shown in FIG. 6, after the rotation process, a crop process (step S4) is executed. The crop process is a process of cutting out a necessary area from a captured image. In this embodiment, as shown in FIG. 7, in the crop process, when the vehicle 10 moves a distance exceeding a predetermined threshold, the crop processing unit 524 deletes a moved area A2 corresponding to the distance moved by the vehicle 10 from the rotated image Im3 as an unnecessary area. As a result, the crop processing unit 524 generates a processed image Im4 in which an unmoved area A1 including the vehicle 10 is cut out as a necessary area from the rotated image Im3. At this time, the crop processing unit 524 estimates the moved area A2 by recognizing the distance moved by the vehicle 10 from, for example, the estimated movement vector amount of the vehicle 10. Note that, in the detection method for the vehicle 10, either the rotation process or the crop process may be executed first. Also, the crop processing method is not limited to the above.

[0054] As shown in Fig. 6, following the cropping process step, a type acquisition step (step S5) is executed. The type acquisition step is a step of acquiring type information indicating a body type of the vehicle 10 included in the captured image. In the type acquisition step, the type acquisition unit 525 acquires type information indicating a body type of the vehicle 10 included in the captured image. Note that the type acquisition step may be executed at any point after the image acquisition step is completed and before the detection step (step S7) is started.

[0055] After the type acquisition step, a model acquisition step (step S6) is executed. The model acquisition step is a step of acquiring a first detection model Md1 selected according to one body type identified by the type information acquired in the type acquisition step from among a plurality of first detection models Md1 prepared for each body type. In this embodiment, in the model acquisition step, the model acquisition unit 526 selects and acquires a first detection model Md1 according to one body type identified by the type information acquired in the type acquisition step from among a plurality of first detection models Md1 pre-stored in the storage unit 53 of the detection device 5. Note that the model acquisition step may be executed at any time after the type acquisition step is completed and before the detection step is started.

[0056] After the model acquisition step, the detection step is executed. As shown in FIG. 7, the detection step is a step of detecting the outer shape of the vehicle 10 included in the captured image by using a detection model such as the first detection model Md1. In this embodiment, in the detection step, the detection unit 527 inputs the processed image Im4 to the first detection model Md1 selected in the model acquisition step. As a result, the detection unit 527 detects the outer shape of the vehicle 10 included in the processed image Im4. Then, the detection unit 527 generates a first mask image Im5 in which a mask area Ms is added to the target area by masking the target area in the processed image Im4. The first transmission unit 528 transmits the first mask image Im5 to the position calculation device 6. Note that the detection method of the vehicle 10 is not limited to the above.

[0057] As shown in FIG. 6, after the detection step, a perspective transformation step (step S8) is executed. As shown in FIG. 7, the perspective transformation step is a step of performing perspective transformation on the first mask image Im5. In the perspective transformation step, the data acquisition unit 621 acquires the first mask image Im5 from the detection device 5. Then, the perspective transformation unit 622 performs perspective transformation on the first mask image Im5 to generate a second mask image Im6. Specifically, the perspective transformation unit 622 performs perspective transformation on the first mask image Im5 into a bird's-eye image viewed from a viewpoint above the vehicle 10 (for example, directly above the vehicle 10) that is substantially perpendicular to the road surface 20, using, for example, a perspective transformation parameter Pa2 stored in advance in the storage unit 63 of the position calculation device 6. The perspective transformation parameter Pa2 is, for example, a parameter related to position information and internal parameters of the imaging device 9 obtained by calibration. As a result, the perspective transformation unit 622 generates a second mask image Im6 expressed in the image coordinate system from the first mask image Im5 expressed in the camera coordinate system. The image coordinate system has a point on the image plane projected by the perspective transformation as its origin, and has coordinate axes indicated by the Xi axis and the Yi axis perpendicular to the Xi axis. Note that the perspective transformation method is not limited to the above. The perspective transformation parameter Pa2 may be any parameter other than those described above.

[0058] As shown in Fig. 6, after the perspective transformation step, a coordinate point calculation step (step S9) is executed. Fig. 8 is a diagram for explaining the details of the coordinate point calculation step. The coordinate point calculation step is a step for calculating an image coordinate point P3 indicating the position of the vehicle 10 in the image coordinate system.

[0059] In the coordinate point calculation step, the coordinate point calculation unit 623 calculates a base coordinate point P0 from a first circumscribing rectangle R1 set in the mask area Ms in the first mask image Im5, which is an image before perspective transformation. FIG. 9 is a diagram for explaining a method for calculating the base coordinate point P0. In order to calculate the base coordinate point P0, the coordinate point calculation unit 623 sets a base circumscribing rectangle R0 for the mask area Ms in the first mask image Im5. Next, the coordinate point calculation unit 623 rotates the first mask image Im5 by a required rotation amount with the center of gravity C of the mask area Ms as the rotation center so that the direction of the movement vector V of the vehicle 10 corresponding to the mask area Ms in the first mask image Im5 faces a predetermined direction. The predetermined direction is, for example, the upward direction on the screen of the display device that displays the first mask image Im5. Next, the coordinate point calculation unit 623 sets a first circumscribing rectangle R1 for the mask area Ms of the rotated first mask image Im5 so that the long side is parallel to the direction of the movement vector V. Next, the coordinate point calculation unit 623 rotates the first mask image Im5 to which the first circumscribing rectangle R1 has been added by the above-mentioned rotation amount with the center of gravity C of the mask area Ms as the rotation center inversely. As a result, the coordinate point calculation unit 623 sets the coordinate point of the vertex having the coordinates that are the closest to the positioning point 10e of the vehicle 10, which is one of the four vertices of the first circumscribing rectangle R1, as the base coordinate point P0. Then, as shown in FIG. 8, the coordinate point calculation unit 623 performs perspective transformation on the first mask image Im5 after the reverse rotation, that is, the first mask image Im5 after the calculation of the base coordinate point P0. As a result, the coordinate point calculation unit 623 determines the coordinate point corresponding to the base coordinate point P0 in the first circumscribing rectangle R1 transformed by the perspective transformation as the first coordinate point P1.

[0060] Furthermore, the coordinate point calculation unit 623 sets a second circumscribing rectangle R2 for the mask area Ms in the second mask image Im6 obtained by performing perspective transformation on the first mask image Im5. Then, the coordinate point calculation unit 623 sets the vertex indicating the same position as the first coordinate point P1 as the second coordinate point P2 among the vertices of the second circumscribing rectangle R2. That is, the first coordinate point P1 and the second coordinate point P2 are coordinate points indicating the same position, and therefore have a correlation with each other.

[0061] Furthermore, the coordinate point calculation unit 623 performs correction to replace the coordinates (Xi1, Yi1) of the first coordinate point P1 with the coordinates (Xi2, Yi2) of the second coordinate point P2 depending on the magnitude relationship between the coordinate values ​​of the first coordinate point P1 and the second coordinate point P2. If the coordinate value Xi1 in the Xi direction of the first coordinate point P1 is greater than the coordinate value Xi2 in the Xi direction of the second coordinate point P2 (Xi1>Xi2), the coordinate point calculation unit 623 replaces the coordinate value Xi1 in the Xi direction of the first coordinate point P1 with the coordinate value Xi2 in the Xi direction of the second coordinate point P2. When the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is greater than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction (Yi1>Yi2), the coordinate point calculation unit 623 replaces the coordinate value Yi1 of the first coordinate point P1 in the Yi direction with the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. In the example shown in FIG. 8, the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. Also, the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is smaller than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. Therefore, the image coordinate point P3 has coordinates (Xi2, Yi1). In this way, the coordinate point calculation unit 623 calculates the image coordinate point P3 indicating the position of the vehicle 10 in the image coordinate system by correcting the first coordinate point P1 using the second coordinate point P2. The method of calculating the image coordinate point P3 is not limited to the above.

[0062] As shown in FIG. 6, after the coordinate point calculation step, a position conversion step (step S10) is executed. The position conversion step is a step of converting the image coordinate point P3 into a vehicle coordinate point to calculate a vehicle coordinate point indicating the position of the vehicle 10 in the global coordinate system. In the position conversion step, the position conversion unit 624 converts the image coordinate point P3 into a vehicle coordinate point using the imaging parameters and vehicle parameters acquired by the data acquisition unit 621. The vehicle parameters are parameters related to the distance from a reference point to a positioning point 10e of the vehicle 10. In this embodiment, the vehicle parameters are heights h (see FIG. 10 described later) of the positioning point 10e of the vehicle 10 from the road surface 20. The vehicle parameters may be variables according to the body type of the vehicle 10, or may be predetermined constants.

[0063] The position conversion unit 624 converts the image coordinate point P3 into a vehicle coordinate point using the relational expressions (1) to (3) described below, which use the vehicle coordinate point as a target variable and include the image coordinate point P3, the imaging parameters, and the vehicle parameters as explanatory variables. In this case, the position conversion unit 624 substitutes the coordinate value of the image coordinate point P3 calculated by the coordinate point calculation unit 623 into the relational expressions represented by the expressions (1) to (3). Furthermore, the position conversion unit 624 substitutes the imaging parameters acquired by the data acquisition unit 621, i.e., the values ​​of the imaging parameters corresponding to the imaging device 9 that acquired the original image Im1, into the relational expressions represented by the expressions (1) to (3).

[0064] FIG. 10 is a first diagram for explaining the details of the position conversion process. FIG. 10 illustrates a state in which the vehicle 10 is viewed from the left side. FIG. 11 is a second diagram for explaining the details of the position conversion process. FIG. 11 illustrates a state in which the vehicle 10 is viewed from the roof side. The global coordinate system shown in FIG. 10 and FIG. 11 is a coordinate system having a fixed coordinate point Pf indicating an arbitrary reference position of the road surface 20 as the origin, and coordinate axes indicated by the Xg axis and the Yg axis perpendicular to the Xg axis. The imaging coordinate point Pc is the position of the imaging device 9 that acquired the original image Im1 used in calculating the image coordinate point P3, and is a coordinate point indicating the position of the imaging device 9 in the global coordinate system. The fixed coordinate point Pf and the imaging coordinate point Pc are stored in advance in the storage unit 63 of the position calculation device 6.

[0065] 10, the observation distance on the XgYg plane between the position of the imaging device 9 and the position of the vehicle 10 (image coordinate point P3) is Do. The observation error is ΔD. The height [m] of the imaging device 9 from the road surface 20 as an imaging parameter is H. The height [m] of the positioning point 10e of the vehicle 10 from the road surface 20 as a vehicle parameter is h. In this case, the observation error ΔD can be expressed by the following equation (1). ΔD=h / H×Do Equation (1) That is, the larger the observation distance Do, the larger the observation error ΔD.

[0066] Next, if the actual distance between the position of the imaging device 9 and the position of the positioning point 10e of the vehicle 10 (hereinafter, referred to as the first distance) is D, the first distance D can be expressed by the following formula (2). D=Do×(1-h / H) Equation (2) That is, the first distance D is determined by the observation distance Do, the height H of the imaging device 9 as an imaging parameter, and the height h of the positioning point 10e of the vehicle 10 as a vehicle parameter.

[0067] Then, as shown in FIG. 11, if the estimated distance between the reference position and the position of vehicle 10 is Dp, and the actual distance between the reference position and vehicle 10 (hereinafter, the second distance) is Dt, the second distance Dt can be expressed by the following equation (3). Dt = Dp × (1-h / H) Equation (3)

[0068] Here, the estimated distance Dp can be calculated using an actual distance (hereinafter, the third distance Dc) calculated from the fixed coordinate point Pf and the imaged coordinate point Pc, the image coordinate point P3, and the fixed coordinate point Pf. Therefore, the position conversion unit 624 can calculate the vehicle coordinate point Pv using the second distance Dt calculated by correcting the estimated distance Dp using the above formula (3) and the fixed coordinate point Pf. At this time, the calculated vehicle coordinate point Pv is a coordinate point indicating the position of the vehicle 10 in the global coordinate system, and therefore corresponds to the position of the vehicle 10 in the real space. The second transmission unit 625 transmits the vehicle coordinate point Pv to the remote control device 7.

[0069] A-3. Vehicle operation control method: Fig. 12 is a flowchart showing a method for controlling the operation of the vehicle 10 traveling in the remote automatic driving mode. The method for controlling the operation shown in Fig. 12 is repeatedly executed, for example, every time a vehicle coordinate point Pv is received from the position calculation device 6 as information indicating the position of the vehicle 10.

[0070] The information acquisition unit 721 of the remote control device 7 acquires driving information including the vehicle coordinate point Pv (step S101). Then, the control value creation unit 722 uses the driving information to create a control value that specifies the driving operation of the vehicle 10 (step S102). Then, the third transmission unit 723 transmits the control value to the vehicle 10 (step S103). The vehicle control device 150 mounted on the vehicle 10 drives the drive unit 110 and the like according to the received control value (step S104).

[0071] According to the first embodiment, when the vehicle 10 is made to travel automatically by remote control, the position calculation system 1 can calculate the position of the vehicle 10 by using a captured image including the vehicle 10. Specifically, the detection device 5 inputs the captured image to a detection model that is a trained machine learning model capable of detecting the vehicle 10 included in the captured image. This enables the detection device 5 to detect the vehicle 10 included in the captured image. Then, the position calculation device 6 can calculate a vehicle coordinate point Pv indicating the position of the vehicle 10 included in the captured image by using a first mask image Im5 that is a detection result by the detection device 5.

[0072] Here, since the vehicle 10 has a different external shape depending on the body type, the feature amount extracted when the detection model is trained differs depending on the body type. Therefore, when a detection model in which only a plurality of training images including the vehicles 10 classified into different body types are trained in the same number for each body type is applied to the vehicles 10 classified into any body type, the detection accuracy of the vehicles 10 of some body types may be reduced. When the detection accuracy of the vehicles 10 of some body types is reduced, in order to improve the detection accuracy of the vehicles 10 of the body type with the reduced detection accuracy, it is possible to add training images including the vehicles 10 of the body type with the reduced detection accuracy and re-train the detection model. However, by training more training images including the vehicles 10 of some body types than training images including the vehicles 10 of other body types, there is a possibility that the feature amount included in the added training images is trained more intensively than other feature amounts. This may result in a new difference in detection accuracy between body types. When the detection accuracy of the vehicle 10 included in the captured image is reduced, the calculation accuracy of the vehicle coordinate point Pv indicating the position of the vehicle 10 is reduced. When the calculation accuracy of the vehicle coordinate point Pv is reduced, there is a risk that the desired control value cannot be created.

[0073] Therefore, in the first embodiment, the first training image including the vehicle 10 classified into one body type is trained more than the second training image including the vehicle 10 classified into another body type, so that the first detection model Md1 suitable for detecting the vehicle 10 classified into one body type is prepared for each body type. According to the first embodiment, the detection device 5 acquires type information indicating one body type of the vehicle 10 included in the captured image, and can acquire the first detection model Md1 selected according to the one body type specified by the acquired type information from among the multiple first detection models Md1 prepared for each body type. Then, the detection device 5 inputs the captured image to the first detection model Md1 corresponding to the body type of the vehicle 10 included in the captured image. This allows the detection device 5 to accurately detect the vehicle 10 included in the captured image. That is, by acquiring the first detection model Md1 suitable for detecting the vehicle 10 classified into one body type, each region constituting the captured image can be accurately classified into a target region and a non-target region. As a result, it is possible to prevent the detection accuracy of the vehicle 10 included in the captured image from decreasing depending on the body type of the vehicle 10. As a result, it is possible to prevent the calculation accuracy of the vehicle coordinate point Pv from decreasing. This makes it possible to reduce the difference between the position of the vehicle 10 calculated by the position calculation device 6 and the actual position of the vehicle 10. Therefore, it is possible to create more appropriate control values.

[0074] According to the first embodiment, the first teacher data set includes M (M is an integer equal to or greater than 2) first training images and N (N is an integer equal to or greater than 0 and less than M) second training images. In this case, the first teacher data set may include only the first training images without including the second training images. In this case, for example, the detection accuracy of the first detection model Md1 can be improved by increasing the number M of first training images including vehicles 10 classified into one body type. The first teacher data set may also include the first training images and N number of second training images that is smaller than the first training images. In this case, by including the second training images in the first teacher data set, for example, the detection accuracy of a part of the vehicle 10 having a common external shape regardless of the body type can be improved.

[0075] Furthermore, according to the first embodiment, when detecting the vehicle 10 included in the captured image, the distortion of the original image Im1 can be corrected to generate a corrected image Im2. In this way, the detection accuracy of the vehicle 10 included in the captured image can be further improved. This can improve the calculation accuracy of the position of the vehicle 10.

[0076] Furthermore, according to the first embodiment, when detecting a vehicle 10 included in a captured image, a rotated image Im3 can be generated by rotating the corrected image Im2 so that the direction of the movement vector V of the vehicle 10 faces a predetermined direction. In this way, the vehicle 10 included in the captured image can be detected with the direction of the movement vector V being unified. This can further improve the detection accuracy of the vehicle 10 included in the captured image.

[0077] Furthermore, according to the first embodiment, when detecting the vehicle 10 included in the captured image, a crop process is performed to cut out a necessary area including the vehicle 10 from the rotated image Im3, thereby generating a processed image Im4 from which unnecessary areas have been removed. In this way, the vehicle 10 can be detected in a state in which elements other than the vehicle 10 to be detected have been removed from the captured image. This can further improve the detection accuracy of the vehicle 10 included in the captured image.

[0078] Furthermore, according to the first embodiment, when detecting a vehicle 10 included in a captured image, a processed image Im4 can be generated by performing a cropping process to cut out an unmoved area A1 including the vehicle 10 from the rotated image Im3. In this way, the area of ​​the vehicle 10 in the captured image can be made larger than when the cropping process is not performed. This makes it easier to detect a vehicle 10 that is farther away from the imaging device 9. Therefore, the detection accuracy for a vehicle 10 that is farther away from the imaging device 9 can be improved.

[0079] According to the first embodiment, by inputting the processed image Im4 to the first detection model Md1, the target area among the areas constituting the processed image Im4 is masked to generate a first mask image Im5 in which a mask area Ms is added to the target area. At this time, according to the first embodiment, a DNN having a CNN structure that realizes semantic segmentation and instance segmentation can be used as the algorithm of the first detection model Md1. This makes it possible to suppress a decrease in the detection accuracy of the vehicle 10 due to the diversity of non-target areas in the captured image.

[0080] Furthermore, according to the first embodiment, the second mask image Im6 can be generated by performing perspective transformation on the first mask image Im5, thereby making it possible to transform the camera coordinate system into the image coordinate system.

[0081] According to the first embodiment, the first circumscribing rectangle R1 is set for the mask area Ms before the perspective transformation of the first mask image Im5, so that the base coordinate point P0, which is the vertex of the first circumscribing rectangle R1 having the coordinates closest to the positioning point 10e of the vehicle 10, can be calculated. Then, the first coordinate point P1, which is the coordinate point corresponding to the base coordinate point P0, can be calculated by performing perspective transformation on the first mask image Im5 after the calculation of the base coordinate point P0. Furthermore, the second circumscribing rectangle R2 is set for the mask area Ms of the second mask image Im6, so that the second coordinate point P2, which is the vertex of the second circumscribing rectangle R2 having the coordinates closest to the positioning point 10e of the vehicle 10, can be calculated. Then, the first coordinate point P1 is corrected using the second coordinate point P2, so that the image coordinate point P3 can be calculated. In this way, the image coordinate point P3 can be calculated more accurately by comparing and correcting the coordinate points before and after the perspective transformation. This can further improve the calculation accuracy of the position of the vehicle 10.

[0082] Furthermore, according to the first embodiment, by inputting a captured image to the specific model Md3, which is a trained machine learning model, it is possible to acquire type information indicating one body type of the vehicle 10 included in the captured image. In other words, it is possible to acquire type information about the vehicle 10 included in the captured image by using machine learning.

[0083] Furthermore, according to the first embodiment, it is possible to obtain imaging parameters related to the imaging device 9 that obtained the original image Im1. Then, the image coordinate point P3 can be converted to the vehicle coordinate point Pv by substituting the calculated image coordinate point P3 and the acquired imaging parameter values ​​into a relational expression that uses the vehicle coordinate point Pv as a response variable and includes the image coordinate point P3, the imaging parameters, and the vehicle parameters as explanatory variables. This allows the image coordinate point P3 to be converted to the vehicle coordinate point Pv by converting the image coordinate system into a global coordinate system, and the position of the positioning point 10e of the vehicle 10 in the global coordinate system to be calculated as the vehicle coordinate point Pv.

[0084] Moreover, according to the first embodiment, the imaging parameter is the height H of the imaging device 9 from the road surface 20, which is calculated based on the position of the imaging device 9 in the global coordinate system. Moreover, the vehicle parameter is the height h of the positioning point 10e of the vehicle 10 from the road surface 20. In this way, the observation error ΔD can be calculated based on the similarity relationship between the imaging parameter and the vehicle parameter. Then, the image coordinate point P3 can be converted into the vehicle coordinate point Pv using the calculated observation error ΔD.

[0085] Moreover, according to the first embodiment, the position of the vehicle 10 can be calculated without installing any installation objects such as sensors, markers, transceivers, etc., used for calculating the position of the vehicle 10 on the vehicle 10. Also, the position of the vehicle 10 can be calculated without mounting the position calculation device 6 on the vehicle 10. This can enhance the versatility of the position calculation system 1.

[0086] B. Second embodiment: FIG. 13 is a diagram showing a schematic configuration of a detection device 5a in the second embodiment. In this embodiment, the detection model used when detecting a vehicle 10 included in a captured image is different from that in the first embodiment. As a result, part of the configuration of the detection device 5a and part of the processing of the method for detecting the vehicle 10 are different from those in the first embodiment. The other configurations are the same as those in the first embodiment. The same reference numerals are used for the steps that are the same as those in the first embodiment, and the description thereof will be omitted.

[0087] In this embodiment, the storage unit 53a of the detection device 5a stores one second detection model Md2 as a detection model. The second detection model Md2 is a trained machine learning model used to detect the vehicle 10 included in the captured image, similar to the first detection model Md1. The second detection model Md2 is a machine learning model trained by inputting a second teacher data set. The second teacher data set has a plurality of training images each including the vehicle 10 classified into different body types, a region correct answer label associated with each of a plurality of regions constituting each training image, and a type correct answer label associated with each of the plurality of training images. As in the first embodiment, the region correct answer label is a correct answer label indicating whether each region in the training image is a target region indicating the vehicle 10 or a non-target region indicating other than the vehicle 10. The type correct answer label is a correct answer label indicating one body type of the vehicle 10 included in the training image. In this embodiment, the second detection model Md2 executes the following process when a captured image is input together with one body type identified by the type information acquired by the type acquisition unit 525. In this case, the second detection model Md2 identifies a target area included in the input captured image. Then, the second detection model Md2 generates a first mask image Im5 in which a mask area Ms is added to the target area by masking the target area. For example, CNN is used as an algorithm of the second detection model Md2. Note that the configuration of the second detection model Md2 is not limited to the above. For example, the second detection model Md2 may be a trained machine learning model that uses an algorithm other than a neural network.

[0088] 14 is a diagram showing details of the CPU 52a mounted on the detection device 5a in the second embodiment. The CPU 52a of the detection device 5a in this embodiment includes a model acquisition unit 526a instead of the model acquisition unit 526 in the first embodiment. The model acquisition unit 526a acquires a second detection model Md2. The CPU 52a of the detection device 5a also includes a detection unit 527a instead of the detection unit 527 in the first embodiment. The detection unit 527a inputs a captured image and one body type identified by the type information acquired by the type acquisition unit 525 to the second detection model Md2, and detects the vehicle 10 included in the captured image by identifying a target area among the areas constituting the input captured image.

[0089] Fig. 15 is a flowchart showing a method for detecting a vehicle 10 and a method for calculating a position of the vehicle 10 in the second embodiment. The method shown in Fig. 15 is repeatedly executed at predetermined time intervals after the vehicle 10 to be detected starts traveling in the remote autonomous driving mode, for example.

[0090] In this embodiment, in a model acquisition step (step S6a), the model acquisition unit 526a acquires a second detection model Md2. Then, in a detection step (step S7a), the detection unit 527a inputs the processed image Im4 and one body type identified by the type information acquired by the type acquisition unit 525 to the second detection model Md2. As a result, the detection unit 527a detects the outer shape of the vehicle 10 included in the processed image Im4. Then, the detection unit 527a generates a first mask image Im5 in which a mask area Ms is added to the target area by masking the target area in the processed image Im4.

[0091] According to the second embodiment, the second detection model Md2 is prepared as a detection model in which a plurality of training images including the vehicle 10 and a type correct answer label indicating the body type of the vehicle 10 included in each training image are associated and learned. As a result, by inputting a captured image associated with a body type to the second detection model Md2, the vehicle 10 included in the input captured image can be accurately detected. That is, each area constituting the captured image input to the second detection model Md2 can be accurately classified into a target area and a non-target area. As a result, it is possible to suppress a decrease in the detection accuracy of the vehicle 10 included in the captured image according to the body type of the vehicle 10. As a result, it is possible to suppress a decrease in the calculation accuracy of the vehicle coordinate point Pv. As a result, it is possible to reduce the difference between the position of the vehicle 10 calculated by the position calculation device 6 and the actual position of the vehicle 10. Therefore, it is possible to create a more appropriate control value.

[0092] C. Other embodiments: C-1. Alternative embodiment 1: The one body type may indicate a type of a single vehicle 10. The type of the vehicle 10 is, for example, a type of the vehicle 10 classified by a vehicle name given to the vehicles 10 having the same or a common external shape. In this case, the storage unit 53 of the detection device 5 stores at least one of a plurality of first detection models Md1 prepared for each type of the vehicle 10 and one second detection model Md2 that is learned by associating the one type of the vehicle 10 with a training image. At this time, the first training image is an image including the vehicle 10 classified into the one type. The second training image is an image including the vehicle 10 classified into another type different from the one type. The type acquisition unit 525 of the detection device 5, 5a acquires type information indicating the type of the vehicle 10 included in the captured image. The model acquisition unit 526 acquires the first detection model Md1 selected according to the one type identified by the type information acquired by the type acquisition unit 525 from the plurality of first detection models Md1 prepared for each type. With this configuration, it is possible to more accurately capture differences in the external shapes of the vehicles 10 that differ depending on the type of the vehicles 10. This makes it possible to improve the detection accuracy of the vehicles 10 included in the captured image.

[0093] C-2. Alternative embodiment 2: The type acquisition unit 525 may acquire type information indicating one body type of the vehicle 10 included in the captured image by using scheduled information stored in advance in the storage unit 53 of the detection device 5. The scheduled information is information indicating a scheduled time when the vehicle 10 classified into one body type travels through the imaging range RG, which is associated with each imaging range RG of the multiple imaging devices 9 that acquire the original image Im1. In this form, it is possible to acquire type information indicating one body type of the vehicle 10 included in the captured image by using the scheduled information. This makes it possible to acquire type information about the vehicle 10 included in the captured image without performing image analysis. Therefore, it is possible to reduce the processing load of the type acquisition unit 525 in the type acquisition process.

[0094] C-3. Alternative embodiment 3: The training images included in the first teacher data set and the second teacher data set may be images that have not been subjected to processing such as distortion correction, rotation, and cropping, like the original image Im1. In this form, no image processing is required when preparing the first teacher data set and the second teacher data set. This can reduce the processing load when learning the first detection model Md1 and the second detection model Md2.

[0095] C-4. Alternative embodiment 4: The training image may be any one of the corrected image Im2, the rotated image Im3, and the processed image Im4. In this embodiment, the detection accuracy of the vehicle 10 can be improved when the captured image obtained by processing the original image Im1 is input to either the first detection model Md1 or the second detection model Md2, as in the first and second embodiments.

[0096] C-5. Alternative embodiment 5: In the detection method of the vehicle 10 shown in Fig. 6 and Fig. 15, the distortion correction step is not an essential step. For example, if the steps after the rotation processing step are executed without executing the distortion correction step, the rotation processing unit 523 rotates the original image Im1 instead of the corrected image Im2 in the rotation processing step. Even in this form, the image coordinate point P3 can be calculated.

[0097] C-6. Alternative embodiment 6: In the detection method of the vehicle 10 shown in Fig. 6 and Fig. 15, the rotation process is not an essential step. For example, if the distortion correction step and the rotation process step are not performed, the crop processing unit 524 performs crop processing on the original image Im1 instead of the rotated image Im3 in the crop processing step. Even in this form, the image coordinate point P3 can be calculated.

[0098] C-7. Alternative embodiment 7: In the detection method of the vehicle 10 shown in Fig. 6 and Fig. 15, the cropping process is not an essential process. For example, if the distortion correction process, the rotation process, and the cropping process are not performed, the detection units 527 and 527a generate a first mask image Im5 by adding a mask area Ms to the original image Im1 in place of the processed image Im4 in the detection process. Even in this form, the image coordinate point P3 can be calculated.

[0099] C-8. Other embodiment 8: The captured image may include a plurality of vehicles 10. In this case, the CPU 52, 52a of the detection device 5, 5a may include, for example, a deletion unit that deletes a mask region Ms of the vehicle 10 that is not to be subject to position calculation from the first mask image Im5. For example, the deletion unit determines that a mask region Ms that exists outside the recognition target region among the mask regions Ms generated in the detection process is a mask region Ms of the vehicle 10 that is not to be subject to position calculation, and deletes it from the first mask image Im5. The recognition target region is, for example, a predetermined region in the first mask image Im5 in which the vehicle 10 moves. The predetermined region in which the vehicle 10 moves is, for example, a region corresponding to the region of the grid line 21. The recognition target region is stored in advance in the storage unit 53, 53a of the detection device 5, 5a. In this embodiment, when a plurality of vehicles 10 are captured in the captured image, the influence of the vehicle 10 that is not to be subject to position calculation can be eliminated. This improves the calculation accuracy of the position of the vehicle 10.

[0100] C-9. Other embodiment 9: The captured image may include a plurality of vehicles 10. In this case, a DNN that performs instance segmentation may be used as the algorithm of the first detection model Md1 and the second detection model Md2. In this form, the plurality of vehicles 10 included in the captured image can be classified, and a first mask image Im5 in which each vehicle 10 is masked can be generated. As a result, when the captured image includes a plurality of vehicles 10, the vehicle 10 to be the target of position calculation can be selected, and the position of the selected vehicle 10 can be calculated.

[0101] C-10. Other embodiment 10: The position calculation device 6 may calculate the position of the stationary vehicle 10. When calculating the position of the stationary vehicle 10, the position calculation device 6 calculates the position of the vehicle 10 by using, for example, an initial vector direction of the vehicle 10 estimated from a captured image acquired first after the start of the position calculation device 6, instead of the direction of the movement vector V of the vehicle 10 while traveling. In this manner, even when the vehicle 10 is stopped, the position of the vehicle 10 can be calculated by using the captured image.

[0102] C-11. Other embodiment 11: At least one of the detection devices 5 and 5a, the position calculation device 6, and the remote control device 7 may be configured as an integrated device. Furthermore, in another embodiment, each of the detection device 5, the position calculation device 6, and the remote control device 7 may be realized by, for example, cloud computing configured by one or more computers. In such a form, the configurations of the detection devices 5 and 5a, the position calculation device 6, and the remote control device 7 can be appropriately changed.

[0103] C-12. Other embodiment 12: The image acquisition unit 521 of the detection device 5, 5a may acquire the original image Im1 via an external device (e.g., the remote control device 7) other than the imaging device 9, without acquiring the original image Im1 directly from the imaging device 9. Even in this embodiment, the image acquisition unit 521 of the detection device 5, 5a can acquire the original image Im1 acquired by the imaging device 9.

[0104] C-13. Other embodiment 13: In each of the above embodiments, the vehicle 10 may have a configuration capable of moving by remote control, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 10 may have at least the vehicle control device 150, the drive device 110, the steering device 120, the braking device 130, and the vehicle communication unit 140 in order to perform the three functions of "running", "turning", and "stopping" by remote control. That is, the vehicle 10 capable of moving by remote control may not have at least a part of interior parts such as a driver's seat and a dashboard, may not have at least a part of exterior parts such as a bumper and a fender, and may not have a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicle 10 before the vehicle 10 is shipped from the factory, or the remaining parts such as the body shell may be attached to the vehicle 10 after the vehicle 10 is shipped from the factory in a state in which the remaining parts such as the body shell are not attached to the vehicle 10. The platform form may also be detected and the position may be calculated in the same manner as the vehicle 10 in each embodiment.

[0105] C-14. Other embodiment 14: The detection device 5, 5a may detect the vehicle 10 using the first detection model Md1 that is stored in a device other than the detection device 5, 5a and selected by the device other than the detection device 5, 5a. Even in this form, the first detection model Md1 according to the body type of the vehicle 10 can be obtained.

[0106] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0107] 1...position calculation system, 2...roadway, 5,5a...detection device, 6...position calculation device, 7...remote control device, 9...imaging device, 10...vehicle, 10e...positioning point, 20...road surface, 21...grid line, 51...communication unit of detection device, 52,52a...CPU of detection device, 53,53a...storage unit of detection device, 61...communication unit of position calculation device, 62...CPU of position calculation device, 63...storage unit of position calculation device, 71...communication unit of remote control device, 72...CPU of remote control device, 73...storage unit of remote control device, 110...driving device , 120... steering device, 130... braking device, 140... vehicle communication unit, 150... vehicle control device, 160... vehicle-mounted sensor group, 521... image acquisition unit, 522... distortion correction unit, 523... rotation processing unit, 524... crop processing unit, 525... type acquisition unit, 526... model acquisition unit, 527, 527a... detection unit, 528... first transmission unit, 621... data acquisition unit, 622... perspective transformation unit, 623... coordinate point calculation unit, 624... position transformation unit, 625... second transmission unit, 721... information acquisition unit, 722... control value creation unit, 723 ...third transmission unit, 901...first imaging device, 902...second imaging device, A1...unmoved area, A2...moved area, C...center of gravity, D...first distance, ΔD...observation error, Db...camera database, Dc...third distance, Do...observed distance, Dp...estimated distance, Dt...second distance, h...height of vehicle positioning point from road surface, H...height of imaging device from road surface, Im1...original image, Im2...corrected image, Im3...rotated image, Im4...processed image, Im5...first mask image, Im6...second mask image, Md1, Md1a to M d1c...first detection model, Md2...second detection model, Md3...specific model, Ms...mask area, Nt...network, P0...base coordinate point, P1...first coordinate point, P2...second coordinate point, P3...image coordinate point, Pa1...distortion correction parameter, Pa2...perspective transformation parameter, Pc...imaging coordinate point, Pf...fixed coordinate point, Pv...vehicle coordinate point, R0...base circumscribing rectangle, R1...first circumscribing rectangle, R2...second circumscribing rectangle, RG...imaging range, RG1...first imaging range, RG2...second imaging range, V...movement vector

Claims

1. A detection device for detecting a vehicle included in a captured image, an image acquisition unit for acquiring the captured image; a type acquisition unit that acquires type information indicating a body type of the vehicle included in the captured image, the body type being classified according to an external shape of the vehicle; and a model acquisition unit that acquires a first detection model selected from a plurality of first detection models, which are machine learning models prepared for each of the body types, according to the one body type identified by the type information acquired by the type acquisition unit; a detection unit that detects the vehicle included in the captured image by inputting the captured image to the first detection model acquired by the model acquisition unit and identifying a target area indicating the vehicle in the captured image.

2. 2. The detection device according to claim 1, A detection device, wherein each of the multiple first detection models has been pre-trained to identify the target area by inputting multiple training images including M (M is an integer equal to or greater than 2) first training images each including the vehicle classified into one of the body types, and N (N is an integer equal to or greater than 0 and less than M) second training images each including the vehicle classified into another body type different from the one body type.

3. A detection device for detecting a vehicle included in a captured image, an image acquisition unit for acquiring the captured image; a type acquisition unit that acquires type information indicating a body type of the vehicle included in the captured image, the body type being classified according to an external shape of the vehicle; and a detection unit that detects the vehicle included in the captured image by inputting the captured image and the one body type identified by the type information acquired by the type acquisition unit into a second detection model, which is a machine learning model, and identifying a target area in the captured image that indicates the vehicle.

4. 4. The detection device according to claim 3, the second detection model is pre-trained to identify the target region by inputting a plurality of training images including the vehicle and type answer labels associated with each of the plurality of training images, the type answer labels indicating the body type of the vehicle included in the training images.

5. The detection device according to claim 2 or claim 4, A detection device in which each region in the training image is associated with a region answer label indicating whether the region is the target region or a non-target region indicating a region other than the vehicle.

6. The detection device according to claim 1 or 3, The body type indicates a type of the vehicle.

7. The detection device according to claim 1 or 3, The vehicle travels by receiving, from an outside of the vehicle, a travel condition that specifies a travel operation of the vehicle; The type acquisition unit acquires the type information by identifying the one body type of the vehicle included in the captured image using scheduled information that indicates a scheduled time when the vehicle classified into the one body type is scheduled to travel through the captured image range, which is associated with each of the imaging ranges of a plurality of imaging devices that acquire the captured images.

8. The detection device according to claim 1 or 3, The type acquisition unit acquires the type information by inputting the captured image to a specific model, which is a machine learning model that has been trained to output the type information when the captured image is input.

9. The detection device according to claim 1 or 3, further comprising: a rotation processing unit that rotates the captured image so that the moving direction of the vehicle faces a predetermined direction.

10. The detection device according to claim 1 or 3, further comprising: The detection device further comprises a distortion correction unit that corrects distortion of the captured image.

11. A position calculation system for calculating a position of a vehicle included in a captured image, A detection device according to claim 1 or 3; a position calculation device for calculating a position of the vehicle, The detection device further generates a first mask image by masking a target area, which is an area indicating the vehicle, in the captured image and adding a mask area to the target area, and The position calculation device is a perspective transformation unit that performs perspective transformation on the first mask image to generate a second mask image; a coordinate point calculation unit that calculates an image coordinate point that indicates a position of the vehicle in an image coordinate system by setting a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, setting a vertex of a second circumscribing rectangle set in the mask area in the second mask image that indicates the same position as the first coordinate point as a second coordinate point, and correcting the first coordinate point using the second coordinate point; a position conversion unit that converts the image coordinate points into vehicle coordinate points that indicate the position of the vehicle in the global coordinate system using a distance from a reference point of the imaging device, the distance being calculated based on a position of the imaging device in a global coordinate system, and a distance from the reference point of a predetermined positioning point of the vehicle.

12. A method for detecting a vehicle included in a captured image, comprising: an image acquisition step of acquiring the captured image; a type acquiring step of acquiring type information indicating a body type of the vehicle, the body type being classified according to an external shape of the vehicle, the type information being included in the captured image; a model acquisition process for acquiring a first detection model selected from a plurality of first detection models, which are machine learning models prepared for each of the body types, according to the one body type identified by the type information acquired in the type acquisition process; a detection process of detecting the vehicle included in the captured image by inputting the captured image to the first detection model acquired in the model acquisition process and identifying a target area indicating the vehicle in the captured image.

13. A method for detecting a vehicle included in a captured image, comprising: an image acquisition step of acquiring the captured image; a type acquiring step of acquiring type information indicating a body type of the vehicle, the body type being classified according to an external shape of the vehicle, the type information being included in the captured image; a detection process for detecting the vehicle included in the captured image by inputting the captured image and the one body type identified by the type information acquired in the type acquisition process into a second detection model, which is a machine learning model, and identifying a target area in the captured image that indicates the vehicle.

Images