Detection device, position calculation system, and detection method

Abstract

In a detection apparatus configured to detect a vehicle in a captured image, the vehicle is configured to move in a plant that performs a plurality of manufacturing steps to manufacture and load the vehicle, the vehicle being classified into a plurality of states according to an appearance of the vehicle that varies among the plurality of manufacturing steps. The detection device comprises a first processor, and the first processor is configured to obtain a shot image; acquiring state information indicating one state of the vehicle in the captured image; acquiring a first detection model selected according to the one of the states identified by the acquired state information from among a plurality of first detection models which are machine learning models each prepared one by one for the plurality of states; and detecting a vehicle included in the captured image by inputting the captured image into the acquired first detection model and recognizing a target area representing the vehicle in the captured image.

Description

Technical Field

[0001] The present disclosure relates to a detection device, a position calculation system and a detection method. Background Art

[0002] A vehicle that moves automatically by remote control is known (Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-538619 (JP 2017-538619 A). Summary of the invention

[0003] In order to make a vehicle move automatically by remote control, an image of the vehicle may be captured from the outside. However, when a vehicle moves in a factory for manufacturing vehicles, the vehicle may have various appearances according to the manufacturing steps. For example, the appearance of a vehicle in the form of a platform on which various devices such as wheels and a vehicle controller are assembled is greatly different from the appearance of a vehicle on which a body such as a body shell is assembled. The inventors of the present application have found that due to such differences in the appearance of the vehicle according to the manufacturing steps, the detection accuracy of the vehicle contained in the captured image may be reduced.

[0004] The present disclosure may be implemented as follows.

[0005] A first aspect of the present disclosure provides a detection device. In a detection device configured to detect a vehicle contained in a captured image, the vehicle is configured to move in a factory that performs multiple manufacturing steps to manufacture and ship the vehicle, and the vehicle is classified into multiple states according to the appearance of the vehicle that changes in the multiple manufacturing steps. The detection device includes a first processor. The first processor is configured to: acquire a captured image; acquire state information indicating one of the states of the vehicle contained in the captured image; acquire a first detection model selected according to one of the states identified by the acquired state information from a plurality of first detection models that are machine learning models prepared one by one for a plurality of states; and detect the vehicle contained in the captured image by inputting the captured image into the acquired first detection model and identifying a target area representing the vehicle in the captured image. According to this aspect, state information indicating the state of the vehicle contained in the captured image can be acquired, and a first detection model selected according to one of the states identified by the acquired state information can be acquired from a plurality of first detection models that are prepared one by one for a plurality of states. Then, the vehicle contained in the captured image can be accurately detected by inputting the captured image into the first detection model according to the state of the vehicle contained in the captured image. In other words, when the first detection model suitable for detecting a vehicle classified into one of the states is used, each area constituting the captured image can be accurately classified into a target area and an area outside the target area. Therefore, the reduction in the detection accuracy of the vehicle included in the captured image can be suppressed based on the appearance difference among multiple manufacturing steps.

[0006] In the first aspect, each of the plurality of first detection models can be pre-trained to recognize a target area by inputting a plurality of training images, and the plurality of training images can include M (M is an integer greater than or equal to 2) first training images each containing a vehicle classified as one of the states, and N (N is an integer greater than or equal to zero and less than M) second training images each containing a vehicle classified as another state different from one of the states. According to this configuration, the number of first training images each containing a vehicle classified as one of the states to be trained is greater than the number of second training images each containing a vehicle classified as another state different from the one of the states. As a result, first detection models suitable for vehicles classified as one of the states can be prepared one by one for each state.

[0007] In the above configuration, each region in the training image can be associated with a region correct label, and the region correct label can indicate which of the target region and the non-target region representing an object other than a vehicle. According to this configuration, the various regions constituting the captured image can be accurately classified into the target region and the non-target region. Therefore, the detection accuracy of the vehicle contained in the captured image can be improved.

[0008] In the above aspects, the first processor may be configured to acquire the status information by acquiring step information about one of the manufacturing steps being performed on the vehicle, and identifying the status information associated with the one of the manufacturing steps identified by the acquired step information by using a step database that associates the status information with each of the plurality of manufacturing steps. According to this configuration, the step information of one of the manufacturing steps being performed on the vehicle can be acquired. Then, when the status information associated with the one of the manufacturing steps identified by the acquired step information is identified by using the step database, the status information indicating the status of the vehicle contained in the captured image can be acquired.

[0009] In the above aspects, the first processor may be configured to, when a captured image is input, acquire state information by: inputting the captured image into a specific model that is a trained machine learning model to output state information. According to this configuration, when the captured image is input into a specific model that is trained to output state information about a vehicle included in the captured image, state information indicating the state of the vehicle included in the captured image may be acquired.

[0010] In the above aspects, the first processor may be configured to acquire the status information by acquiring the camera information about the camera device, and identifying the status information associated with one of the camera devices identified by the acquired camera information by using a status database that associates the camera information with the status information. According to this configuration, the camera information about the camera device can be acquired. Then, when the status information associated with one of the camera devices identified by the acquired camera information is identified by using the status database that associates the camera information with the status information, the status information indicating the status of the vehicle contained in the captured image can be acquired.

[0011] In the above aspects, the multiple manufacturing steps may include painting a vehicle, and the multiple states may include an unpainted state and a painted state. The unpainted state may indicate the state of the vehicle before being painted in the painted vehicle. The painted state may indicate the state of the vehicle after being painted in the painted vehicle. According to this configuration, the state of the vehicle can be classified into an unpainted state and a painted state based on whether the painting is performed before or after the painting step. Therefore, the reduction in the detection accuracy of the vehicle contained in the captured image can be suppressed based on the difference in the appearance color of the vehicle before and after the painting step is performed.

[0012] In the above aspects, the first processor may be configured to correct the distortion of the captured image. According to this configuration, the distortion of the captured image may be corrected. Therefore, the detection accuracy of the vehicle included in the captured image may be improved.

[0013] In the above aspects, the first processor may be configured to rotate the captured image so that the moving direction of the vehicle points to a predetermined direction. According to this configuration, the captured image may be rotated so that the direction of the vector indicating the moving direction of the vehicle points to the predetermined direction. As a result, the vehicle included in the captured image may be detected in a state where the direction of the vector indicating the moving direction of the vehicle is unified. Therefore, the detection accuracy of the vehicle included in the captured image may be improved.

[0014] The second aspect of the present disclosure provides a detection device. In a detection device configured to detect a vehicle contained in a captured image, the vehicle is configured to move in a factory that performs multiple manufacturing steps to manufacture and ship the vehicle, and the vehicle is classified into multiple states according to the appearance of the vehicle that changes in the multiple manufacturing steps, and the multiple states include a platform state. The platform state is the form of a platform in which the vehicle at least includes the following: wheels; chassis; drive unit, which is configured to accelerate the vehicle; steering device, which is configured to change the driving direction of the vehicle; braking device, which is configured to decelerate the vehicle; vehicle controller, which is configured to control the operation of the vehicle; and vehicle communication unit, which is configured to communicate with other devices other than the vehicle. The detection device includes a first processor. The first processor is configured to: acquire a captured image; acquire state information indicating one of the states of the vehicle contained in the captured image; from a plurality of first detection models that are machine learning models prepared one by one for a plurality of states, acquire a first detection model selected according to one of the states identified by the acquired state information; and detect the vehicle contained in the captured image by inputting the captured image into the acquired first detection model and identifying a target area representing the vehicle in the captured image. According to this aspect, when the form of the vehicle is a platform form, the state of the vehicle can be classified as a platform state. Then, when detecting a vehicle classified as a platform state, the vehicle can be detected by using a first detection model suitable for detecting a vehicle classified as a platform state. When detecting a vehicle classified as a platform state, the vehicle can be detected by inputting a captured image associated with the platform state into a second detection model. Therefore, the detection accuracy of a vehicle classified as a platform state can be improved.

[0015] In the second aspect, each of the plurality of first detection models may be pre-trained to recognize a target area by inputting a plurality of training images, and the plurality of training images may include M (M is an integer greater than or equal to 2) first training images each containing a vehicle classified as one of the states, and N (N is an integer greater than or equal to zero and less than M) second training images each containing a vehicle classified as another state different from one of the states. According to this configuration, the number of first training images each containing a vehicle classified as one of the states to be trained is greater than the number of second training images each containing a vehicle classified as another state different from the one of the states. As a result, first detection models suitable for vehicles classified as one of the states can be prepared one by one for each state.

[0016] The third aspect of the present disclosure provides a detection device. In a detection device configured to detect a vehicle contained in a captured image, the vehicle is configured to move in a factory that performs multiple manufacturing steps to manufacture and ship the vehicle, and the vehicle is classified into multiple states according to the appearance of the vehicle that changes among the multiple manufacturing steps. The detection device includes a first processor. The first processor is configured to: acquire a captured image; acquire state information indicating one of the states of the vehicle contained in the captured image; and detect the vehicle contained in the captured image by inputting the captured image and the one of the states identified by the acquired state information into a second detection model as a machine learning model, and identifying a target area representing the vehicle in the captured image. According to this aspect, by inputting a captured image associated with the state of the vehicle into the second detection model, the vehicle contained in the captured image can be accurately detected. In other words, the various areas constituting the captured image can be accurately classified into target areas and areas outside the target areas. Therefore, the reduction in the detection accuracy of the vehicle contained in the captured image can be suppressed according to the difference in appearance among multiple manufacturing steps.

[0017] In the third aspect described above, the second detection model may be pre-trained to recognize the target area by inputting a plurality of training images each containing a vehicle and a state-correct label. The state-correct label may be associated with each of the plurality of training images and indicate the state of the vehicle contained in the training image. According to this configuration, a second detection model trained under a plurality of training images each containing a vehicle and a state-correct label indicating the state of the vehicle contained in the training image may be prepared.

[0018] The fourth aspect of the present disclosure provides a position calculation system. A position calculation system configured to calculate the position of a vehicle contained in a captured image includes a detection device according to the above aspects. The first processor is also configured to generate a first mask image in which a mask area is added to the target area by applying a mask to a target area in the captured image as an area representing the vehicle. The position calculation system also includes a position calculation device configured to calculate the position of the vehicle. The position calculation device includes a second processor. The second processor is configured to: generate a second mask image by performing a perspective transformation of the first mask image; in the case where the specified vertex of the first circumscribed rectangle set for the mask area in the first mask image is a first coordinate point, and the vertex of the second circumscribed rectangle set for the mask area in the second mask image indicating the same position as the first coordinate point is a second coordinate point, by correcting the first coordinate point using the second coordinate point, to calculate the image coordinate point indicating the position of the vehicle in the image coordinate system; and by using the distance between the camera device and the reference point calculated based on the position of the camera device in the global coordinate system and the distance between the predetermined measurement point of the vehicle and the reference point, the image coordinate point is converted into a vehicle coordinate point indicating the position of the vehicle in the global coordinate system. According to this aspect, the position of the vehicle contained in the captured image can be calculated. A first mask image in which a mask is applied to a target area representing the vehicle among the various areas constituting the captured image, and a second mask image obtained by performing a perspective transformation on the first mask image can be generated. As a result, the image coordinate point can be calculated by calculating the first coordinate point from the first mask image and calculating the second coordinate point from the second mask image. Therefore, the image coordinate point can be calculated more accurately. Therefore, the calculation accuracy of the position of the vehicle can be improved.

[0019] A fifth aspect of the present disclosure provides a detection method. In a detection method for detecting a vehicle contained in a captured image, the vehicle is configured to move in a factory that performs multiple manufacturing steps to manufacture and ship the vehicle, and the vehicle is classified into multiple states according to the appearance of the vehicle that changes in the multiple manufacturing steps. The detection method includes: acquiring a captured image; acquiring state information indicating one of the states of the vehicle contained in the captured image; acquiring a first detection model selected according to one of the states identified by the acquired state information from a plurality of first detection models that are machine learning models prepared one by one for a plurality of states; and detecting the vehicle contained in the captured image by inputting the captured image into the acquired first detection model and identifying a target area representing the vehicle in the captured image. According to this aspect, state information indicating the state of the vehicle contained in the captured image can be acquired, and a first detection model selected according to one of the states identified by the acquired state information can be acquired from a plurality of first detection models that are prepared one by one for a plurality of states. Then, the vehicle contained in the captured image can be accurately detected by inputting the captured image into the first detection model according to the state of the vehicle contained in the captured image. In other words, when the first detection model suitable for detecting a vehicle classified into one of the states is used, each area constituting the captured image can be accurately classified into a target area and an area outside the target area. Therefore, the reduction in the detection accuracy of the vehicle included in the captured image can be suppressed based on the appearance difference among multiple manufacturing steps.

[0020] The sixth aspect of the present disclosure provides a detection method. In the detection method for detecting a vehicle contained in a captured image, the vehicle is configured to move in a factory that performs multiple manufacturing steps to manufacture and ship the vehicle, and the vehicle is classified into multiple states according to the appearance of the vehicle changing among the multiple manufacturing steps. The detection method includes: acquiring a captured image; acquiring state information indicating one of the states of the vehicle contained in the captured image; and detecting the vehicle contained in the captured image by inputting the captured image and the one of the states identified by the acquired state information into a second detection model that is a machine learning model, and identifying a target area representing the vehicle in the captured image. According to this aspect, by inputting a captured image associated with the state of the vehicle into the second detection model, the vehicle contained in the captured image can be accurately detected. In other words, the various areas constituting the captured image can be accurately classified into target areas and areas outside the target areas. Therefore, the reduction in the detection accuracy of the vehicle contained in the captured image can be suppressed according to the difference in appearance among multiple manufacturing steps.

[0021] The present disclosure may be implemented in various modes other than the above-mentioned detection device, position calculation system or detection method. The present disclosure may be implemented, for example, by a manufacturing method for a detection device and a position calculation system, a position calculation method, a control method for a detection device and a position calculation system, and a non-transitory recording medium on which a computer program for implementing the control method is recorded. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like numerals represent like elements, and wherein: Figure 1 is a diagram showing a configuration of a position calculation system; Figure 2 is a diagram showing a manufacturing process of a vehicle according to the first embodiment; Figure 3 is a block diagram showing the configuration of a detection device according to a first embodiment; Figure 4 is a block diagram showing details of a CPU mounted on a detection device according to a first embodiment; Figure 5 is a block diagram showing the configuration of a position calculation device; Figure 6 is a block diagram showing a configuration of a remote controller; Figure 7 is a flow chart showing a detection method and a position calculation method for a vehicle according to a first embodiment; Figure 8 are schematic views showing examples of various images; Fig. 9 is a diagram for explaining details of the coordinate point calculation steps; Fig.10 is a view for explaining a calculation method for basic coordinate points; Fig.11 is a first view for explaining details of the position transformation step; Fig.12 is a second view for explaining details of the position transformation step; Fig.13 is a flow chart showing an operation control method for a vehicle; Fig.14 is a diagram showing a manufacturing process of a vehicle according to a second embodiment; Fig.15 is a block diagram showing a configuration of a detection device according to a second embodiment; Fig.16 is a block diagram showing a configuration of a detection device according to a third embodiment; Fig.17is a block diagram showing details of a CPU mounted on a detection device according to a third embodiment; Fig.18 is a flow chart showing a detection method and a position calculation method for a vehicle according to a third embodiment; Fig.19 is a block diagram showing a configuration of a detection device according to a fourth embodiment; Fig. 20 is a block diagram showing details of a CPU mounted on a detection device according to a fourth embodiment; Fig.21 is a block diagram showing a configuration of a detection device according to a fifth embodiment; and Fig. 22 is a block diagram showing details of a CPU mounted on a detection device according to the fifth embodiment. DETAILED DESCRIPTION A. First Embodiment A-1. Configuration of the position calculation system

[0023] Figure 1 is a diagram showing the configuration of a position calculation system 1. The position calculation system 1 detects a vehicle 10 included in a captured image and calculates the position of the vehicle 10. In an embodiment of the present disclosure, the captured image is an original image as original data acquired by a camera device 9 serving as an external device set in a position different from the position where the vehicle is located, and at least any one of a corrected image, a rotated image, and a processed image obtained by processing the original image. The details of the corrected image, the rotated image, and the processed image will be described later. The position calculation system 1 includes one or more vehicles 10, a detection device 5 that detects the vehicle 10 included in the captured image, and a position calculation device 6 that calculates the position of the vehicle 10 by using the result of detection by the detection device 5.

[0024] The camera device 9 acquires an original image by capturing an image of a camera range RG from outside the vehicle 10, and the camera range RG includes the vehicle 10 as the target of position calculation. In the present embodiment, the camera device 9 sends the acquired original image together with the camera identification information and acquisition time of the original image to the detection device 5. The camera identification information is a unique identifier (ID) assigned to avoid duplication between multiple camera devices 9 so as to identify the camera device 9. The original image is a two-dimensional image formed by a set of pixels set in the Xc-Yc plane in the camera coordinate system. The camera coordinate system refers to a coordinate system having an origin set at the focus of the camera device 9 and having coordinate axes represented by an Xc axis and a Yc axis orthogonal to the Xc axis. The original image contains at least two-dimensional data of the vehicle 10 as the target of position calculation. The original image is preferably a color image, and may also be a grayscale image. The camera device 9 is a camera including an optical system and an imaging device such as a CCD image sensor and a CMOS image sensor.

[0025] In the present embodiment, the camera device 9 acquires an original image overlooking the route 2 and the vehicle 10 moving along the route 2 from above. In order to capture the image of the entire route 2 using one or more camera devices 9, the installation position of each camera device 9 and the number of camera devices 9 are determined in consideration of the image capturing range RG (angle of view) of each camera device 9. Specifically, the camera devices 9 are installed so that the first camera range RG1 and the second camera range RG2 adjacent to each other overlap. The first camera range RG1 is the camera range RG of the first camera device 901. The second camera range RG2 is the camera range RG of the second camera device 902. In addition, each camera device 9 is installed at a position where an image of a pre-set measurement point 10e of a specific part of the vehicle 10 moving along the route 2 can be captured. In the present embodiment, the measurement point 10e is the rear end (left rear end) on the left side of the vehicle 10. The measurement point 10e may not be the left rear end of the vehicle 10. The camera device 9 can not only acquire information from above the vehicle 10, but also acquire information such as the front, rear, and side of the vehicle 10.

[0026] Examples of the vehicle 10 include battery electric vehicles, hybrid electric vehicles, fuel cell electric vehicles, gasoline vehicles, and diesel vehicles. The vehicle 10 may be a private vehicle such as a passenger car, or may be a commercial vehicle such as a truck, a bus, or a construction vehicle.

[0027] The vehicle 10 has a manned driving mode and a remote unmanned driving mode. In the manned driving mode, when the driver in the vehicle 10 operates an input device such as a steering wheel, an accelerator of the vehicle 10, a driving condition of the vehicle 10 is generated. Therefore, the vehicle 10 moves according to the generated driving condition. The driving condition defines the driving operation of the vehicle 10. The driving condition includes, for example, the driving route, position, driving speed and acceleration of the vehicle 10 and the steering angle of the wheel 170. The remote unmanned driving mode includes a remote manual driving mode and a remote automatic driving mode. In the remote manual driving mode, when the operator operates an operator input device set in a position different from the position of the vehicle 10, the driving condition of the vehicle 10 is generated. Therefore, the vehicle 10 receives the driving condition generated by the operator input device and moves according to the received driving condition. In the remote automatic driving mode, a remote controller 7, such as a server, set in a position different from the position of the vehicle 10 creates a control value defining the driving operation of the vehicle 10 and sends the control value to the vehicle 10. Therefore, the vehicle 10 receives the control value and performs automatic driving according to the received control value.

[0028] In the present embodiment, the vehicle 10 moves in a remote automatic driving mode in a factory where a plurality of manufacturing steps are performed to manufacture and ship the vehicle 10. The factory is not limited to the case where it exists in one building, the case where it exists at one site or address, etc., but may exist at a plurality of buildings, a plurality of sites, or a plurality of addresses, etc. The factory may include a storage location such as a yard where the vehicle 10 manufactured as a finished product by performing a plurality of manufacturing steps is stored before shipment. At this time, the vehicle 10 may move not only on private roads but also on public roads.

[0029] Figure 2 FIG. 1 is a diagram showing a manufacturing process of the vehicle 10 according to the first embodiment. Figure 2 In FIG. 1 , some of the multiple manufacturing steps performed in the manufacturing process of the vehicle 10 are representatively shown. Figure 2 In the example shown in , the vehicle 10 as a finished product is manufactured by assembling a painted body shell 180 to a vehicle 101 in a platform form (hereinafter referred to as the platform vehicle 101), and then assembling internal components such as seats and external components 190 such as door panels to the platform vehicle 101.

[0030] The platform manufacturing step is a manufacturing step for manufacturing the platform vehicle 101. The platform vehicle 101 is a vehicle 10 that can realize at least three functions, namely "moving", "turning" and "parking" through remote control of the remote controller 7. Specifically, the platform vehicle 101, for example, at least includes a drive unit 110, a steering device 120, a brake device 130, a vehicle communication unit 140, a vehicle controller 150, a chassis 160 and wheels 170. The drive unit 110 accelerates the vehicle 10. The drive unit 110 includes at least a driving force source and a power supply device for supplying power to the driving force source. When the vehicle 10 is a pure electric vehicle, the driving force source is a motor and the power supply device is a battery, such as a lithium-ion battery. The steering device 120 changes the driving direction of the vehicle 10. The brake device 130 decelerates the vehicle 10. The vehicle communication unit 140 communicates with external devices by using wireless communication, etc. External devices are devices other than this vehicle and other vehicles 10. Devices other than this vehicle are detection devices 5, position calculation devices 6, remote controllers 7, and camera devices 9, etc. The vehicle communication unit 140 is, for example, a wireless communication device. For example, the vehicle communication unit 140 communicates with an external device connected to the network Nt via an access point in the factory. The vehicle controller 150 includes a CPU, a storage unit, and an input / output interface. In the vehicle controller 150, the CPU, the storage unit, and the input / output interface are interconnected, for example, via an internal bus and an interface circuit. The input / output interface communicates with internal devices such as a drive unit 110 installed on the vehicle 10. The input / output interface is connected to the vehicle communication unit 140 to make communication possible. The chassis 160 is a chassis component that supports various devices 110, 120, 130, 140, 150 installed on the vehicle 10, as well as a body shell 180, various components 190, etc. to be assembled in subsequent manufacturing steps. The configuration of the platform-shaped vehicle 10 is not limited to the above configuration.

[0031] The body shell assembly step is a manufacturing step of assembling the body shell 180 to the platform vehicle 101. The platform vehicle 101 and the vehicle 102 to which the body shell 180 is assembled (hereinafter, first assembled vehicle 102) differ in appearance due to differences in appearance shapes.

[0032] The component assembly step is a step of assembling the exterior component 190. The first assembled vehicle 102 and the vehicle 103 (hereinafter, the second assembled vehicle 103) to which the exterior component 190 is assembled are different in appearance due to the difference in appearance shape. At least a part of the interior components may be assembled to the platform vehicle 101 before the body shell assembly step is performed, and may be assembled to the platform vehicle 101 before or after the component assembly step is performed.

[0033] As described above, the vehicle 10 is classified into a plurality of states according to the appearance of the vehicle 10 that changes in a plurality of manufacturing steps. At this time, the difference in appearance between the platform vehicle 101 and the first assembled vehicle 102 is, for example, smaller than the difference in appearance between the first assembled vehicle 102 and the second assembled vehicle 103. Therefore, in the present embodiment, the state in which the vehicle 10 is in a platform form is referred to as a "platform state" as in the case of the platform vehicle 101. As in the case of the first assembled vehicle 102 and the second assembled vehicle 103, the state in which at least the vehicle body shell 180 is assembled to the platform vehicle 101 is referred to as an "assembled state". In other words, in the present embodiment, the vehicle 10 is classified into a plurality of states according to the difference in the appearance shape of the vehicle 10. At this time, the vehicle 10 may be classified so that the state of the vehicle 10 in one of the manufacturing steps is one of the states according to whether the difference in the appearance shape of the vehicle 10 is large or small, or may be classified so that the states of the vehicle 10 in a plurality of manufacturing steps are collectively one of the states. The number of manufacturing steps and the types of manufacturing steps in the manufacturing process of the vehicle 10, the number of states of the vehicle 10, and the types of states of the vehicle 10 classified according to a plurality of manufacturing steps are not limited to the above.

[0034] In the embodiment of the present disclosure, the vehicle 10 is at least any one of the vehicle 10 as a finished product and the vehicles 101, 102, 103, 104, 105 as semi-finished products and products in process in the manufacturing steps of manufacturing the finished product. Hereinafter, when the vehicles 101, 102, 103, 104, 105 in a plurality of states do not need to be distinguished from each other, the vehicles 101, 102, 103, 104, 105 are simply referred to as "vehicles 10".

[0035] Figure 3 1 is a block diagram showing the configuration of the detection device 5 according to the first embodiment. The detection device 5 detects at least the external shape (contour) of the vehicle 10 contained in the captured image by identifying a target area as an area representing the vehicle 10 among the areas in the captured image. The detection device 5 includes a communication unit 51, a storage unit 53, and a CPU 52. In the detection device 5, the communication unit 51, the storage unit 53, and the CPU 52 are connected to each other via, for example, an internal bus and an interface circuit.

[0036] The communication unit 51 of the detection device 5 connects the detection device 5 with other devices such as the vehicle controller 150, the position calculation device 6, the remote controller 7, and the camera 9 to enable communication. The communication unit 51 of the detection device 5 is, for example, a wireless communication device.

[0037] The storage unit 53 of the detection device 5 stores various information, including various programs for controlling the operation of the detection device 5, the first detection model Md1 as a detection model, the step database Db1, and the distortion correction parameter Pa1. The storage unit 53 of the detection device 5 includes, for example, RAM, ROM, and a hard disk drive (HDD).

[0038] The detection model is a trained machine learning model for detecting the vehicle 10 included in the captured image. In the present embodiment, the storage unit 53 of the detection device 5 stores a plurality of first detection models Md1 each prepared one by one for each state as the detection model.

[0039] The first detection model Md1 is a machine learning model trained by inputting a first training data set. The first training data set is prepared one by one for each state. The first training data set has a plurality of training images each containing a vehicle 10, and region correct labels respectively associated with a plurality of regions constituting each training image. The region correct label is a correct label indicating whether each region in each region of the training image represents a target region of the vehicle 10 or a non-target region representing an object other than the vehicle 10. In the present embodiment, each region in each region constituting the training image is a pixel as a component of the training image. Each region in each region constituting the training image may be a plurality of pixels as a plurality of components of the training image. Each region in each region constituting the training image is classified as any one of a target region and a non-target region, for example, based on a calculation result obtained by comparing RGB values ​​of adjacent regions as components of the training image. Each region in each region constituting the training image may be classified as any one of a target region and a non-target region based on a calculated probability that the region is a target region. When a region correct label is assigned to one of the regions constituting the training image, it is associated with information indicating the position of the region with the region correct label in the training image. When each of the respective regions constituting the training image is one pixel, the information indicating the position in the training image is, for example, pixel coordinates indicating the pixel position in the training image.

[0040] The first training data set includes M (M is an integer greater than or equal to 2) first training images and N (N is an integer greater than or equal to zero and less than M) second training images as multiple training images. The first training image is an image containing a vehicle 10 classified as one of the states. The second training image is an image containing a vehicle 10 classified as another state different from the above-mentioned one of the states. The number M of first training images included in the first training data set may be, for example, greater than or equal to 1.5 times the number N of second training images, or may be greater than or equal to 2 times the number N of second training images. When there are multiple other states different from one of the states, the number of first training images included in the first training data set is, for example, greater than the total number of second training images respectively containing vehicles 10 classified as other states different from the above-mentioned one of the states. In other words, the first detection model Md1 is a machine learning model that is trained on the features of the vehicle 10 classified as one of the states by learning that the amount of the first training images is greater than the amount of the second training images.

[0041] In the present embodiment, the storage unit 53 of the detection device 5 stores two first detection models Md1a and Md1b suitable for detecting one of the states determined by the appearance shape of the vehicle 10. The first detection model Md1a is a first detection model Md1 that makes the number of vehicles 10 classified as "platform state" learned by it greater than the number of vehicles 10 classified as states other than "platform state" (for example, "assembled state"). The first detection model Md1a is used to detect the vehicle 10 classified as "platform state". The first detection model Md1b is a first detection model Md1 that makes the number of vehicles 10 classified as "assembled state" learned by it greater than the number of vehicles 10 classified as states other than "assembled state" (for example, "platform state"). The first detection model Md1b is used to detect the vehicle 10 classified as "assembled state". The number of the first detection models Md1 and the type of each first detection model Md1 are not limited to the above.

[0042] In this embodiment, when a captured image is input into the first detection model Md1, the first detection model Md1 identifies a target region among various regions constituting the input captured image. Then, the first detection model Md1 applies a mask to the target region to generate a first mask image with the mask region added to the target region.

[0043] For example, the algorithm of the first detection model Md1 uses a deep neural network (DNN) having a convolutional neural network (CNN) structure that implements semantic segmentation or instance segmentation. The configuration of the first detection model Md1 is not limited to the above. The first detection model Md1 can be, for example, a trained machine learning model that uses an algorithm other than a neural network.

[0044] The step database Db1 is a database in which state information is associated with each of a plurality of manufacturing steps. The state information is information indicating one of the states of the vehicle 10. In other words, the state information is information for identifying a plurality of states in the manufacturing process of the vehicle 10. The configuration of the step database Db1 is not limited to the above.

[0045] The distortion correction parameter Pa1 is a parameter for correcting the distortion of a captured image. The details of the distortion correction parameter Pa1 will be described later.

[0046] Figure 4 : is a block diagram showing the details of the CPU 52 mounted on the detection device 5 according to the first embodiment. The CPU 52 of the detection device 5 functions as an image acquisition unit 521, a distortion correction unit 522, a rotation processing unit 523, a cropping processing unit 524, a state acquisition unit 525, and a model acquisition unit 526 by expanding various programs stored in the storage unit 53 of the detection device 5. In addition, the CPU 52 of the detection device 5 functions as a detection unit 527 and a first transmission unit 528 by expanding various programs stored in the storage unit 53 of the detection device 5.

[0047] The image acquisition unit 521 acquires an original image from the imaging device 9 .

[0048] The distortion correction unit 522 corrects the distortion of the captured image. In the present embodiment, the distortion correction unit 522 generates a corrected image in which the distortion of the original image is corrected.

[0049] The rotation processing unit 523 rotates the captured image so that the direction of a vector indicating the moving direction of the vehicle 10 (hereinafter referred to as a moving vector) points in a predetermined direction. In the present embodiment, the rotation processing unit 523 generates a rotated image by rotating the corrected image.

[0050] The cropping processing unit 524 deletes the area (hereinafter referred to as the non-essential area) other than the area (hereinafter referred to as the necessary area) composed of the vehicle 10 and the area around the vehicle 10 from the various areas constituting the captured image. Therefore, the cropping processing unit 524 cuts out the necessary area from the captured image. In the present embodiment, when the vehicle 10 has moved a distance greater than a predetermined threshold, the cropping processing unit 524 deletes the area that has moved according to the distance that has moved and is a non-essential area from the rotated image. Therefore, the cropping processing unit 524 generates a processed image in which the area that has not moved and is a necessary area containing the vehicle 10 is cut out from the rotated image.

[0051] The state acquisition unit 525 acquires step information about one of the manufacturing steps performed on the vehicle 10 included in the captured image, and state information indicating one of the states of the vehicle 10 included in the captured image. The step information is, for example, a unique identifier (ID) assigned to each manufacturing step for identifying a plurality of manufacturing steps included in the manufacturing process of the vehicle 10. The state acquisition unit 525 identifies one of the manufacturing steps performed on the vehicle 10 included in the captured image by, for example, detecting feature points with which a plurality of manufacturing steps can be identified using information acquired by a sensor (hereinafter referred to as an on-board sensor) mounted on the vehicle 10. The on-board sensor is, for example, any one of an on-board camera, an on-board laser radar, and an on-board radar. The on-board camera captures an image of the state of the area around the vehicle 10. The on-board laser radar and the on-board radar detect objects present in the area around the vehicle 10. Then, the state acquisition unit 525 acquires the state information about the vehicle 10 included in the captured image by using the step database Db1 to identify the state information associated with one of the manufacturing steps identified by the acquired step information. The method of acquiring the step information and the state information is not limited to the above.

[0052] The model acquisition unit 526 acquires the first detection model Md1 selected according to one of the states identified based on the state information acquired by the state acquisition unit 525. In the present embodiment, the model acquisition unit 526 selects and acquires the first detection model Md1 according to one of the states identified based on the state information acquired by the state acquisition unit 525 from among the plurality of first detection models Md1 prepared one by one for each state stored in the storage unit 53 of the detection device 5. In other words, the model acquisition unit 526 selects the first detection model Md1 that causes the learned features of the vehicle 10 classified as one of the states identified based on the state information acquired by the state acquisition unit 525 to be greater than the features of the vehicle 10 classified as another state different from the above one of the states. For example, when the state of the vehicle 10 contained in the captured image is identified as a "platform state" based on the state information acquired by the state acquisition unit 525, the model acquisition unit 526 acquires the first detection model Md1a that is caused to be learned that the vehicle 10 classified as the "platform state" is larger than the vehicle 10 classified as another state.

[0053] The detection unit 527 recognizes the target area from the various areas constituting the captured image by inputting the captured image into the first detection model Md1 acquired by the model acquisition unit 526. Therefore, the detection unit 527 detects the vehicle 10 included in the captured image. In the present embodiment, the detection unit 527 inputs the processed image into the first detection model Md1 selected by the model acquisition unit 526. Therefore, by recognizing the target area from the various areas constituting the processed image, a first mask image is generated from the processed image.

[0054] The first sending unit 528 sends various information to other devices except the detection device 5. The first sending unit 528, for example, sends the result detected by the detection unit 527 to the position calculation device 6. In this embodiment, the result detected by the detection unit 527 is the first mask image. At least one or a part of the functions of the detection device 5 can be implemented as a function of any one of the vehicle controller 150, the position calculation device 6, the remote controller 7, and the camera 9.

[0055] Figure 5is a block diagram showing the configuration of the position calculation device 6. The position calculation device 6 calculates the position of the vehicle 10 contained in the captured image by using the result detected by the detection device 5. The position calculation device 6 adopts the measurement point 10e of the vehicle 10 as the position of the vehicle 10. In the present embodiment, the position calculation device 6 calculates the position of the vehicle 10 contained in the captured image by using the first mask image. The position calculation device 6 includes a communication unit 61, a storage unit 63, and a CPU 62. In the position calculation device 6, the communication unit 61, the storage unit 63, and the CPU 62 are connected to each other via, for example, an internal bus and an interface circuit.

[0056] The communication unit 61 of the position calculation device 6 connects the position calculation device 6 with the vehicle controller 150, the detection device 5, the remote controller 7, and the camera 9 to enable communication. The communication unit 61 of the position calculation device 6 is, for example, a wireless communication device.

[0057] The storage unit 63 of the position calculation device 6 stores various information including various programs for controlling the operation of the position calculation device 6, perspective transformation parameters Pa2 and camera database Db2. The storage unit 63 of the position calculation device 6 includes, for example, RAM, ROM and a hard disk drive (HDD).

[0058] The perspective transformation parameter Pa2 is a parameter for performing perspective transformation of the first mask image. The details of the perspective transformation parameter Pa2 will be described later.

[0059] The camera database Db2 is a database showing the camera parameters calculated based on the installation position of the camera 9 in the global coordinate system for each camera 9. The camera parameters are parameters related to the distance between the camera 9 and a predetermined reference point. In this embodiment, the camera parameters are the height H ( Fig.11 (described later)), and is a height H relative to the road surface 20 on which the vehicle 10 is located. The camera database Db2 is, for example, a database that associates camera identification information with camera parameters on the camera device 9 identified by the camera identification information. The configuration of the camera database Db2 is not limited to the above. The camera range RG, installation position, installation angle, etc. of the camera device 9 can be associated with the camera identification information.

[0060] The CPU 62 of the position calculation device 6 functions as a data acquisition unit 621 , a perspective transformation unit 622 , a coordinate point calculation unit 623 , a position transformation unit 624 , and a second transmission unit 625 by expanding various programs stored in the storage unit 63 of the position calculation device 6 .

[0061] The data acquisition unit 621 acquires various information. In this embodiment, the data acquisition unit 621 acquires the first mask image from the detection device 5. Further, the data acquisition unit 621 acquires the camera parameters on the camera device 9 as the source of acquiring the analysis target original image by querying the camera database Db2 stored in the storage unit 63 of the position calculation device 6.

[0062] The perspective transformation unit 622 generates a second mask image by performing a perspective transformation of the first mask image. The coordinate point calculation unit 623 calculates an image coordinate point by correcting the first coordinate point using the second coordinate point. The first coordinate point is a coordinate point of a specified vertex of a first circumscribed rectangle set for a mask area in the first mask image in the image coordinate system. The second coordinate point is a coordinate point indicating the same position as the first coordinate point among the vertices of a second circumscribed rectangle set for a mask area in the second mask image in the image coordinate system. The image coordinate point is a coordinate point indicating the position of the vehicle 10 in the image coordinate system. The position transformation unit 624 transforms the image coordinate point into a vehicle coordinate point by using the imaging parameters acquired by the data acquisition unit 621 and the image coordinate point calculated by the coordinate point calculation unit 623. The vehicle coordinate point is a coordinate point indicating the position of the vehicle 10 in the global coordinate system.

[0063] The second sending unit 625 sends various information to other devices except the position calculation device 6. For example, the second sending unit 625 sends the vehicle coordinate point as information indicating the position of the vehicle 10 to the remote controller 7. At least one or a part of the functions of the position calculation device 6 can be implemented as a function of any one of the vehicle controller 150, the detection device 5, the remote controller 7, and the camera device 9.

[0064] Figure 6 is a block diagram showing the configuration of the remote controller 7. The remote controller 7 creates a control value that defines the driving operation of the vehicle 10 by using the position information of the vehicle 10, etc., and sends the control value to the vehicle 10. Therefore, the remote controller 7 performs remote control on the operation of the vehicle 10. The remote controller 7 includes a communication unit 71, a storage unit 73, and a CPU 72. The communication unit 71, the storage unit 73, and the CPU 72 of the remote controller 7 are connected to each other, for example, via an internal bus and an interface circuit. The communication unit 71 of the remote controller 7 connects the remote controller 7 with the vehicle controller 150, the detection device 5, the position calculation device 6, and the camera device 9 to enable communication. The communication unit 71 of the remote controller 7 is, for example, a wireless communication device. The storage unit 73 of the remote controller 7 stores various programs for controlling the operation of the remote controller 7. The storage unit 73 of the remote controller 7 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0065] The CPU 72 of the remote controller 7 functions as an information acquisition unit 721 , a control value creation unit 722 , and a third transmission unit 723 by expanding various programs stored in the storage unit 73 of the remote controller 7 .

[0066] The information acquisition unit 721 acquires various information. The information acquisition unit 721 acquires, for example, information about driving conditions of the vehicle 10 (hereinafter referred to as driving information). The driving information includes, for example, a vehicle coordinate point as information indicating the position of the vehicle 10 transmitted from the remote controller 7, a driving speed and an actual steering angle of the vehicle 10 transmitted from the vehicle controller 150, and driving route information pre-stored in the storage unit 73 of the remote controller 7. The driving route information is information indicating a target driving route of the vehicle 10 traveling in the remote automatic driving mode. The type of information included in the driving information is not limited to the above.

[0067] The control value creation unit 722 creates a control value that defines the driving operation of the vehicle 10 by using the driving information acquired by the information acquisition unit 721. Specifically, the control value creation unit 722 creates, for example, a reference control value and a correction control value. The reference control value is a control value for causing the vehicle 10 to travel along the target driving route. The correction control value is a control value for correcting the relative position of the vehicle 10 relative to the target driving route. Each of the reference control value and the correction control value includes, for example, an acceleration control value that defines the acceleration of the vehicle 10 in the forward driving direction and a steering angle control value that defines the steering angle of the vehicle 10. Instead of the control value including the acceleration control value and the steering angle control value, each of the reference control value and the correction control value may also be a control value including any one of a trajectory control value and a destination control value. The trajectory control value is a control value that defines the moving path of the vehicle 10 by arranging the target moving positions of the vehicle 10 set at predetermined time intervals in chronological order. The destination control value is a control value that indicates a target arrival time of the target arrival place of the vehicle 10.

[0068] The third sending unit 723 sends various information to other devices other than the remote controller 7. For example, the third sending unit 723 sends the control value created by the control value creating unit 722 to the vehicle 10 as the target to be controlled. At least one or a part of the functions of the remote controller 7 may be implemented as a function of any one of the vehicle controller 150, the detection device 5, the position calculation device 6, and the camera device 9. A-2. Detection method and position calculation method for vehicle

[0069] Figure 7 is a flowchart showing a detection method and a position calculation method for the vehicle 10 according to the first embodiment. Figure 7The method shown in , for example, is repeatedly executed at predetermined time intervals when or after the vehicle 10 as the target to be detected starts traveling in the remote automatic driving mode.

[0070] Figure 8 It shows that when executing Figure 7 Schematic diagram of examples of various images when the detection method and position calculation method shown in FIG. Figure 8 In the Figure 7 In this embodiment, the case where the vehicle 10 travels (moves) along the driving direction on the road 20 on which the grid lines 21 are drawn along the Xg axis parallel to the driving direction of the vehicle 10 and the Yg axis orthogonal to the Xg axis is described as an example. The Xg axis and the Yg axis are coordinate axes in the global coordinate system. In another embodiment, the grid lines 21 can be omitted.

[0071] like Figure 7 As shown in , first, an image acquisition step (step S1) is performed. Figure 8 As shown in , the image acquisition step is a step of acquiring an original image Im1 including the vehicle 10. In the present embodiment, in the image acquisition step, the image acquisition unit 521 acquires the original image Im1 from the camera 9.

[0072] like Figure 7 As shown in FIG. 1 , after the image acquisition step, a distortion correction step (step S2) is performed. The distortion correction step is a step of correcting the distortion of the captured image. In this embodiment, as shown in FIG. Figure 8 As shown in , in the distortion correction step, the distortion correction unit 522 generates a corrected image Im2 by correcting the distortion of the original image Im1. Specifically, the distortion correction unit 522, for example, corrects the distortion of the original image Im1 by using the distortion correction parameter Pa1 pre-stored in the storage unit 53 of the detection device 5. The distortion correction parameter Pa1 is, for example, a parameter related to the position information of the grid line 21 obtained by calibration. The distortion correction method is not limited to the above. The distortion correction parameter Pa1 may be a selected parameter other than the above.

[0073] like Figure 7 As shown in FIG. 1 , after the distortion correction step, a rotation processing step (step S3) is performed. The rotation processing step is a step of rotating the captured image so that the direction of the moving vector of the vehicle 10 contained in the captured image points to a predetermined direction. In this embodiment, as shown in FIG. Figure 8As shown in , in the rotation processing step, the rotation processing unit 523 rotates the corrected image Im2 so that the direction of the moving vector of the vehicle 10 contained in the corrected image Im2 points to a predetermined direction. Therefore, the rotation processing unit 523 generates a rotated image Im3. Specifically, the rotation processing unit 523, for example, rotates the corrected image Im2 around the center of gravity of the vehicle 10 in the corrected image Im2 so that the direction of the moving vector of the vehicle 10 points upward on the screen of the display device that displays the corrected image Im2. The movement of the feature point (for example, the center of gravity) of the vehicle 10 can be represented as the direction of the moving vector by the optical flow method, for example. The moving vector amount and the moving vector direction of the vehicle 10 are estimated based on, for example, the position change of the feature point set as needed on the corrected image Im2 between image frames. The rotation processing method is not limited to the above.

[0074] like Figure 7 As shown in FIG. 1 , after the rotation processing step, the cropping processing step (step S4) is performed. The cropping processing step is a step of cutting out a necessary area from the captured image. In this embodiment, as shown in FIG. Figure 8 As shown in , in the cropping processing step, when the vehicle 10 has moved a distance greater than a predetermined threshold value, the cropping processing unit 524 deletes the moved area A2 according to the distance the vehicle 10 has moved as a non-essential area from the rotated image Im3. Therefore, the cropping processing unit 524 generates a processed image Im4 in which the non-moved area A1 including the vehicle 10 is cut out from the rotated image Im3 as a necessary area. At this time, the cropping processing unit 524 estimates the moving area A2 by, for example, identifying the distance the vehicle 10 has moved from the estimated moving vector amount of the vehicle 10. In the detection method for the vehicle 10, any one of the rotation processing step and the cropping processing step may be performed first. The cropping processing method is not limited to the above.

[0075] like Figure 7 As shown in , after the cropping process step, the state acquisition step (step S5) is performed. The state acquisition step is a step of acquiring state information indicating one of the states of the vehicle 10 contained in the captured image. In the present embodiment, in the state acquisition step, the state acquisition unit 525 first acquires step information about one of the manufacturing steps performed on the vehicle 10 contained in the captured image. Then, the state acquisition unit 525 uses the step database Db1 to identify the state information associated with one of the manufacturing steps identified based on the acquired step information. Therefore, the state acquisition unit 525 acquires the state information about the vehicle 10 contained in the captured image. The state acquisition step only needs to be performed at any time point after the image acquisition step is completed and before the detection step (step S7) starts.

[0076] After the state acquisition step, the model acquisition step (step S6) is performed. The model acquisition step is a step of acquiring a first detection model Md1 selected according to one of the states identified based on the state information acquired in the state acquisition step from a plurality of first detection models Md1 prepared one by one for each state. In the present embodiment, in the model acquisition step, the model acquisition unit 526 selects and acquires the first detection model Md1 from a plurality of first detection models Md1 pre-stored in the storage unit 53 of the detection device 5 according to one of the states identified based on the state information acquired in the state acquisition step. The model acquisition step only needs to be performed at any time point after the state acquisition step is completed and before the detection step begins.

[0077] After the model acquisition step, the detection step is performed. Figure 8 As shown in , the detection step is a step of detecting the outer shape of the vehicle 10 contained in the captured image by using a detection model (such as a first detection model Md1). In the present embodiment, in the detection step, the detection unit 527 inputs the processed image Im4 into the first detection model Md1 selected in the model acquisition step. Therefore, the detection unit 527 detects the outer shape of the vehicle 10 contained in the processed image Im4. Then, the detection unit 527 generates a first mask image Im5 in which the mask area Ms is added to the target area by applying a mask to the target area in the processed image Im4. The first sending unit 528 sends the first mask image Im5 to the position calculation device 6. The detection method for the vehicle 10 is not limited to the above.

[0078] like Figure 7As shown in , after the detection step, a perspective transformation step (step S8) is performed. The perspective transformation step is a step of performing a perspective transformation of the first mask image Im5. In the perspective transformation step, the data acquisition unit 621 acquires the first mask image Im5 from the detection device 5. Then, the perspective transformation unit 622 generates the second mask image Im6 by performing a perspective transformation of the first mask image Im5. Specifically, the perspective transformation unit 622, for example, performs a perspective transformation of a bird's-eye view image when overlooking the first mask image Im5 from a viewpoint substantially perpendicular to the road surface 20 above the vehicle 10 (for example, directly above the vehicle 10) by using the perspective transformation parameter Pa2 pre-stored in the storage unit 63 of the position calculation device 6. The perspective transformation parameter Pa2 is, for example, a parameter regarding the position information of the camera device 9 obtained by calibration and an internal parameter. Therefore, the perspective transformation unit 622 generates the second mask image Im6 shown in the image coordinate system based on the first mask image Im5 shown in the camera coordinate system. The image coordinate system is a coordinate system having a point of the image plane onto which the image is projected by perspective transformation as an origin, and having coordinate axes represented by the Xi axis and the Yi axis orthogonal to the Xi axis. The perspective transformation method is not limited to the above. The perspective transformation parameter Pa2 may be a selected parameter other than the above.

[0079] like Figure 7 As shown in , after the perspective transformation step, a coordinate point calculation step (step S9) is performed. Fig. 9 1 is a view for explaining the details of the coordinate point calculation step. The coordinate point calculation step is a step of calculating an image coordinate point P3 indicating the position of the vehicle 10 in the image coordinate system.

[0080] In the coordinate point calculation step, the coordinate point calculation unit 623 calculates the base coordinate point P0 based on the first circumscribed rectangle R1 set for the mask region Ms in the first mask image Im5 which is the image before perspective transformation. Fig.10: is a view for explaining a calculation method for a basic coordinate point P0. In order to calculate the basic coordinate point P0, the coordinate point calculation unit 623 sets a basic circumscribed rectangle R0 for the mask area Ms in the first mask image Im5. Subsequently, the coordinate point calculation unit 623 rotates the first mask image Im5 by a necessary rotation amount around the center of gravity C of the mask area Ms so that the direction of the moving vector V of the vehicle 10 corresponding to the mask area Ms in the first mask image Im5 points to a predetermined direction. The predetermined direction is, for example, an upward direction on the screen of a display device displaying the first mask image Im5. Subsequently, the coordinate point calculation unit 623 sets a first circumscribed rectangle R1 for the mask area Ms of the rotated first mask image Im5 so that its long side is parallel to the direction of the moving vector. Subsequently, the coordinate point calculation unit 623 rotates the first mask image Im5 to which the first circumscribed rectangle R1 is added in the reverse direction around the center of gravity C of the mask area M by the rotation amount. Therefore, the coordinate point calculation unit 623 sets the base coordinate point P0 as the coordinate point of the vertex of one of the four vertices of the first circumscribed rectangle R1 having the coordinate closest to the measurement point 10e of the vehicle 10. Then, as Fig. 9 As shown in , the coordinate point calculation unit 623 performs perspective transformation on the first mask image Im5 after reverse rotation (i.e., the first mask image Im5 after the basic coordinate point P0 is calculated). Therefore, the coordinate point calculation unit 623 sets the first coordinate point P1 as the coordinate point corresponding to the basic coordinate point P0 in the first circumscribed rectangle R1 deformed by the perspective transformation.

[0081] Further, the coordinate point calculation unit 623 sets a second circumscribed rectangle R2 for the mask area Ms in the second mask image Im6 obtained by performing perspective transformation on the first mask image Im5. Then, the coordinate point calculation unit 623 sets the second coordinate point P2 as a vertex indicating the same position as the first coordinate point P1 among the vertices of the second circumscribed rectangle R2. In other words, the first coordinate point P1 and the second coordinate point P2 are coordinate points indicating the same position, and therefore the first coordinate point P1 and the second coordinate point P2 are related to each other.

[0082] Further, the coordinate point calculation unit 623 performs correction to replace the coordinates (Xi1, i1) of the first coordinate point P1 with the coordinates (Xi2, i2) of the second coordinate point P2 based on the size relationship between the coordinate values ​​of the first coordinate point P1 and the coordinate values ​​of the second coordinate point P2. When the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction (Xi1>Xi2), the coordinate point calculation unit 623 replaces the coordinate value Xi1 of the first coordinate point P1 in the Xi direction with the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. When the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is greater than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction (Yi1>Yi2), the coordinate point calculation unit 623 replaces the coordinate value Yi1 of the first coordinate point P1 in the Yi direction with the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. Fig. 9 In the example shown in , the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. The coordinate value i1 of the first coordinate point P1 in the Yi direction is less than the coordinate value i2 of the second coordinate point P2 in the Yi direction. Therefore, the image coordinate point P3 has the coordinates (Xi2, i1). In this way, the coordinate point calculation unit 623 calculates the image coordinate point P3 indicating the position of the vehicle 10 in the image coordinate system by correcting the first coordinate point P1 using the second coordinate point P2. The calculation method of the image coordinate point P3 is not limited to the above.

[0083] like Figure 7 As shown in , after the coordinate point calculation step, the position transformation step (step S10) is performed. The position transformation step is a step of calculating a vehicle coordinate point indicating the position of the vehicle 10 in the global coordinate system by transforming the image coordinate point P3 into a vehicle coordinate point. In the position transformation step, the position transformation unit 624 transforms the image coordinate point P3 into a vehicle coordinate point by using the camera parameters and the vehicle parameters acquired by the data acquisition unit 621. The vehicle parameter is a parameter regarding the distance between the measurement point 10e of the vehicle 10 and the reference point. In the present embodiment, the vehicle parameter is the height h ( Fig.11 (described later)).

[0084] The position transformation unit 624 transforms the image coordinate point P3 into the vehicle coordinate point by using relational expressions (1) to (3) (described later) including the vehicle coordinate point as a response variable and including the image coordinate point P3, the imaging parameter, and the vehicle parameter as explanatory variables. In this case, the position transformation unit 624 substitutes the coordinate value of the image coordinate point P3 calculated by the coordinate point calculation unit 623 into the relational expressions represented by the expressions (1) to (3). Further, the position transformation unit 624 substitutes the imaging parameter acquired by the data acquisition unit 621, that is, the value of the imaging parameter corresponding to the imaging device 9 that has acquired the original image Im1, into the relational expressions represented by the expressions (1) to (3).

[0085] Fig.11 It is a first view for explaining the details of the position conversion step. Fig.11 A state is shown when the vehicle 10 is viewed from the left-hand side. Fig.12 It is a second view for explaining the details of the position conversion step. Fig.12 A state is shown when the vehicle 10 is viewed from the top side. Fig.11 and Fig.12 The global coordinate system shown in is a coordinate system having an origin set at a fixed coordinate point Pf indicating a selected reference position of the road surface 20, and having coordinate axes represented by an Xg axis and a Yg axis orthogonal to the Xg axis. The imaging coordinate point Pc is a coordinate point indicating the position of the imaging device 9 that has acquired the original image Im1 used to calculate the image coordinate point P3, and indicating the position of the imaging device 9 in the global coordinate system. The fixed coordinate point Pf and the imaging coordinate point Pc are stored in advance in the storage unit 63 of the position calculation device 6.

[0086] like Fig.11 As shown in , the observation distance between the position of the camera 9 and the position of the vehicle 10 (image coordinate point P3) on the Xg-Yg plane is defined as Do. The observation error is defined as ΔD. The height [m] of the camera 9 from the road surface 20 is defined as H as a camera parameter. The height [m] of the measurement point 10e of the vehicle 10 from the road surface 20 is defined as h as a vehicle parameter. In this case, the observation error ΔD is represented by the following expression (1). ΔD = h / H × Do (1) In other words, the observation error ΔD increases as the observation distance Do increases.

[0087] Then, the actual distance (hereinafter referred to as a first distance) between the position of the imaging device 9 and the position of the measurement point 10 e of the vehicle 10 is D, and the first distance D is represented by the following expression (2). D = Do × (1 - h / H) (2) In other words, the first distance D is determined based on the observation distance Do, the height H of the imaging device 9 as an imaging parameter, and the height h of the measurement point 10 e of the vehicle 10 as a vehicle parameter.

[0088] Then, if Fig.12 As shown in , when the estimated distance between the reference position and the position of the vehicle 10 is Dp, and the actual distance between the reference position and the vehicle 10 (hereinafter referred to as the second distance) is Dt, the second distance Dt is represented by the following expression (3). Dt = Dp × (1 - h / H) (3)

[0089] Here, the estimated distance Dp can be calculated by using the actual distance (hereinafter referred to as the third distance Dc) obtained from the fixed coordinate point Pf and the camera coordinate point Pc, the image coordinate point P3, and the fixed coordinate point Pf. Therefore, the position transformation unit 624 is allowed to calculate the vehicle coordinate point Pv by using the fixed coordinate point Pf and the second distance Dt obtained by correcting the estimated distance Dp by using expression (3). At this time, the calculated vehicle coordinate point Pv is a coordinate point indicating the position of the vehicle 10 in the global coordinate system, so the calculated vehicle coordinate point Pv corresponds to the position of the vehicle 10 in the actual space. The second sending unit 625 sends the vehicle coordinate point Pv to the remote controller 7. A-3. Operation control method for vehicle

[0090] Fig.13 is a flow chart illustrating an operation control method for vehicle 10 moving in a remote automated driving mode. Fig.13 The operation control method shown in , for example, is repeatedly executed each time the vehicle coordinate point Pv is received from the position calculation device 6 as information indicating the position of the vehicle 10 .

[0091] The information acquisition unit 721 of the remote controller 7 acquires the driving information including the vehicle coordinate point Pv (step S101). Then, the control value creation unit 722 creates the control value that defines the driving operation of the vehicle 10 by using the driving information (step S102). Then, the third sending unit 723 sends the control value to the vehicle 10 (step S103). The vehicle controller 150 installed on the vehicle 10 drives the drive unit 110, etc. according to the received control value (step S104).

[0092] According to the first embodiment, when the vehicle 10 is caused to move automatically by remote control, the position calculation system 1 is able to calculate the position of the vehicle 10 by using a captured image containing the vehicle 10. Specifically, the detection device 5 inputs the captured image into a detection model that is a machine learning model capable of detecting the vehicle 10 contained in the captured image. Therefore, the detection device 5 is able to detect the vehicle 10 contained in the captured image. Then, the position calculation device 6 is able to calculate the vehicle coordinate point Pv indicating the position of the vehicle 10 contained in the captured image by using the first mask image Im5 as a result of detection by the detection device 5.

[0093] Here, the appearance of the vehicle 10 varies in a plurality of manufacturing steps, so the vehicle 10 can be classified into a plurality of states according to the difference in appearance. Therefore, the features extracted when training the detection model depend on the appearance state of the vehicle 10. Therefore, when the detection model that is caused to learn a plurality of training images respectively containing vehicles 10 classified into different states only in the same number for each state is also applied to all vehicles 10 respectively classified into each state, the accuracy of detecting the vehicle 10 in one or a part of the states will decrease. When the accuracy of detecting the vehicle 10 in one or some states decreases, it is conceivable to train the detection model again by adding training images containing the vehicle 10 in the one or some states in which the detection accuracy decreases. However, when the learned training images containing the vehicle 10 in one or some states are larger than the training images containing the vehicle 10 in other states, the features contained in the added training images may be learned more deeply than the other features. As a result, new differences in detection accuracy may appear among the plurality of states. When the detection accuracy of the vehicle 10 contained in the captured image decreases, the calculation accuracy of the vehicle coordinate point Pv indicating the position of the vehicle 10 decreases. When the calculation accuracy of the vehicle coordinate point Pv decreases, a desired control value may not be created.

[0094] For this reason, in the first embodiment, the first detection model Md1 is caused to learn a larger number of first training images containing the vehicle 10 classified into one of the states than the second training images containing the vehicle 10 classified into the other state. Therefore, the first detection model Md1 suitable for detecting the vehicle 10 classified into one of the states is prepared one by one for each state. According to the first embodiment, the detection device 5 can acquire state information indicating one of the states of the vehicle 10 contained in the captured image, and select the first detection model Md1 according to one of the states identified by the acquired state information from the plurality of first detection models Md1 prepared one by one for the respective states. In other words, when the position of the vehicle 10 moving in the factory is calculated, etc., the first detection model Md1 can be selected according to the state of the vehicle 10 classified into any one of the plurality of states according to the appearance of the vehicle 10 that changes in a plurality of manufacturing steps. Then, the detection device 5 inputs the captured image into the first detection model Md1 according to the state of the vehicle 10 contained in the captured image. Therefore, the detection device 5 can accurately detect the vehicle 10 contained in the captured image. In other words, when the first detection model Md1 suitable for detecting the vehicle 10 classified into one of the states is selected, the respective areas constituting the captured image can be accurately classified into the target area and the non-target area. Therefore, it is possible to suppress the reduction in the detection accuracy of the vehicle 10 included in the captured image in accordance with the difference in appearance in a plurality of manufacturing steps. As a result, it is possible to suppress the reduction in the calculation accuracy of the vehicle coordinate point Pv. Therefore, it is possible to reduce the difference between the position of the vehicle 10 calculated by the position calculation device 6 and the actual position of the vehicle 10. Therefore, it is possible to create a more appropriate control value.

[0095] According to the first embodiment, the first training data set includes M (M is an integer greater than or equal to 2) first training images and N (N is an integer greater than or equal to zero and less than M) second training images. In this case, the first training data set may include only the first training images, but not the second training images. In this case, for example, since the number M of first training images each containing the vehicle 10 classified into one of the states increases, the detection accuracy of the first detection model Md1 can be improved. The first training data set may include the first training images, and the number N of second training images that is less than the first training images. In this case, since the first training data set includes the second training images, for example, the detection accuracy of a portion of the vehicle 10 that has the same appearance regardless of one of the states can be improved.

[0096] According to the first embodiment, when the form of the vehicle 10 is a platform form, the state of the vehicle 10 is allowed to be classified as a platform state. Then, when detecting a vehicle 10 classified as a platform state, the vehicle 10 can be detected by selecting a first detection model Md1 suitable for detecting a vehicle 10 classified as a platform state and inputting a captured image. When detecting a vehicle 10 classified as a platform state, the vehicle 10 can be detected by inputting a captured image associated with the fact that the vehicle 10 is in a platform state into a second detection model Md2. Therefore, the detection accuracy of the vehicle 10 classified as a platform state can be improved.

[0097] According to the first embodiment, when the platform vehicle 101 in the platform form is included in the captured image, it is possible to detect the outer shape of the platform vehicle 101. Therefore, the position of the vehicle 10 can be calculated by using the captured image not only for the vehicle 10 as a finished vehicle but also for the vehicle 10 as a semi-finished product and a product in process.

[0098] According to the first embodiment, the platform vehicle 101 is significantly different in appearance from the first assembled vehicle 102 and the second assembled vehicle 103. Therefore, the state of the vehicle 10 can be classified into a platform state and an assembled state before and after assembling a vehicle body such as a vehicle body shell 180. Therefore, it is possible to suppress a decrease in the detection accuracy of the vehicle 10 included in the captured image in accordance with the difference in the appearance shape of the vehicle 10 before and after the vehicle body assembly.

[0099] According to the first embodiment, when detecting the vehicle 10 included in the captured image, the corrected image Im2 can be generated by correcting the distortion of the original image Im1. With this configuration, the detection accuracy of the vehicle 10 included in the captured image can be improved. Therefore, the calculation accuracy of the position of the vehicle 10 can be further improved.

[0100] According to the first embodiment, when detecting the vehicle 10 included in the captured image, the corrected image Im2 is rotated so that the direction of the movement vector V of the vehicle 10 points to a predetermined direction, thereby generating a rotated image Im3. As a result, the vehicle 10 included in the captured image can be detected in a state where the direction of the movement vector V is unified. Therefore, the detection accuracy of the vehicle 10 included in the captured image can be improved.

[0101] According to the first embodiment, when detecting the vehicle 10 included in the captured image, a processed image Im4 that does not include an unnecessary area can be generated by performing a cropping process to cut out a necessary area including the vehicle 10 from the rotated image Im3. Therefore, the area of ​​the vehicle 10 in the captured image can be increased compared to the case where the cropping process step is not performed. Therefore, the vehicle 10 that is farther away from the camera 9 can be easily detected. Therefore, the detection accuracy of the vehicle 10 that is farther away from the camera 9 can be improved.

[0102] According to the first embodiment, when the processed image Im4 is input into the first detection model Md1, a mask is applied to the target area among the various areas constituting the processed image Im4. Therefore, a first mask image Im5 in which the mask area Ms is added to the target area can be generated. At this time, according to the first embodiment, the algorithm of the first detection model Md1 can use a DNN having a CNN structure that realizes semantic segmentation or instance segmentation. Therefore, the reduction in detection accuracy of the vehicle 10 due to various non-target areas in the captured image can be suppressed.

[0103] According to the first embodiment, the second mask image Im6 can be generated by performing perspective transformation on the first mask image Im5. Therefore, the camera coordinate system can be transformed into the image coordinate system.

[0104] According to the first embodiment, when the first circumscribed rectangle R1 is set for the mask region Ms before the perspective transformation is performed on the first mask image Im5, the basic coordinate point P0 can be calculated, which is the vertex in the first circumscribed rectangle R1 whose coordinates are closest to the measurement point 10e of the vehicle 10. Then, after the basic coordinate point P0 is calculated, the first coordinate point P1 as the coordinate point corresponding to the basic coordinate point P0 can be calculated by performing the perspective transformation on the first mask image Im5. Further, when the second circumscribed rectangle R2 is set for the mask region Ms of the second mask image Im6, the second coordinate point P2 can be calculated, which is the vertex in the second circumscribed rectangle R2 whose coordinates are closest to the measurement point 10e of the vehicle 10. Then, the image coordinate point P3 can be calculated by correcting the first coordinate point P1 using the second coordinate point P2. In this way, the image coordinate point P3 can be further accurately calculated by comparing and correcting the coordinate points before and after the perspective transformation. Therefore, the calculation accuracy of the position of the vehicle 10 can be further improved.

[0105] According to the first embodiment, it is possible to acquire step information of one of the manufacturing steps being performed on the vehicle 10. Then, when the state information about the vehicle 10 is acquired by identifying the state information associated with the one of the manufacturing steps identified by the acquired step information using the step database Db1, the state of the vehicle 10 contained in the captured image can be identified.

[0106] According to the first embodiment, the imaging parameters of the imaging device 9 that has acquired the original image Im1 can be acquired. Then, when the calculated image coordinate point P3 and the acquired imaging parameter values ​​are substituted into a relational expression including the vehicle coordinate point Pv as a response variable and including the image coordinate point P3, the imaging parameter, and the vehicle parameter as explanatory variables, the image coordinate point P3 can be transformed into the vehicle coordinate point Pv. Therefore, the image coordinate system can be transformed into the global coordinate system, and the position of the measurement point 10e of the vehicle 10 in the global coordinate system can be calculated as the vehicle coordinate point Pv.

[0107] According to the first embodiment, the imaging parameter is the height H of the imaging device 9 from the road surface 20 calculated based on the position of the imaging device 9 in the global coordinate system. The vehicle parameter is the height h of the measurement point 10e of the vehicle 10 from the road surface 20. As a result, the observation error ΔD can be calculated based on the similarity relationship between the imaging parameter and the vehicle parameter. Then, the image coordinate point P3 can be transformed into the vehicle coordinate point Pv by using the calculated observation error ΔD.

[0108] According to the first embodiment, the position of the vehicle 10 can be calculated without the vehicle 10 having any installation (such as a marker and a transceiver) for calculating the position of the vehicle 10. The position of the vehicle 10 can be calculated without the position calculation device 6 being installed in the vehicle 10. Therefore, the versatility of the position calculation system 1 can be improved. B. Second Embodiment

[0109] Fig.14 FIG. 2 is a diagram showing a manufacturing process of the vehicle 10 according to the second embodiment. Fig.14 , some of the multiple manufacturing steps performed in the manufacturing process of the vehicle 10 are representatively shown. Fig.15 1 is a block diagram showing the configuration of the detection device 5a according to the second embodiment. The present embodiment differs from the first embodiment in a part of the manufacturing process of the vehicle 10. Therefore, a part of the configuration of the detection device 5a is different from that of the first embodiment. The remaining configuration is similar to that of the first embodiment. The same components as those of the first embodiment are assigned similar reference symbols, and their description is omitted.

[0110] like Fig.14As shown in, in this embodiment, the vehicle 10 as a finished product is manufactured by assembling the body shell 180 to the platform vehicle 101, then assembling the internal components and the external components 190, and painting the body (body part) composed of the body shell 180, the external components 190, etc. In other words, in this embodiment, the vehicle 10 as a finished product is manufactured by sequentially performing the platform manufacturing step, the body shell assembly step, the component assembly step and the painting step. The painting step is a step of painting the vehicle 10. The vehicle 104 before being painted in the painting step (hereinafter referred to as the unpainted vehicle 104) and the vehicle 105 after being painted in the painting step (hereinafter referred to as the painted vehicle 105) are different from each other in appearance color, so the appearance of the vehicle 10 is also different. Therefore, in this embodiment, the storage unit 53a of the detection device 5a stores three first detection models Md1c, Md1d, and Md1e as detection models suitable for detecting the vehicle 10 classified as one of the states determined according to the appearance shape and appearance color of the vehicle 10. In the following description, the state of the vehicle 10 before being painted in the painting step (such as the unpainted vehicle 104) is referred to as the "unpainted state". The state of the vehicle 10 after being painted in the painting step (such as the painted vehicle 105) is referred to as the "painted state".

[0111] The first detection model Md1c is a first detection model Md1 that makes the number of vehicles 10 classified as "platform state" that it learns to be greater than the number of vehicles 10 classified as states other than "platform state". In the first detection model Md1c, the first training image is an image containing a vehicle 10 classified as "platform state". The second training image is, for example, an image containing a vehicle 10 classified as "assembled state" and "unpainted state". The first detection model Md1c is used to detect a vehicle 10 classified as "platform state".

[0112] The first detection model Md1d is a first detection model Md1 that makes the number of vehicles 10 classified as "assembled" and "unpainted" that it learns to be greater than the number of vehicles 10 classified as states other than "assembled" and "unpainted". In the first detection model Md1d, the first training image is an image containing a vehicle 10 classified as "assembled" and "unpainted". The second training image is, for example, an image containing a vehicle 10 classified as "assembled" and "painted". The first detection model Md1d is used to detect a vehicle 10 in an "assembled" and "unpainted" state.

[0113] The first detection model Md1e is a first detection model Md1 that makes the number of vehicles 10 classified as "assembled state" and "painted state" learned by it greater than the number of vehicles 10 classified as states other than "assembled state" and "painted state". In the first detection model Md1e, the first training image is an image containing vehicles 10 classified as "assembled state" and "painted state". The second training image is, for example, an image containing vehicles 10 classified as "assembled state" and "unpainted state". The first detection model Md1e is used to detect vehicles 10 classified as "assembled state" and "painted state".

[0114] According to the second embodiment, the unpainted vehicle 104 and the painted vehicle 105 are significantly different in appearance from each other. Therefore, the state of the vehicle 10 can be classified into an unpainted state and a painted state before and after being painted in the painting step. Therefore, the reduction in the detection accuracy of the vehicle 10 included in the captured image can be suppressed based on the difference in the appearance color of the vehicle 10 before and after the painting step is performed. C. Third Embodiment

[0115] Fig.16 is a block diagram showing the configuration of the detection device 5b according to the third embodiment. This embodiment is different from the first embodiment in the detection model for detecting the vehicle 10 contained in the captured image. Therefore, part of the configuration of the detection device 5b and part of the processing of the detection method for the vehicle 10 are different from those of the first embodiment. The remaining configuration is similar to that of the first embodiment. The same steps as those in the first embodiment and the same components as those in the first embodiment are assigned similar reference symbols, and their descriptions are omitted.

[0116] In the present embodiment, the storage unit 53b of the detection device 5b stores a second detection model Md2 as a detection model. The second detection model Md2, as well as the first detection model Md1, is a trained machine learning model for detecting the vehicle 10 contained in the captured image. The second detection model Md2 is a machine learning model trained by inputting a second training data set. The second training data set has a plurality of training images, each of which contains a vehicle 10 classified into different states, a region correct label associated with each of a plurality of regions constituting each training image, and a state correct label associated with each of the plurality of training images. As in the case of the first embodiment, the region correct label is a correct label indicating whether each of the regions in the training image represents a target region of the vehicle 10 or a non-target region representing an object other than the vehicle 10. The state correct label is a correct label indicating one of the states of the vehicle 10 contained in the training image. In the present embodiment, when the captured image is input into the second detection model Md2 together with one of the states identified by the state information acquired by the state acquisition unit 525, the second detection model Md2 performs the following processing. In this case, the second detection model Md2 identifies the target area contained in the input captured image. Then, the second detection model Md2 applies a mask to the target area to generate a first mask image Im5 in which the mask area Ms is added to the target area. For example, CNN is used as the algorithm of the second detection model Md2. The configuration of the second detection model Md2 is not limited to the above. The second detection model Md2 can be, for example, a trained machine learning model using an algorithm other than a neural network.

[0117] Fig.17 is a block diagram showing the details of the CPU 52b installed on the detection device 5b according to the third embodiment. The CPU 52b of the detection device 5b according to the present embodiment includes a model acquisition unit 526b, instead of the model acquisition unit 526 according to the first embodiment. The model acquisition unit 526b acquires the second detection model Md2. The CPU 52b of the detection device 5b includes a detection unit 527b, instead of the detection unit 527 according to the first embodiment. The detection unit 527b inputs the captured image and one of the states identified by the state information acquired by the state acquisition unit 525 into the second detection model Md2 to identify the target area from among the various areas constituting the input captured image. Therefore, the detection unit 527b detects the vehicle 10 contained in the captured image.

[0118] Fig.18 is a flowchart showing a detection method and a position calculation method for the vehicle 10 according to the third embodiment. Fig.18The method shown in , for example, is repeatedly executed at predetermined time intervals when or after the vehicle 10 as a target to be detected starts moving in the remote automatic driving mode.

[0119] In this embodiment, in the model acquisition step (step S6b), the model acquisition unit 526b acquires the second detection model Md2. Then, in the detection step (step S7b), the detection unit 527b inputs the processed image Im4 and one of the states identified by the state information acquired by the state acquisition unit 525 into the second detection model Md2. Therefore, the detection unit 527b detects the shape of the vehicle 10 contained in the processed image Im4. Then, the detection unit 527b generates a first mask image Im5 in which the mask area Ms is added to the target area by applying a mask to the target area in the processed image Im4.

[0120] According to the third embodiment, a second detection model Md2 is prepared as a detection model trained when each of a plurality of training images each containing a vehicle 10 is associated with a state correct label indicating the state of the vehicle 10 contained in the training image. Therefore, when a captured image associated with the state of the vehicle 10 is input into the second detection model Md2, the vehicle 10 contained in the input captured image can be accurately detected. In other words, the respective regions constituting the captured image input into the second detection model Md2 can be accurately classified into target regions and non-target regions. Therefore, the reduction in the detection accuracy of the vehicle 10 contained in the captured image can be suppressed based on the difference in appearance in a plurality of manufacturing steps. As a result, the reduction in the calculation accuracy of the vehicle coordinate point Pv can be suppressed. Therefore, the difference between the position of the vehicle 10 calculated by the position calculation device 6 and the actual position of the vehicle 10 can be reduced. Therefore, a more appropriate control value can be created. D. Fourth Embodiment

[0121] Fig.19 1 is a block diagram showing the configuration of the detection device 5c according to the fourth embodiment. The present embodiment is different from the first embodiment in the method of obtaining state information. Therefore, part of the configuration of the detection device 5c is different from that of the first embodiment. The remaining configuration is similar to that of the first embodiment. The same components as the first embodiment are assigned similar reference symbols, and their description is omitted.

[0122] In the present embodiment, the storage unit 53c of the detection device 5c stores a specific model Md3 instead of storing the step database Db1. The specific model Md3 is a trained machine learning model for identifying one of the states of the vehicle 10 contained in the captured image and obtaining state information indicating one of the identified states. Specifically, the specific model Md3 is a trained machine learning model that is caused to output state information about the vehicle 10 contained in the captured image when the captured image is input. The specific model Md3 learns features according to the state of the vehicle 10 to identify one of the states of the vehicle 10. For example, CNN is used as the algorithm of the specific model M3. The configuration of the specific model Md3 is not limited to the above. The specific model Md3 may be, for example, a trained machine learning model that uses an algorithm other than a neural network.

[0123] Fig. 20 is a block diagram showing details of the CPU 52c mounted on the detection device 5c according to the fourth embodiment. In this embodiment, the state acquisition unit 525c acquires state information on the vehicle 10 contained in the captured image by inputting the captured image into the specific model Md3.

[0124] According to the fourth embodiment, the specific model Md3 is trained to output state information about the vehicle 10 contained in the captured image when the captured image is input, and when the captured image is input into the specific model Md3, the state information about the vehicle 10 contained in the captured image can be acquired. In other words, the state information of the vehicle 10 can be acquired by using machine learning. E. Fifth Embodiment

[0125] Fig.21 1 is a block diagram showing the configuration of the detection device 5d according to the fifth embodiment. The present embodiment is different from the first embodiment in the method of acquiring state information. Therefore, part of the configuration of the detection device 5d is different from that of the first embodiment. The remaining configuration is similar to that of the first embodiment. The description of the same parts as those in the first embodiment is omitted.

[0126] In the present embodiment, the storage unit 53d of the detection device 5d stores the state database Db3 instead of the step database Db1. The state database Db3 is a database that associates the camera information about the camera device 9 with the state information. The camera information is information for identifying the camera device 9 that has acquired the original image Im1. The camera information is, for example, camera identification information. In this case, the state database Db3 is, for example, a database that associates the camera identification information, the camera range RG of the camera device 9 identified by the camera identification information, and the state information indicating the state of the vehicle 10 according to the manufacturing steps performed in the camera range RG. The configuration of the state database Db3 and the camera information is not limited to the above. The camera information may include, for example, the camera range RG instead of the camera identification information.

[0127] Fig. 22 5 is a block diagram showing the details of the CPU 52d mounted on the detection device 5d according to the fifth embodiment. In the present embodiment, the state acquisition unit 525d acquires the camera identification information indicating the camera 9 that has acquired the original image Im1 as the imaging information about the camera 9 that has acquired the original image Im1. Then, the state acquisition unit 525d acquires the state information about the vehicle 10 contained in the captured image by identifying the state information associated with one camera 9 identified by the acquired imaging information using the state database Db3.

[0128] According to the fifth embodiment, it is possible to acquire imaging information about the imaging device 9. Then, when the status information associated with one imaging device 9 identified by the acquired imaging information is identified by using the status database Db3 that associates imaging information with status information, it is possible to acquire status information indicating the status of the vehicle 10 included in the captured image. Therefore, it is possible to acquire the status information about the vehicle 10 included in the captured image without performing image analysis, and thus it is possible to reduce the processing load of the status acquisition unit 525d in the status acquisition step. F. Alternative Embodiments F-1. First Alternative Embodiment

[0129] The vehicle 10 as a finished product can be manufactured by assembling a plurality of painted units. In this case, the vehicle 10 as a finished product can be, for example, manufactured by assembling a front unit, a rear unit, a left unit, a right unit, a top unit, a bottom unit, etc. The front unit is a semi-finished product formed by integrally molding a component group forming the front side of the vehicle 10. The front unit includes, for example, a headlight, a front bumper, a front grille, a hood, etc. The rear unit is a semi-finished product formed by integrally molding a component group forming the rear side of the vehicle 10. The rear unit includes, for example, a rear light, a rear bumper, and a trunk. The left unit is a semi-finished product formed by integrally molding a component group forming the left side of the vehicle 10. The right unit is a semi-finished product formed by integrally molding a component group forming the right side of the vehicle 10. Each of the left unit and the right unit includes, for example, a side beam and a column. The top unit is a semi-finished product formed by integrally molding a component group forming the upper side of the vehicle 10. The top unit includes, for example, a roof. The bottom unit is a semi-finished product formed by integrally molding a component group forming the bottom side of the vehicle 10. The bottom unit includes, for example, a chassis 160 and a seat. In this form, the state of the vehicle 10 can also be classified into a plurality of states according to the appearance of the vehicle 10. Therefore, by inputting a captured image into a first detection model Md1 suitable for one of the states determined based on the appearance of the vehicle 10, or inputting a captured image associated with the state of the vehicle 10 into a second detection model Md2, it is possible to suppress a decrease in the detection accuracy of the vehicle 10 according to the appearance of the vehicle 10. F-2. Second Alternative Embodiment

[0130] In the captured image, as the contrast between the vehicle 10 and the line 2 decreases, the color difference between the target area and the non-target area decreases, and therefore, the detection accuracy of the vehicle 10 based on the exterior color of the vehicle 10 and the color of the line 2 may decrease. For this reason, the first detection model Md1 for detecting the vehicle 10 classified as a painted state can be prepared in multiple numbers based on the exterior color of the vehicle 10. When, for example, a vehicle 10 classified as a painted state is detected, the following two first detection models Md1 can be prepared. In this case, the first detection model Md1 for detecting the vehicle 10 painted with a light-colored paint having a brightness higher than or equal to a predetermined threshold, and the first detection model Md1 for detecting the vehicle 10 painted with a dark-colored paint having a brightness lower than the threshold can be prepared. According to such a configuration, the state of the vehicle 10 can be classified into multiple states based on the color of the paint used in the painting step. Therefore, the decrease in the accuracy of detecting the vehicle 10 contained in the captured image based on the color of the paint used in the painting step, that is, the difference in the exterior color of the vehicle 10, can be suppressed. F-3. Third Alternative Embodiment

[0131] The training images included in the first training data set and the second training data set may be images similar to the original image Im1 that have not been subjected to processing such as distortion correction, rotation, and cropping. According to such a configuration, when preparing the first training data set and the second training data set, image processing is not required. Therefore, the processing load when training the first detection model Md1 and the second detection model Md2 can be reduced. F-4. Fourth Alternative Embodiment

[0132] Among them, the training images included in the first training data set and the second training data set can each be any one of the corrected image Im2, the rotated image Im3, and the processed image Im4. According to such a configuration, as in the case of the first embodiment and the third embodiment, when the captured image obtained by processing the original image Im1 is input to any one of the first detection model Md1 and the second detection model Md2, the detection accuracy of the vehicle 10 can be improved. F-5. Fifth Alternative Embodiment

[0133] exist Figure 7 and Fig.18 In the detection method for the vehicle 10 shown in , the distortion correction step is not an indispensable step. For example, when the steps in the rotation processing step are performed without performing the distortion correction step, the rotation processing unit 523 rotates the original image Im1 in the rotation processing step instead of rotating the corrected image Im2. According to such a configuration, the image coordinate point P3 can also be calculated. F-6. Sixth Alternative Embodiment

[0134] exist Figure 7 and Fig.18 In the detection method for the vehicle 10 shown in , the rotation processing step is not an indispensable step. When, for example, the distortion correction step and the rotation processing step are not performed, the cropping processing unit 524 crops the original image Im1 instead of the rotated image Im3 in the cropping processing step. According to such a configuration, the image coordinate point P3 can also be calculated. F-7. Seventh Alternative Embodiment

[0135] exist Figure 7 and Fig.18 In the detection method for the vehicle 10 shown in , the cropping process step is not an indispensable step. When, for example, the distortion correction step, the rotation process step, and the cropping process step are not performed, the detection unit 527 or the detection unit 527b generates the first mask image Im5 in which the mask area Ms is added in the original image Im1 instead of in the processed image Im4 in the detection step. According to such a configuration, the image coordinate point P3 can also be calculated. F-8. Eighth Alternative Embodiment

[0136] The captured image may include a plurality of vehicles 10. In this case, the CPU 52 of the detection device 5, the CPU 52 of the detection device 5a, the CPU 52b of the detection device 5b, the CPU 52c of the detection device 5c, or the CPU 52d of the detection device 5d may include, for example, a deletion unit for deleting a mask area Ms on the vehicle 10 to be excluded from the target of position calculation from the first mask image Im5. The deletion unit determines, for example, among the mask areas Ms generated in the detection step, a mask area Ms existing outside the recognition target area as a mask area of ​​the vehicle 10 to be excluded from the target of position calculation, and deletes the mask area Ms existing outside the recognition target area from the first mask image Im5. The recognition target area is, for example, a predetermined area in which the vehicle 10 moves in the first mask image Im5. The predetermined area in which the vehicle 10 moves is, for example, an area corresponding to an area having grid lines 21. The recognition target area is pre-stored in the storage unit 53 of the detection device 5, the storage unit 53a of the detection device 5a, the storage unit 53b of the detection device 5b, the storage unit 53c of the detection device 5c, or the storage unit 53d of the detection device 5d. According to such a configuration, when a plurality of vehicles 10 are included in the captured image, the influence of the vehicle 10 other than the target set in the position calculation can also be excluded. Therefore, the calculation accuracy of the position of the vehicle 10 can be improved. F-9. Ninth Alternative Embodiment

[0137] The captured image may contain a plurality of vehicles 10. In this case, a DNN that performs instance segmentation may be used as an algorithm for the first detection model Md1 and an algorithm for the second detection model Md2. According to such a configuration, a plurality of vehicles 10 contained in the captured image may be classified, and a first mask image Im5 having a mask for each of the vehicles 10 may be generated. Therefore, when a plurality of vehicles 10 are contained in the captured image, a vehicle 10 that is a target of position calculation may be selected, and the position of the selected vehicle 10 may be calculated. F-10. Tenth Alternative Embodiment

[0138] The position calculation system 1 can calculate the position of the stationary vehicle 10. When calculating the position of the stationary vehicle 10, the position calculation system 1 calculates the position of the vehicle 10 by, for example, using the initial vector direction of the vehicle 10 estimated from the original image Im1 acquired for the first time after the position calculation system 1 is started, instead of using the direction of the moving vector V of the moving vehicle 10. According to such a configuration, when the vehicle 10 is stopped, the position of the vehicle 10 can also be calculated by using the captured image. F-11. Eleventh Alternative Embodiment

[0139] At least some of the detection devices 5, 5a, 5b, 5c, 5d, the position calculation device 6, and the remote controller 7 may be provided as one body. In addition, the respective units of the detection devices 5, 5a, 5b, 5c, 5d, the position calculation device 6, and the remote controller 7 may be implemented, for example, by cloud computing composed of one or more computers. According to such a configuration, the configurations of the detection devices 5, 5a, 5b, 5c, 5d, the position calculation device 6, and the remote controller 7 may be changed as needed. F-12. Twelfth Alternative Embodiment

[0140] The image acquisition unit 521 of each detection device 5, 5a, 5b, 5c, 5d can acquire the original image Im1 via other external devices (such as the remote controller 7) other than the camera 9, without directly acquiring the original image Im1 from the camera 9. According to such a configuration, the image acquisition unit 521 of each of the detection devices 5, 5a, 5b, 5c, 5d can also acquire the original image Im1 acquired by the camera 9. F-13. Thirteenth Alternative Embodiment

[0141] The detection device 5 can detect the vehicle 10 by using the first detection model Md1 stored in other devices except the detection device 5 and selected by other devices except the detection device 5. The detection device 5a can detect the vehicle 10 by using the first detection model Md1 stored in other devices except the detection device 5a and selected by other devices except the detection device 5a. The detection device 5b can detect the vehicle 10 by using the first detection model Md1 stored in other devices except the detection device 5b and selected by other devices except the detection device 5b. The detection device 5c can detect the vehicle 10 by using the first detection model Md1 stored in other devices except the detection device 5c and selected by other devices except the detection device 5c. The detection device 5d can detect the vehicle 10 by using the first detection model Md1 stored in other devices except the detection device 5d and selected by other devices except the detection device 5d. According to such a configuration, the first detection model Md1 according to the state of the vehicle 10 can also be acquired.

[0142] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various ways without departing from the intention of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in the various aspects described in the summary of the invention can be replaced or combined as needed to solve some or all of the above-mentioned inconveniences, or to achieve some or all of the above-mentioned beneficial effects. When technical features are not described as indispensable in the specification, these technical features can be deleted as needed.

Claims

1. a detection device configured to detect a vehicle contained in a captured image, The vehicle is configured to move in a factory where a plurality of manufacturing steps are performed to manufacture and ship the vehicle, The vehicle is classified into a plurality of states according to an appearance of the vehicle, the appearance changing during the plurality of manufacturing steps, It is characterized in that The detection device comprises a first processor, wherein the first processor is configured to: Acquire the captured image; acquiring state information indicating one of the states of the vehicle contained in the captured image; From among a plurality of first detection models that are machine learning models prepared one by one for the plurality of states, a first detection model selected according to the one of the states identified by the acquired state information is obtained; as well as The vehicle included in the captured image is detected by inputting the captured image into the acquired first detection model and identifying a target area representing the vehicle in the captured image.

2. The detection device according to claim 1, characterized in that: Each of the multiple first detection models is pre-trained to identify the target area by inputting multiple training images, and the multiple training images include M (M is an integer greater than or equal to 2) first training images each containing a vehicle classified as one of the states, and N (N is an integer greater than or equal to zero and less than M) second training images each containing a vehicle classified as another state different from one of the states.

3. The detection device according to claim 2, characterized in that: Each region in the training image is associated with a region correct label, and the region correct label indicates which of the target region and a non-target region representing an object other than the vehicle.

4. The detection device according to claim 1, characterized in that: The first processor is configured to obtain the status information by: obtaining step information about one of the manufacturing steps being performed on the vehicle, and identifying the status information associated with the one of the manufacturing steps identified by the obtained step information by using a step database that associates the status information with each of the multiple manufacturing steps.

5. The detection device according to claim 1, characterized in that: The first processor is configured to, when the captured image is input, acquire the state information by: inputting the captured image into a specific model that is a trained machine learning model to output the state information.

6. The detection device according to claim 1, characterized in that: The first processor is configured to acquire the status information by acquiring imaging information about the imaging devices and identifying the status information associated with one of the imaging devices identified by the acquired imaging information by using a status database that associates the imaging information with the status information.

7. The detection device according to claim 1, characterized in that: The plurality of manufacturing steps include painting the vehicle, and The plurality of states include an unpainted state indicating a state of the vehicle before being painted in painting the vehicle and a painted state indicating a state of the vehicle after being painted in painting the vehicle.

8. The detection device according to claim 1, characterized in that: The first processor is configured to correct distortion of the captured image.

9. The detection device according to claim 1, characterized in that: The first processor is configured to rotate the captured image so that the moving direction of the vehicle points to a predetermined direction.

10. A position calculation system configured to calculate a position of a vehicle contained in a captured image, characterized in that: The position calculation system include: According to the detection device of claim 1, the first processor is further configured to generate a first mask image having a mask area added to the target area as an area representing the vehicle in the captured image by applying a mask to the target area; as well as A position calculation device configured to calculate the position of the vehicle, the position calculation device comprising a second processor, the second processor being configured to: generating a second mask image by performing a perspective transformation of the first mask image, calculating an image coordinate point indicating a position of the vehicle in an image coordinate system by correcting the first coordinate point using the second coordinate point, when a specified vertex of a first circumscribed rectangle set for the mask area in the first mask image is a first coordinate point, and a vertex of a second circumscribed rectangle set for the mask area in the second mask image indicating the same position as the first coordinate point is a second coordinate point; as well as The image coordinate point is converted into a vehicle coordinate point indicating the position of the vehicle in the global coordinate system by using a distance between the camera and a reference point calculated based on the position of the camera in the global coordinate system and a distance between a predetermined measurement point of the vehicle and the reference point.

11. A detection device configured to detect a vehicle contained in a captured image, The vehicle is configured to move in a factory where a plurality of manufacturing steps are performed to manufacture and ship the vehicle, The vehicle is classified into a plurality of states according to an appearance of the vehicle, the appearance changing during the plurality of manufacturing steps, The plurality of states include a platform state, wherein the platform state is a state of a platform in which the vehicle includes at least: wheel, Chassis, a drive unit configured to accelerate the vehicle, a steering device configured to change the direction of travel of the vehicle, a braking device configured to decelerate the vehicle, a vehicle controller configured to control operation of the vehicle, and a vehicle communication unit configured to communicate with other devices other than the host vehicle, It is characterized in that The detection device comprises a first processor, wherein the first processor is configured to: Acquire the captured image; acquiring state information indicating one of the states of the vehicle contained in the captured image; From among a plurality of first detection models that are machine learning models prepared one by one for the plurality of states, a first detection model selected according to the one of the states identified by the acquired state information is obtained; as well as The vehicle included in the captured image is detected by inputting the captured image into the acquired first detection model and identifying a target area representing the vehicle in the captured image.

12. The detection device according to claim 11, characterized in that: Each of the multiple first detection models is pre-trained to identify the target area by inputting multiple training images, and the multiple training images include M (M is an integer greater than or equal to 2) first training images each containing a vehicle classified as one of the states, and N (N is an integer greater than or equal to zero and less than M) second training images each containing a vehicle classified as another state different from one of the states.

13. The detection device according to claim 12, characterized in that: Each region in the training image is associated with a region correct label, and the region correct label indicates which of the target region and a non-target region representing an object other than the vehicle.

14. The detection device according to claim 11, characterized in that: The first processor is configured to obtain the status information by: obtaining step information about one of the manufacturing steps being performed on the vehicle, and identifying the status information associated with the one of the manufacturing steps identified by the obtained step information by using a step database that associates the status information with each of the multiple manufacturing steps.

15. The detection device according to claim 11, characterized in that: The first processor is configured to, when the captured image is input, acquire the state information by: inputting the captured image into a specific model that is a trained machine learning model to output the state information.

16. The detection device according to claim 11, characterized in that: The first processor is configured to acquire the status information by acquiring imaging information about the imaging devices and identifying the status information associated with one of the imaging devices identified by the acquired imaging information by using a status database that associates the imaging information with the status information.

17. The detection device according to claim 11, characterized in that: The plurality of manufacturing steps include painting the vehicle, and The plurality of states include an unpainted state indicating a state of the vehicle before being painted in painting the vehicle and a painted state indicating a state of the vehicle after being painted in painting the vehicle.

18. The detection device according to claim 11, characterized in that: The first processor is configured to correct distortion of the captured image.

19. The detection device according to claim 11, characterized in that: The first processor is configured to rotate the captured image so that the moving direction of the vehicle points to a predetermined direction.

20. A position calculation system configured to calculate a position of a vehicle contained in a captured image, characterized in that: The position calculation system include: According to the detection device of claim 11, the first processor is further configured to generate a first mask image having a mask area added to the target area as a region representing the vehicle in the captured image by applying a mask to the target area; as well as A position calculation device configured to calculate the position of the vehicle, the position calculation device comprising a second processor, the second processor being configured to: generating a second mask image by performing a perspective transformation of the first mask image, calculating an image coordinate point indicating a position of the vehicle in an image coordinate system by correcting the first coordinate point using the second coordinate point, when a specified vertex of a first circumscribed rectangle set for the mask area in the first mask image is a first coordinate point, and a vertex of a second circumscribed rectangle set for the mask area in the second mask image indicating the same position as the first coordinate point is a second coordinate point; as well as The image coordinate point is converted into a vehicle coordinate point indicating the position of the vehicle in the global coordinate system by using a distance between the camera and a reference point calculated based on the position of the camera in the global coordinate system and a distance between a predetermined measurement point of the vehicle and the reference point.

21. A detection device configured to detect a vehicle contained in a captured image, The vehicle is configured to move in a factory where a plurality of manufacturing steps are performed to manufacture and ship the vehicle, The vehicle is classified into a plurality of states according to an appearance of the vehicle, the appearance changing during the plurality of manufacturing steps, It is characterized in that The detection device comprises a first processor, wherein the first processor is configured to: Acquire the captured image; acquiring state information indicating one of the states of the vehicle contained in the captured image; and The vehicle contained in the captured image is detected by inputting the captured image and one of the states identified by the acquired state information into a second detection model serving as a machine learning model, and identifying a target area indicating the vehicle in the captured image.

22. The detection device according to claim 21, characterized in that: The second detection model is pre-trained to identify the target area by inputting multiple training images each containing the vehicle and a state-correct label, and the state-correct label is associated with each of the multiple training images and indicates the state of the vehicle contained in the training image.

23. The detection device according to claim 22, characterized in that: Each region in the training image is associated with a region correct label, and the region correct label indicates which of the target region and a non-target region representing an object other than the vehicle.

24. The detection device according to claim 21, characterized in that: The first processor is configured to obtain the status information by: obtaining step information about one of the manufacturing steps being performed on the vehicle, and identifying the status information associated with the one of the manufacturing steps identified by the obtained step information by using a step database that associates the status information with each of the multiple manufacturing steps.

25. The detection device according to claim 21, characterized in that: The first processor is configured to, when the captured image is input, acquire the state information by: inputting the captured image into a specific model that is a trained machine learning model to output the state information.

26. The detection device according to claim 21, characterized in that: The first processor is configured to acquire the status information by acquiring imaging information about the imaging devices and identifying the status information associated with one of the imaging devices identified by the acquired imaging information by using a status database that associates the imaging information with the status information.

27. The detection device according to claim 21, characterized in that: The plurality of manufacturing steps include painting the vehicle, and The plurality of states include an unpainted state indicating a state of the vehicle before being painted in painting the vehicle and a painted state indicating a state of the vehicle after being painted in painting the vehicle.

28. The detection device according to claim 21, characterized in that: The first processor is configured to correct distortion of the captured image.

29. The detection device according to claim 21, characterized in that: The first processor is configured to rotate the captured image so that the moving direction of the vehicle points to a predetermined direction.

30. A position calculation system configured to calculate a position of a vehicle contained in a captured image, characterized in that: The position calculation system include: According to the detection device of claim 21, the first processor is further configured to generate a first mask image having a mask area added to the target area by applying a mask to the target area in the captured image as an area representing the vehicle; as well as A position calculation device configured to calculate the position of the vehicle, the position calculation device comprising a second processor, the second processor being configured to: generating a second mask image by performing a perspective transformation of the first mask image, calculating an image coordinate point indicating a position of the vehicle in an image coordinate system by correcting the first coordinate point using the second coordinate point, when a specified vertex of a first circumscribed rectangle set for the mask area in the first mask image is a first coordinate point, and a vertex of a second circumscribed rectangle set for the mask area in the second mask image indicating the same position as the first coordinate point is a second coordinate point; as well as The image coordinate point is converted into a vehicle coordinate point indicating the position of the vehicle in the global coordinate system by using a distance between the camera and a reference point calculated based on the position of the camera in the global coordinate system and a distance between a predetermined measurement point of the vehicle and the reference point.

31. A detection method for detecting a vehicle contained in a captured image, The vehicle is configured to move in a factory where a plurality of manufacturing steps are performed to manufacture and ship the vehicle, The vehicle is classified into a plurality of states according to an appearance of the vehicle, the appearance changing during the plurality of manufacturing steps, It is characterized in that The detection method comprises: Acquire the captured image; acquiring state information indicating one of the states of the vehicle contained in the captured image; From among a plurality of first detection models that are machine learning models prepared one by one for the plurality of states, obtaining a first detection model selected according to the one of the states identified by the acquired state information; and The vehicle included in the captured image is detected by inputting the captured image into the acquired first detection model and identifying a target area representing the vehicle in the captured image.

32. A detection method for detecting a vehicle contained in a captured image, The vehicle is configured to move in a factory where a plurality of manufacturing steps are performed to manufacture and ship the vehicle, The vehicle is classified into a plurality of states according to an appearance of the vehicle, the appearance changing during the plurality of manufacturing steps, It is characterized in that The detection method comprises: Acquire captured images; acquiring state information indicating one of the states of the vehicle contained in the captured image; and The vehicle contained in the captured image is detected by inputting the captured image and one of the states identified by the acquired state information into a second detection model which is a machine learning model, and identifying a target area representing the vehicle in the captured image.

Images