Remote control system, remote control device, and remote control method

Abstract

Provided is a technique for improving the likelihood of obtaining a detection image suitable for unmanned driving. A system is provided with: a moving body that moves by unmanned driving using a predetermined detection site included in a detection image in which the moving body is captured; an image acquisition unit that acquires a detection image; an environment information acquisition unit that acquires environment information indicating an environment in which a moving body moves by unmanned driving and which is likely to affect the detection image; and a first determination unit that determines, on the basis of the acquired environment information, at least one of a portion of the moving body to be used as the detection portion and an imaging direction of the detection image.

Description

Technical Field

[0001] The present disclosure relates to a system, a control device and a control method. Background Art

[0002] Patent Document 1 discloses a technology for driving a vehicle by unmanned driving in a vehicle manufacturing process. Prior art literature Patent Literature

[0003] Patent Document 1: Japanese Patent Application No. 2017-538619 Summary of the invention Technical problem to be solved by the invention

[0004] There is a known technology for moving a moving object such as a vehicle by unmanned driving using images captured by a camera. However, if only the moving object is captured, for example, when the moving object is exposed to strong sunlight or is turning, it may not be possible to obtain images suitable for unmanned driving. Means for solving technical problems

[0005] The present disclosure can be implemented as the following aspects.

[0006] (1) According to one aspect of the present disclosure, a system is provided. The system comprises: a mobile body that moves by an unmanned driving system, the unmanned driving system using a predetermined detection portion included in a detection image in which the mobile body is photographed; an image acquisition unit that acquires the detection image; an environment information acquisition unit that acquires environment information representing an environment in which the mobile body moves by the unmanned driving system and which may affect the acquired detection image; and a first determination unit that determines at least one of a portion of the mobile body used as the detection portion and a shooting direction of the detection image based on the acquired environment information. According to this aspect, the part of the moving body used as the detection part and the shooting direction of the detection image can be determined according to the environment in which the moving body moves by unmanned driving. Therefore, the possibility of obtaining a detection image suitable for unmanned driving of the moving body is increased. (2) In the above method, the environmental information may also include at least one of information indicating the time when the mobile body moves by the unmanned driving and information indicating the place where the mobile body moves by the unmanned driving. According to such a method, the part of the mobile body used as the detection part and the shooting direction of the detection image can be determined according to the time and place when the mobile body moves by the unmanned driving. (3) In the above-mentioned aspect, it is also possible to include: a plurality of cameras that capture the moving object; and a second determination unit that determines, based on the environmental information, the camera that can capture the detection image in a manner that satisfies the condition related to at least one of the determined part and the capturing direction as a responsible camera for capturing the detection image, and the image acquisition unit acquires the detection image from the responsible camera. According to such an aspect, the camera responsible for capturing the detection image can be appropriately determined based on the environmental information. (4) In the above aspect, the first determining unit may determine at least the part to be used as the detection part, and the first determining unit may selectively determine the part to be either the left part or the right part of the moving body based on the environmental information. According to such an aspect, the left part or the right part of the moving body can be used as the detection part based on the environmental information. (5) In the above aspect, the first determination unit may alternatively determine the location to be the left rear corner or the right rear corner of the moving body based on the environmental information. According to this aspect, the left rear corner or the right rear corner of the moving body can be used as the detection location based on the environmental information. (6) In the above-mentioned method, a third determination unit may be provided, which determines the content of the estimation processing based on at least one of the determined part and the shooting direction, wherein the estimation processing is a processing for estimating the position of the mobile object using the detection part in the detection image, and the position of the mobile object is used in generating the control command for the unmanned driving. According to such a method, the content of the estimation processing can be determined based on the part of the mobile object determined as the detection part and the determined shooting direction. Therefore, the possibility of estimating the position of the mobile object with higher accuracy in the estimation processing increases. (7) In the above-mentioned aspect, the first determination unit may determine at least the portion to be used as the detection portion, the first determination unit may selectively determine the portion to be one of a first portion and a second portion located at positions symmetrical to each other in the width direction of the moving body based on the environmental information, the third determination unit may determine to perform the following processing in the estimation process when the second portion is used as the detection portion, namely, generating a reverse image in which the detection image is mirror-inverted, and detecting the second portion in the reverse image, and the third determination unit may determine to perform the following processing in the estimation process when the first portion is used as the detection portion, namely, detecting the first portion in the detection image that is not mirror-inverted. According to such an aspect, in the case of using the first portion as the detection portion and in the case of using the second portion as the detection portion, the first portion or the second portion can be detected by the same procedure except that the detection image is mirror-inverted. (8) In the above-mentioned aspect, the third determination unit may determine a program used in the estimation process based on at least one of the determined part and the shooting direction, thereby determining the content of the estimation process. According to such an aspect, by having the third determination unit determine the program, the content of the estimation process can be determined based on the part of the moving object determined as the detection part and the determined shooting direction. In addition to being the above-mentioned system, the present disclosure may also be implemented, for example, by a mobile body, a control device, a control method, a computer program for implementing the control method, and a non-temporary recording medium recording the computer program. In addition, for example, the above-mentioned system may also input a detection image into a pre-prepared machine learning model for unmanned driving of the mobile body, thereby detecting the detection part in the detection image. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a conceptual diagram showing the configuration of the system. Figure 2 It is a block diagram showing the configuration of a vehicle and a control device. Figure 3A This is a flowchart showing a process procedure of the travel control of the vehicle in the first embodiment. Figure 3B This is a flowchart of the decision process. Figure 4 This is the first diagram for explaining the determination process in the first embodiment. Figure 5 This is a second diagram for explaining the determination process in the first embodiment. Figure 6 This is the third diagram for explaining the determination process in the first embodiment. Figure 7 This is a flowchart of the estimation process. Figure 8 It is an explanatory diagram showing an example of a state of analyzing an image in the estimation process. Fig. 9 This is the first diagram for explaining the determination process in the second embodiment. Fig.10 This is a second diagram for explaining the determination process in the second embodiment. Fig.11 It is a block diagram showing the configuration of a system in the third embodiment. Fig.12 This is a flowchart showing a process procedure of driving control of a vehicle in the third embodiment. DETAILED DESCRIPTION

[0008] A. First Implementation Method: Figure 1 It is a conceptual diagram showing the configuration of the system 10 in the first embodiment. Figure 2 1 is a block diagram showing the configuration of a vehicle 100 and a control device 200. In the present embodiment, the system 10 is configured as a remote control system for driving the vehicle 100 as a mobile body by remote control. The system 10 includes: one or more vehicles 100, a control device 200 for performing remote control of the vehicle 100, a plurality of cameras 300 for capturing images Pi of the vehicle 100, and a process management device 400 for managing the manufacturing process of the vehicle 100.

[0009] In the present disclosure, "mobile body" refers to an object that can move, such as a vehicle, an electric vertical take-off and landing aircraft (so-called flying car). The vehicle can be a vehicle that runs on wheels or a vehicle that runs on tracks, such as a car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. Vehicles include electric vehicles (BEV: Battery Electric Vehicle), gasoline vehicles, hybrid vehicles, and fuel cell vehicles. In the case where the mobile body is other than a vehicle, the expressions "vehicle" and "car" in the present disclosure can be appropriately replaced with "mobile body", and the expression "running" can be appropriately replaced with "moving".

[0010] The vehicle 100 is configured to be able to travel by unmanned driving. "Unmanned driving" refers to driving that does not rely on driving operations by passengers. Driving operations refer to operations related to at least one of "driving", "turning" and "stopping" of the vehicle 100. Unmanned driving is achieved by automatic or manual remote control using a device located outside the vehicle 100, or autonomous control of the vehicle 100. In the vehicle 100 that travels by unmanned driving, passengers who do not perform driving operations can also ride. Passengers who do not perform driving operations include, for example, people who only sit on the seats of the vehicle 100, and people who perform operations different from driving operations such as assembly, inspection, and operation of switch groups while riding in the vehicle 100. In addition, driving performed by driving operations by passengers is sometimes referred to as "manned driving".

[0011] In this specification, “remote control” includes “full remote control” in which all actions of the vehicle 100 are determined completely from outside the vehicle 100, and “partial remote control” in which part of the actions of the vehicle 100 are determined from outside the vehicle 100. In addition, “autonomous control” includes “full autonomous control” in which the vehicle 100 autonomously controls its own actions without receiving any information from a device outside the vehicle 100, and “partial autonomous control” in which the vehicle 100 autonomously controls its own actions using information received from a device outside the vehicle 100.

[0012] The vehicle 100 in the present embodiment is an electric vehicle (BEV: Battery Electric Vehicle). The vehicle 100 is not limited to an electric vehicle, and may be, for example, a gasoline vehicle, a hybrid vehicle, or a fuel cell vehicle.

[0013] In the present embodiment, remote control of the vehicle 100 is performed in a factory where the vehicle 100 is manufactured. Figure 1 As shown, the factory in this embodiment includes a first location PL1 and a second location PL2. The first location PL1 is, for example, a location where the vehicle 100 is assembled, and the second location PL2 is, for example, a location where the vehicle 100 is inspected. Any position in the factory is represented by an xyz coordinate value of a reference coordinate system RC. The reference coordinate system RC is, for example, defined as a global coordinate system. The first location PL1 and the second location PL2 are connected by a travel road SR on which the vehicle 100 can travel. The travel road SR may not be Figure 1The series of driving roads shown in the figure may be, for example, a driving road that branches into a plurality of driving roads in the middle, or a driving road where a plurality of driving roads merge in the middle. In this case, the driving road that the vehicle 100 is to travel may be determined based on the type of the vehicle 100 (e.g., vehicle type, model, grade, power source), the inspection result of the vehicle 100, the congestion level of each driving road, the congestion level of the next process, etc. In determining such a driving road, for example, the manufacturing information described later may be used.

[0014] In addition, the first place PL1 and the second place PL2 may be set in the same building or in different buildings in the same land. In addition, the first place PL1 and the second place PL2 may be set outdoors instead of indoors. In addition, the first place PL1 and the second place PL2 may be dispersedly set in multiple lands, for example, dispersedly set in a first factory and a second factory adjacent to each other across a public road or a private road. In this case, the first factory and the second factory may be collectively referred to as factories, and a portion of the public road or a portion of the private road may be included in the driving road SR.

[0015] A plurality of cameras 300 configured to be able to photograph the vehicle 100 are provided around the travel road SR. The control device 200 can use the captured images Pi captured by each camera 300 to obtain the relative position and orientation of the vehicle 100 with respect to the target route TR and the travel direction of the vehicle 100 in real time. The position, orientation, and travel direction thus detected are used in the generation of control instructions for remotely controlling the vehicle 100. In the present embodiment, each camera 300 is configured to be able to capture an image of the travel road SR viewed from above. In addition, the position of each camera 300 is fixed, and the relative relationship between the reference coordinate system RC and the device coordinate system of each camera 300 (hereinafter also referred to as the camera coordinate system) is known. The coordinate conversion matrix for converting the coordinate values ​​of the reference coordinate system RC and the coordinate values ​​of the device coordinate system of each camera 300 is pre-stored in the control device 200. In addition, in the remote control of the vehicle 100, various sensors such as various vehicle-mounted cameras, LiDAR (Light Detection And Ranging), millimeter wave radars, ultrasonic sensors, infrared sensors, etc. installed in the vehicle 100 can also be used auxiliaryly.

[0016] The control device 200 in this embodiment is configured as a remote control device that generates the above-mentioned control command and sends it to the vehicle 100. More specifically, the control device 200 generates a control command for causing the vehicle 100 to travel along the target route TR, and sends the control command to the vehicle 100. The vehicle 100 travels according to the received control command. Therefore, the system 10 can move the vehicle 100 from the first place PL1 to the second place PL2 by remote control without using a transport device such as a crane or a conveyor. Hereinafter, the control device 200 is also referred to as "server 200".

[0017] like Figure 2 As shown, the vehicle 100 includes: a vehicle control device 110 for controlling various parts of the vehicle 100, an actuator group 120 driven under the control of the vehicle control device 110, a communication device 130 for communicating with the control device 200 through wireless communication, and a GNSS (Global Navigation Satellite System) receiver 140 for obtaining position information of the vehicle 100. In this embodiment, the actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the direction of travel of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes a battery, a driving motor driven by the power of the battery, and a driving wheel rotated by the driving motor. The actuator of the driving device includes a driving motor. In addition, the actuator group 120 may also include an actuator for swinging the wiper of the vehicle 100, an actuator for opening and closing the electric window of the vehicle 100, etc.

[0018] The vehicle control device 110 is composed of a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected to each other in a bidirectional manner via the internal bus 114. The input / output interface 113 is connected to an actuator group 120, a communication device 130, and a GNSS receiver 140.

[0019] In the present embodiment, the processor 111 functions as a vehicle control unit 115 and a position information acquisition unit 116 by executing a program PG1 pre-stored in the memory 112. The vehicle control unit 115 controls the actuator group 120. When the driver is riding in the vehicle 100, the vehicle control unit 115 controls the actuator group 120 according to the operation of the driver, thereby enabling the vehicle 100 to travel. Regardless of whether the driver is riding in the vehicle 100, the vehicle control unit 115 controls the actuator group 120 according to the control command sent from the control device 200, thereby enabling the vehicle 100 to travel. The position information acquisition unit 116 acquires position information indicating the current position of the vehicle 100 using the GNSS receiver 140. However, the position information acquisition unit 116 and the GNSS receiver 140 may be omitted.

[0020] The control device 200 is composed of a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected to each other in a bidirectional manner via the internal bus 204. The input / output interface 203 is connected to a communication device 205 for communicating with the vehicle 100, the camera 300, and the process management device 400 by wireless communication.

[0021] In the present embodiment, the processor 201 functions as the image acquisition unit 210, the environment information acquisition unit 220, the first determination unit 230, the second determination unit 240, the third determination unit 245, the estimation unit 250, and the instruction generation unit 260 by executing the program PG2 pre-stored in the memory 202. In addition to the program PG2, the memory 202 also stores condition data 271, camera data 272, and an estimation program 280 described later. The estimation program 280 includes an analysis program described later.

[0022] The image acquisition unit 210 acquires the captured image Pi from the camera 300. The captured image Pi including the predetermined detection part in the captured image Pi is also referred to as a detection image. As the detection part, a part of the vehicle 100 is used. The detection part in the detection image is used for unmanned driving. More specifically, the detection part in the detection image is used to estimate the position of the vehicle 100, and the estimated position of the vehicle 100 is used in the generation of a control command for unmanned driving.

[0023] The environmental information acquisition unit 220 acquires environmental information indicating an environment in which the vehicle 100 is traveling by unmanned driving (hereinafter also referred to as a driving environment). The driving environment indicated by the environmental information may affect the detection image obtained by the image acquisition unit 210. The environmental information in this embodiment includes driving location information indicating a place in which the vehicle 100 is traveling by unmanned driving (hereinafter also referred to as a driving location). The driving location of the vehicle 100 affects the relative position and angle between the vehicle 100 and each camera 300, and therefore may affect the detection image. The driving location information in this embodiment is the current position information of the vehicle 100 on the driving road SR. In addition, the "current position information of the vehicle 100" only needs to be information that at least roughly indicates the current location of the vehicle 100. The control device 200 and the process management device 400 use, for example, the GNSS receiver 140, an area sensor (not shown) provided around the driving road SR, and various sensors (such as the camera 300) used for the above-mentioned unmanned driving to acquire such current position information. Furthermore, using the current position information of the vehicle 100 and the target route TR, it is possible to identify a location where the vehicle 100 is scheduled to travel in the future, for example, a section where the vehicle 100 is scheduled to travel in the future.

[0024] The first determination unit 230 determines at least one of the detection portion, i.e., the detection portion, and the shooting direction of the detection image, based on the environmental information acquired by the environmental information acquisition unit 220. In the present embodiment, the first determination unit 230 determines the detection portion. Hereinafter, the condition related to at least one of the detection portion and the shooting direction determined by the first determination unit 230 is also referred to as an image condition.

[0025] In the present embodiment, the first determination unit 230 determines the detection portion based on the environmental information by referring to the condition data 271. The condition data 271 in the present embodiment is data recording the correspondence between the driving environment and the portion of the vehicle 100 that can be used as the detection portion. More specifically, the condition data 271 in the present embodiment is data recording the portion of the vehicle 100 that can be used as the detection portion in each section included in the driving road SR.

[0026] The second decision unit 240 determines the camera 300 that is responsible for capturing the detection image, i.e., the responsible camera, based on the environmental information. The responsible camera determined in this way is a camera 300 that can capture the vehicle 100 according to the determined image conditions. In this embodiment, the responsible camera is a camera 300 that can capture the captured image Pi including the detection part.

[0027] In the present embodiment, the second decision unit 240 refers to the camera data 272 based on the environmental information to determine the responsible camera. The camera data 272 is data that records the correspondence between the driving environment and each camera 300. More specifically, the camera data 272 in the present embodiment is data that records the identification information of each camera 300 responsible for capturing the detection image in each section included in the driving road SR. The camera data 272 is defined to determine the camera 300 that can capture the detection image that meets the determined image condition in any section included in the driving road SR as the responsible camera.

[0028] The estimation unit 250 performs estimation processing. The estimation processing is a process of estimating the position of the vehicle 100 based on the unmanned control object using the detection part in the detection image. The estimation processing in the present embodiment includes an analysis process and a process of estimating the position of the vehicle 100 using the result of the analysis process. The analysis process is a process of detecting the detection part in the detection image by analyzing the detection image. In the present embodiment, the estimation process is performed by the estimation unit 250 executing the estimation program 280. In addition, the estimation unit 250 can estimate the position and direction of the vehicle 100 using, for example, the driving history of the vehicle 100 and the position information detected by the GNSS receiver 140 mounted on the vehicle 100. In addition, the driving direction of the vehicle 100 can be estimated using the direction of the vehicle 100, the driving history, the history of control instructions, etc.

[0029] The estimation unit 250 in this embodiment includes an analysis unit 251 that performs the above-mentioned analysis processing. The analysis unit 251 in this embodiment performs the analysis processing by executing the first analysis program 281 and the second analysis program 282 as analysis programs respectively. The analysis program refers to a program that enables a computer that executes the analysis program to analyze the detection image and detect the detection part in the detection image.

[0030] The third decision unit 245 determines the content of the estimation process according to the image condition, that is, at least one of the determined detection part and the shooting direction. The third decision unit 245 in this embodiment determines the program used in the estimation process according to the image condition, thereby determining the content of the estimation process according to the image condition. In more detail, the third decision unit 245 determines the detection program according to the image condition. The detection program refers to an analysis program used to detect the detection part in the detection image in the analysis process. In addition, in this specification, "determination of the program corresponding to the image condition" refers to the processing step of determining the program according to the image condition. Therefore, the meaning of "determination of the program corresponding to the image condition" includes, for example, selecting an analysis program as a determined object program according to the image condition from a plurality of programs prepared in advance. In this case, the selection of the detection program can be realized, for example, by executing a conditional branch corresponding to the image condition in a comprehensive program including a plurality of programs. In addition, the meaning of "determination of the program corresponding to the image condition" includes, for example, preparing the determined object program by rewriting part or all of the code included in the program according to the image condition.

[0031] The estimation program 280 is a comprehensive program including, as analysis programs, a first analysis program 281 and a second analysis program 282. The first analysis program 281 and the second analysis program 282 will be described in detail later.

[0032] The command generation unit 260 uses the position and orientation of the vehicle 100 estimated by the estimation unit 250 to generate a control command for unmanned driving and sends it to the vehicle 100. The control command is a command to make the vehicle 100 travel along the target route TR stored in the memory 202. In addition, the target path TR in this embodiment is equivalent to the reference path described later. In addition, in this embodiment, the control command includes the travel control signal described later. In other embodiments, the control command may include generation information for generating the travel control signal instead of including the travel control signal, or may include generation information for generating the travel control signal in addition to the travel control signal. As the generation information, for example, the vehicle position information, path, and target position described later can be used.

[0033] The process management device 400 is composed of, for example, a computer, and manages the entire manufacturing process of the vehicle 100 in the factory. For example, when a vehicle 100 starts traveling along the target route TR, individual information indicating an identification number, a model number, etc. for identifying the vehicle 100 is sent from the process management device 400 to the control device 200. The individual information is equivalent to the manufacturing information used in the management of the manufacturing process of the vehicle 100. In addition, the position of the vehicle 100 detected by the control device 200 is also sent to the process management device 400. In addition, the function of the process management device 400 may be installed on the same device as the control device 200.

[0034] Figure 3A 1 is a flowchart showing the processing steps of the driving control of the vehicle 100 in the first embodiment. In S1, the server 200 obtains the vehicle position information using the detection result output from the sensor located outside the vehicle 100, that is, the external sensor. The vehicle position information is the position information that becomes the basis for generating the driving control signal. In the present embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the reference coordinate system of the factory. In the present embodiment, the reference coordinate system of the factory is the global coordinate system, and any position in the factory can be represented by the X, Y, and Z coordinates in the global coordinate system. In the present embodiment, the external sensor is a camera installed in the factory, and the captured image is output from the external sensor as the detection result. That is, in S1, the server 200 obtains the vehicle position information using the captured image obtained from the camera as the external sensor.

[0035] In detail, in S1, the server 200 detects the shape of the vehicle 100 from the captured image, calculates the coordinates of the positioning point of the vehicle 100 in the coordinate system of the captured image, that is, the local coordinate system, and converts the calculated coordinates into coordinates in the global coordinate system, thereby obtaining the position of the vehicle 100. The shape of the vehicle 100 contained in the captured image can be detected by, for example, inputting the captured image into a detection model using artificial intelligence. The detection model is prepared, for example, in the system 10 or outside the system 10 and is pre-stored in the memory of the server 200. As the detection model, for example, a machine learning model that has been learned in a manner to achieve one of semantic segmentation and instance segmentation can be cited. As the machine learning model, for example, a convolutional neural network (hereinafter referred to as CNN) that has been learned by supervised learning using a learning data set can be used. The learning data set, for example, has a plurality of training images containing the vehicle 100, and labels indicating whether each area in the training image is an area representing the vehicle 100 or an area other than the vehicle 100. When learning CNN, it is preferred to update the parameters of CNN by back propagation (error back propagation method) to reduce the error between the output result of the detection model and the label. In addition, the server 200 estimates the direction of the movement vector of the vehicle 100 calculated from the position change of the feature point of the vehicle 100 between the frames of the captured image, for example, using the optical flow method, so as to obtain the direction of the vehicle 100.

[0036] Specifically, in S1 of the present embodiment, the above-mentioned detection image is used to obtain the vehicle position information. In addition, the vehicle position information is obtained by executing the estimation process. The camera responsible for taking the detection image and the content of the estimation process are determined by the determination process described later.

[0037] In S2, the server 200 determines the target position to which the vehicle 100 should go next. In the present embodiment, the target position is represented by the coordinates of X, Y, and Z in the global coordinate system. In the memory of the server 200, a reference path is pre-stored as the path that the vehicle 100 should travel. The path is represented by a node showing a departure point, a node showing a passing point, a node showing a destination, and a link connecting each node. The server 200 uses the vehicle position information and the reference path to determine the target position to which the vehicle 100 should go next. The server 200 determines the target position on the reference path that is ahead of the current position of the vehicle 100.

[0038] In S3, the server 200 generates a driving control signal for driving the vehicle 100 toward the determined target position. In the present embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. The server 200 calculates the driving speed of the vehicle 100 according to the change in the position of the vehicle 100, and compares the calculated driving speed with the target speed. In general, the server 200 determines the acceleration in a manner to accelerate the vehicle 100 when the driving speed is lower than the target speed, and determines the acceleration in a manner to decelerate the vehicle 100 when the driving speed is higher than the target speed. In addition, when the vehicle 100 is located on the reference path, the server 200 determines the steering angle and acceleration in a manner to prevent the vehicle 100 from deviating from the reference path. When the vehicle 100 is not located on the reference path, in other words, when the vehicle 100 deviates from the reference path, the server 200 determines the steering angle and acceleration in a manner to return the vehicle 100 to the reference path. In other embodiments, the travel control signal may include the speed of the vehicle 100 as a parameter instead of the acceleration of the vehicle 100 , or may include the speed of the vehicle 100 as a parameter in addition to the acceleration of the vehicle 100 .

[0039] In S4, the server 200 transmits the generated travel control signal to the vehicle 100. The server 200 repeatedly acquires vehicle position information, determines a target position, generates a travel control signal, transmits the travel control signal, and the like in a predetermined cycle.

[0040] In S5, the vehicle 100 receives the travel control signal transmitted from the server 200. In S6, the vehicle 100 controls the actuator of the vehicle 100 by using the received travel control signal, thereby causing the vehicle 100 to travel at the acceleration and steering angle indicated by the travel control signal. The vehicle 100 repeatedly receives the travel control signal and controls the actuator of the vehicle 100 in a predetermined cycle. According to the system 10 in this embodiment, the vehicle 100 can be driven by remote control, so that the vehicle 100 can be moved without using a transport device such as a crane or a conveyor.

[0041] Figure 3B 2 is a flowchart of a determination process in the present embodiment. The determination process is a process for realizing the control method in the present embodiment. The determination process is executed by the processor 201 of the control device 200 at predetermined time intervals, for example.

[0042] exist Figure 3BIn S110, the environmental information acquisition unit 220 acquires environmental information of the vehicle 100 which is a control object based on unmanned driving. In S120, the first determination unit 230 determines the image condition based on the environmental information obtained in step S110. As described above, in S120 of the present embodiment, the detection part is determined. In S130, the second determination unit 240 determines the responsible camera based on the environmental information obtained in step S110. In S140, the third determination unit 245 determines the detection program based on the image condition determined in S120. In S140 of the present embodiment, by determining the detection program in this way, the content of the inference processing is determined according to the image condition. More specifically, in S140, the content of the analysis processing is determined.

[0043] Figure 4 This is the first diagram for explaining the decision process. Figure 5 This is a second diagram for explaining the determination process. Figure 6 This is the third diagram for explaining the decision process. Figures 4 to 6 1 and 2 show examples of a situation in which the vehicle 100 travels along the target route TR on the travel road SR by remote control. Figure 4 The diagram shows a situation where the vehicle 100A is traveling in the section sc1 at time t1. Figure 5 The diagram shows a situation where the vehicle 100A is traveling in the section sc2 at time t2 after the time t1. Figure 6 The diagram shows a situation where the vehicle 100A is traveling in the section sc3 at time t3 after the time t2.

[0044] exist Figures 4 to 6 In the example of , as the detection part, either the first part Dp1 or the second part Dp2 is used. Figures 4 to 6 , a portion used as a detection portion on one side is indicated by a shaded mark, and a portion not used as a detection portion on the other side is indicated by a white mark. In the present embodiment, the first portion Dp1 is a left portion of the vehicle 100 located to the left of the central position in the vehicle width direction, and the second portion Dp2 is a right portion of the vehicle 100 located to the right of the central position. More specifically, the first portion Dp1 is the left rear corner of the vehicle 100, and the second portion Dp2 is the right rear corner of the vehicle 100. That is, the first portion Dp1 and the second portion Dp2 are portions located at positions that are substantially symmetrical to each other in the vehicle width direction of the vehicle 100. In the above-mentioned conditional data 271, the interval sc1 is associated with the first portion Dp1, the interval sc2 is associated with the second portion Dp2, and the interval sc3 is associated with the first portion Dp1. In addition, Figure 2The first analysis program 281 shown is used when the first portion Dp1 is used as the detection site. In addition, the second analysis program 282 is used when the second portion Dp2 is used as the detection site.

[0045] In addition, in the present embodiment, in the camera data 272, the section sc1 is associated with the camera 300A, the section sc2 is associated with the camera 300A and the camera 300B, and the section sc3 is associated with the camera 300B. The camera 300A can capture the first portion Dp1 in the section sc1, and can capture the second portion Dp2 in the section sc2. The camera 300B can capture the second portion Dp2 in the section sc2, and can capture the first portion Dp1 in the section sc3. In addition, in the case where a plurality of cameras 300 are associated with a certain section as in the section sc2, in the section, for example, it is preferable to determine the camera 300 that can capture the detection portion more appropriately as the responsible camera. In this case, for example, the camera 300 that is closer to the vehicle 100 can be determined as the responsible camera, or the responsible camera can be determined in a manner that can further suppress obstacles other than the vehicle 100 from being reflected in the detection image. Furthermore, for example, when a plurality of cameras 300 can capture detection images in accordance with the decision of the first decision unit 230 in a certain section, only a camera 300 that can capture an image of the detection area more appropriately may be associated with the section.

[0046] For example, in Figure 4 In the example, Figure 3B In S110, the driving location information indicating that the current position of the vehicle 100A is the section sc1 is obtained as the environmental information of the vehicle 100A. Next, in S120, based on the driving location information of the vehicle 100A obtained in S110, the condition data 271 is referred to, and the detection part is determined to be the first part Dp1 corresponding to the section sc1. Then, in S130, based on the environmental information obtained in S110, the camera data 272 is referred to, and the camera 300A that can capture the first part Dp1 is determined to be the responsible camera corresponding to the section sc1. In addition, in S140, the first analysis program 281 is determined as the detection program corresponding to the first part Dp1.

[0047] In addition, Figure 5 In the example, Figure 3BIn S110, the driving location information indicating that the driving location of the vehicle 100A is the interval sc2 is obtained. In S120, corresponding to the interval sc2, it is determined that the second part Dp2 is used as the detection part. In S130, corresponding to the interval sc2, the responsible camera is determined. In this case, as described above, for example, the camera 300A and the camera 300B that can capture the captured image Pi including the second part Dp2 and is closer to the vehicle 100 is determined as the responsible camera. In S140, corresponding to the second part Dp2, the second analysis program 282 is determined as the detection program. In addition, regarding Figure 6 Examples, also with Figure 4 , Figure 5 About the same.

[0048] In addition, when the first part Dp1 is used as the detection part in the section sc2 as well, unlike the present embodiment, since the vehicle 100 turns right between the sections sc1 and sc2, it may be necessary to arrange a camera that can capture the first part Dp1 of the vehicle 100 traveling in the section sc2 around the travel road SR. In addition, for example, depending on the arrangement of the ceiling, beams, walls, etc. around the travel road SR, it may be difficult to arrange the camera so that the first part Dp1 of the vehicle 100 traveling in the section sc2 can be appropriately captured. In the present embodiment, the number of cameras used in the remote control can be reduced, and the influence of the restriction of the installation position of the camera on the remote control can be suppressed.

[0049] Figure 7 : is a flowchart of the estimation process in this embodiment. In this embodiment, the estimation process is performed every time a detection image is sent from the responsible camera to the control device 200. In addition, Figure 7 S210 to S235 Figure 3A is executed in S1. Figure 7 The S240 Figure 3A Executed in S2 and S3.

[0050] exist Figure 7 In S205, the image acquisition unit 210 acquires the detection image sent from the responsible camera. Figure 7 In S210, the estimation unit 250 determines Figure 3BBased on the result of S140, it is determined whether the first analysis program 281 is used in the analysis of the detection image. If it is determined in S210 that the first analysis program 281 is used, the analysis unit 251 performs the first processing implemented by the first analysis program 281 in S220 to S230. If it is not determined in S210 that the first analysis program 281 is used, the analysis unit 251 performs the second processing implemented by the second analysis program 282 in S215 and S220 to S230. In addition, hereinafter, S220 implemented by the first analysis program 281 is also referred to as S220A, and S220 implemented by the second analysis program 282 is also referred to as S220B. The same is true for S225 and S230.

[0051] The second process is different from the first process and includes the inversion process of S215. The inversion process refers to the process of generating an inverted image after the detection image is mirror-inverted. Therefore, compared with the analysis of the detection image that is not mirror-inverted in S220A to S230A, the inverted image is analyzed in S220B to S230B. Such a process of mirror-inverting an image is also called "flipping".

[0052] The first process and the second process are substantially the same except that the second process includes an inversion process. That is, in S220B to S230B, the same algorithm as in S220A to S230A can be used, i.e., the inverted image can be parsed in the same steps. In more detail, in the case where the first part Dp1 is used as the detection part, in S220A to S230A, the first part Dp1 is detected as the left rear corner in the non-inverted detection image. On the other hand, in the case where the second part Dp2 is used as the detection part, the second part Dp2, which is actually the right rear corner, is simulatedly detected as the left rear corner in the inverted image in S220B to S230B. That is, in the present embodiment, in the inverted image, the second part Dp2 is detected as a part equivalent to the first part Dp1 in the detection image that is not mirror-inverted.

[0053] Figure 8 1 is an explanatory diagram showing an example of a state in which the image Im1 is analyzed during the estimation process. Figure 8The image Im1 shown is an image obtained by photographing the vehicle 100 traveling on the travel road SR along the traveling direction. The image Im1 is represented by a camera coordinate system having the focus of the camera 300 as the origin and having an Xc axis and a Yc axis orthogonal to the Xc axis as coordinate axes. The image Im1 may be a reversed image or a detection image that is not mirror-inverted. For the image Im1, various corrections and preprocessing such as distortion correction processing for correcting the distortion of the image Im1, rotation processing for rotating the image Im1 so that the direction of the movement vector V of the vehicle 100 contained in the image Im1 is oriented in a predetermined direction, and trimming processing for deleting unnecessary areas that do not contain the vehicle 100 from the image Im1 can be appropriately performed. In the detection of the size and direction of the movement vector V of the vehicle 100 contained in the captured image Pi, for example, an optical flow method is used.

[0054] exist Figure 7 In S220 to S230 of FIG. 1 , the analyzing unit 251 performs the analyzing process. In the analyzing process of this embodiment, the analyzing unit 251 uses the image Im1 to calculate Figure 8 The coordinates of the positioning point P3 shown are calculated as the coordinates of the vicinity of the detection part in the image coordinate system described later. The positioning point P3 detected in this way represents the detection part in the detection image.

[0055] In S220, as a detection process for detecting the vehicle 100 in the image Im1, the analysis unit 251 performs mask processing. The mask processing is a process of detecting the vehicle 100 in the image Im1 and masking the object area containing the vehicle 100 in the image Im1, thereby generating an image Im2 equivalent to the image Im1 with the object area masked. Hereinafter, the area masked by the mask processing will also be referred to as the mask area Ms. In the mask processing, the estimation unit 250 generates the image Im2, for example, by inputting the image Im1 into a machine learning model (not shown) that has been machine-learned in advance in a manner that masks the vehicle 100 contained in the input image. As the machine learning model, for example, a deep neural network (hereinafter also referred to as DNN) having a structure of a convolutional neural network (hereinafter also referred to as CNN) that realizes semantic segmentation and instance segmentation is used. In addition, the machine learning model is not limited to the above, and for example, it can also be a machine learning model that has been learned using an algorithm other than a neural network.

[0056] In S225, the analyzing unit 251 performs perspective transformation processing. The perspective transformation processing is a process of generating an image Im3 equivalent to the perspective transformed image Im2 by performing perspective transformation on the image Im2. In the perspective transformation processing, the analyzing unit 251 uses, for example, perspective transformation parameters related to the position information and internal parameters of the camera 300 to perspectively transform the image Im2 into a bird's-eye view image observed from above the vehicle 100 (for example, directly above the vehicle 100) which is approximately perpendicular to the road surface of the driving road SR. The image Im3 generated as a bird's-eye view image in this way is represented by an image coordinate system different from the camera coordinate system. The image coordinate system is a coordinate system with a point on the image plane projected by perspective transformation as the origin and having the Xi axis and the Yi axis orthogonal to the Xi axis as coordinate axes. The image Im3 includes a mask area Msb equivalent to the mask area Ms deformed by perspective transformation.

[0057] In S230, the analysis unit 251 performs a positioning point calculation process to calculate the coordinates of the positioning point P3. In the positioning point calculation process in this embodiment, the analysis unit 251 calculates the coordinates of the positioning point P3 using the coordinates (Xi1, Yi1) of the first coordinate point P1 and the coordinates (Xi2, Yi2) of the second coordinate point P2.

[0058] The first coordinate point P1 is determined as a coordinate point among the vertices of the quadrilateral R1b in the image Im3 that corresponds to the basic coordinate point P0 in the image Im2. The quadrilateral R1b is equivalent to the first circumscribed rectangle R1 deformed by the perspective transformation. The first circumscribed rectangle R1 is a rectangle that circumscribes the mask area Ms in the image Im2 and has a long side parallel to the movement vector V of the vehicle 100. The basic coordinate point P0 is a coordinate point that represents a vertex among the vertices of the first circumscribed rectangle R1 that corresponds to the detection part of the vehicle 100. In more detail, the basic coordinate point P0 is a coordinate point that represents a vertex among the vertices of the first circumscribed rectangle R1 that is located to the left and rear relative to the center of gravity C of the vehicle 100 when the direction of the movement vector V is regarded as the front.

[0059] The second coordinate point P2 is a coordinate point of a vertex corresponding to the detection part of the vehicle 100 among the four vertices of the second circumscribed rectangle R2 in the image Im3. In more detail, the second coordinate point P2 is a coordinate point of a vertex located to the left and rear relative to the center of gravity C of the vehicle 100 when the direction of the movement vector V is regarded as the front among the vertices of the second circumscribed rectangle R2. The second circumscribed rectangle R2 is a rectangle circumscribed to the mask area Msb and having a side parallel to the Xi axis and a side parallel to the Yi axis. In this way, since the first coordinate point P1 and the second coordinate point P2 are both coordinate points determined according to the detection part, they have a correlation relationship with each other.

[0060] When the coordinate value Xi1 of the first coordinate point P1 is greater than the coordinate value Xi2 of the second coordinate point P2, the estimation unit 250 determines the Xi coordinate of the positioning point P3 to be the coordinate value Xi2. On the contrary, when the coordinate value Xi1 is less than the coordinate value Xi2, the estimation unit 250 determines the Xi coordinate of the positioning point P3 to be the coordinate value Xi1. In addition, when the coordinate value Yi1 of the first coordinate point P1 is greater than the coordinate value Yi2 of the second coordinate point P2, the estimation unit 250 determines the Yi coordinate of the positioning point P3 to be the coordinate value Yi1. On the contrary, when the coordinate value Yi1 is less than the coordinate value Yi2, the estimation unit 250 determines the Yi coordinate of the positioning point P3 to be the coordinate value Yi2. In this way, by determining the Xi coordinate and Yi coordinate of the positioning point P3, the positioning point P3 is detected.

[0061] In S235, the estimation unit 250 performs a vehicle coordinate calculation process. The vehicle coordinate calculation process is a process of calculating a vehicle coordinate point using the positioning point P3. The vehicle coordinate point is a coordinate point representing the positioning point P3 in the reference coordinate system RC. In S235, the estimation unit 250 calculates the vehicle coordinate point using, for example, a predetermined estimation formula and the coordinate value of the positioning point P3 calculated in S230. The vehicle coordinate point calculated in this way is equivalent to the "position" contained in the above-mentioned vehicle position information. In addition, the method of estimating the position of the vehicle 100 using the detection part is not limited to the above-mentioned method. For example, the vehicle 100 in the detection image can be detected by a process different from the mask process, the detection part in the detection image can be detected by a process different from the perspective transformation process or the positioning point calculation process, and the vehicle coordinate point can be calculated by a process different from the vehicle coordinate calculation process.

[0062] In S240 , the command generation unit 260 generates a control command using the vehicle coordinate point calculated in the estimation process. The control command generated in this way is transmitted to the vehicle 100 .

[0063] According to the system 10 in the present embodiment described above, the first decision unit 230 determines at least one of the detection portion and the shooting direction of the detection image based on the environmental information of the vehicle 100. In this way, the detection portion and the shooting direction of the detection image can be determined according to the driving environment of the vehicle 100. Therefore, the possibility of obtaining a detection image suitable for remote control of the vehicle 100 is increased.

[0064] In addition, in the present embodiment, the environmental information includes the traveling location information of the vehicle 100. Therefore, the detection portion and the shooting direction of the detection image can be determined according to the location where the vehicle 100 is traveling by remote control.

[0065] In the present embodiment, the second decision unit 240 decides, based on the environmental information, a camera 300 among the plurality of cameras 300 that can capture a detection image to satisfy the determined image condition as a responsible camera. Therefore, the responsible camera can be appropriately determined based on the environmental information.

[0066] In addition, in the present embodiment, the first decision unit 230 selectively determines the detection part to be the left side or the right side of the vehicle 100 based on the environmental information. In this way, the left side or the right side of the vehicle 100 can be used as the detection part according to the environmental information. Therefore, by setting the camera 300 on the left side or the right side of the driving road SR, it is possible to easily capture a detection image containing the detection part using the camera 300. In particular, in the present embodiment, the first decision unit 230 selectively determines the detection part to be the left rear corner or the right rear corner of the vehicle 100 based on the environmental information. Therefore, by using the camera 300 set on the left side or the right side of the driving road SR, the vehicle 100 is photographed from the rear side of the vehicle 100, so that the detection image containing the detection part can be easily captured. In addition, in this way, the detection part can be detected with high accuracy using the circumscribed rectangle in the analysis process.

[0067] In addition, in the present embodiment, the third determination unit 245 changes the content of the estimation process according to the determined image condition. In this way, the content of the estimation process can be changed according to the determined image condition. Therefore, the possibility of being able to estimate the position of the vehicle 100 with higher accuracy in the estimation process increases.

[0068] In addition, in the present embodiment, the third decision unit 245 determines to perform the second process in the estimation process when the second part Dp2 is used as the detection part, and determines to perform the first process in the estimation process when the first part Dp1 is used as the detection part. The first process is a process of detecting the first part Dp1 in the detection image that is not mirror-inverted. The second process is a process of detecting the second part Dp2 in the inverted image. Therefore, in the case of using the first part Dp1 as the detection part and in the case of using the second part Dp2 as the detection part, the first part Dp1 or the second part Dp2 can be detected by the same procedure except that the detection image is mirror-inverted.

[0069] In addition, in this embodiment, the third determination unit 245 determines the program used in the estimation process according to the determined image condition, thereby determining the content of the estimation process. Therefore, by having the third determination unit 245 determine the program, the content of the estimation process can be changed according to the determined image condition.

[0070] B. Second Implementation Method: Fig. 9This is the first diagram for explaining the determination process in the second embodiment. Fig.10 2 is a second diagram for explaining the determination process in the second embodiment. Figures 4 to 6 In much the same way, Fig. 9 and Fig.10 An example of a situation in which the vehicle 100 travels along the target route TR on the travel road SRb by remote control is shown. In this embodiment, unlike the first embodiment, the environmental information includes travel time information indicating the time when the vehicle 100 travels. In addition, the first decision unit 230 in this embodiment is different from the first embodiment, and it determines the shooting direction of the detection image. The parts of the configuration of the system 10 in the second embodiment that are not particularly described are the same as those in the first embodiment.

[0071] Fig. 9 The diagram shows a situation where the vehicle 100B is traveling on the travel road SRb at time t4. Fig.10 The figure shows a situation where the vehicle 100B is traveling on the travel road SRb at time t5 which is different from time t4. Time t4 is a time when sunlight may shine on the vehicle 100 traveling on the travel road SRb from the right side of the vehicle 100 on a sunny day. Therefore, when the vehicle 100 is photographed from the right side of the vehicle 100 at time t4, a suitable detection image may not be obtained due to the influence of sunlight shining on the right side of the face of the vehicle 100. Time t5 is a time when sunlight may shine on the vehicle 100 traveling on the travel road SRb from the left side of the vehicle 100 on a sunny day. Therefore, when the vehicle 100 is photographed from the left side of the vehicle 100 at time t5, a suitable detection image may not be obtained due to the influence of sunlight shining on the left side of the face of the vehicle 100. In this way, the driving time of the vehicle 100 may affect the detection image.

[0072] The driving time information in this embodiment is information indicating the current time. In other embodiments, the driving time information may also be information indicating the approximate time period during which the vehicle 100 is traveling by remote control, for example. The condition data 271 in this embodiment is data recording the correspondence between the time period and the shooting direction of the detection image. The camera data 272 in this embodiment is data recording the correspondence between the time period and each camera 300.

[0073] In the present embodiment, in the condition data 271, the time period including the time t4 but not including the time t5 is associated with the first shooting direction d1, and the time period including the time t5 but not including the time t4 is associated with the second shooting direction d2. The first shooting direction d1 is a shooting direction for shooting the vehicle 100 from the left rear side. The second shooting direction d2 is a shooting direction for shooting the vehicle 100 from the right rear side. In addition, in the present embodiment, in the camera data 272, the time period including the time t4 but not including the time t5 is associated with the camera 300C, and the time period including the time t5 but not including the time t4 is associated with the camera 300D. The camera 300C is configured to be able to shoot the vehicle 100 from the left rear side. The camera 300D is configured to be able to shoot the vehicle 100 from the right rear side. In addition, in the present embodiment, the first analysis program 281 is used when the shooting direction is the first shooting direction d1. In addition, the second analysis program 282 is used when the shooting direction is the second shooting direction d2.

[0074] exist Fig. 9 In the example, Figure 3B In S110, the driving time information indicating that the current time is time t4 is obtained as the environmental information of the vehicle 100B. In S120, based on the driving time information of the vehicle 100B obtained in S110, the condition data 271 is referred to, and the shooting direction is determined to be the first shooting direction d1 corresponding to time t4. In S130, based on the environmental information obtained in S110, the camera data 272 is referred to, and the camera 300C that can shoot the vehicle 100B from the first shooting direction d1 is determined to be the responsible camera corresponding to time t4. Fig.10 Examples, also with Fig. 9 About the same.

[0075] According to the second embodiment described above, the environmental information includes the driving time information of the vehicle 100. In this way, the detection part and the shooting direction of the detection image can be determined according to the time when the vehicle 100 is driven by remote control. Fig. 9 and Fig.10 In the example of FIG. 1 , at time t4 when sunlight may shine on the vehicle 100 from the right side of the vehicle 100, the vehicle 100 can be photographed from the left side to obtain a detection image. In addition, at time t5 when sunlight may shine on the vehicle 100 from the left side of the vehicle 100, the vehicle 100 can be photographed from the right side to obtain a detection image. Therefore, the influence of sunlight on the detection image can be suppressed. In this way, according to the present embodiment, the possibility of obtaining a detection image suitable for remote control of the vehicle 100 is also increased.

[0076] In other embodiments, the environmental information may include both the driving time information and the driving location information. In this case, the first decision unit 230 may also use the condition data 271 in determining the detection part and the shooting direction, and the condition data 271 records the correspondence between the driving time information and the driving location information and the part of the vehicle 100 that can be used as the detection part and the shooting direction. The condition data 271 in this case may also be data that records the correspondence between the section where the vehicle 100 travels and the detection part and the shooting direction in the section for each time period.

[0077] C. Third Implementation Method: Fig.11 1 is a block diagram showing the configuration of the system 10 in the third embodiment. The system 10 in this embodiment is different from the first embodiment and does not include the control device 200. In addition, the vehicle in this embodiment can travel by autonomous control of the vehicle. In addition, the device configuration of the vehicle in this embodiment is the same as that of the first embodiment, so for convenience, the vehicle in this embodiment is represented as a vehicle 100. The parts of the configuration of the system 10 and the vehicle 100 in the third embodiment that are not particularly described are the same as those in the first embodiment.

[0078] In the present embodiment, the communication device 130 of the vehicle 100 can communicate with the camera 300 and the process management device 400 as external sensors. The processor 111 of the vehicle control device 110 functions as a vehicle control unit 115v, a position information acquisition unit 116, an image acquisition unit 210, an environment information acquisition unit 220, a first determination unit 230, a second determination unit 240, a third determination unit 245, an estimation unit 250, and a command generation unit 260 by executing the program PG1 stored in the memory 112. The vehicle control unit 115v controls the actuator group 120 using the travel control signal generated by the vehicle 100, and can drive the vehicle 100 by autonomous control. In addition to the program PG1, the memory 112 also stores condition data 271, camera data 272, an estimation program 280, a target route TR, and a detection model (not shown). The vehicle control device 110 in the third embodiment is equivalent to the "control device" in the present disclosure.

[0079] Fig.121 is a flowchart showing the processing steps of the driving control of the vehicle in the third embodiment. In S901, the vehicle 100 uses the detection result output from the camera as an external sensor to obtain the vehicle position information. In S902, the vehicle 100 determines the target position to which the vehicle 100 should go next. In S903, the vehicle 100 generates a driving control signal for driving the vehicle 100 toward the determined target position. In S904, the vehicle 100 controls the actuator of the vehicle 100 by using the generated driving control signal, so that the vehicle 100 drives according to the parameters represented by the driving control signal. The vehicle 100 repeatedly obtains the vehicle position information, determines the target position, generates the driving control signal, and controls the actuator in a prescribed cycle. According to the driving control in this embodiment, even if the vehicle 100 is not remotely controlled by the server 200, the vehicle 100 can be driven by the autonomous control of the vehicle 100.

[0080] D. Other implementation methods: (D1) In each of the above-described embodiments, the first determination unit 230 may determine both the detection site and the imaging direction of the detection image.

[0081] (D2) In each of the above embodiments, the environmental information may include various information indicating the driving environment of the vehicle 100 instead of the driving time information and the driving place information, or may include various information indicating the driving environment of the vehicle 100 in addition to such information. For example, the environmental information may include weather information indicating the weather in the area including the driving road SR. In this case, the first decision unit 230 may use the condition data 271 recording the correspondence between the weather and the part of the vehicle 100 that can be used as the detection part and the shooting direction when determining the detection part and the shooting direction. In this case, the first decision unit 230 refers to the condition data 271 based on the weather information, so that, for example, the detection part, the shooting direction, and the responsible camera can be different in sunny days and rainy days. In addition, when the environmental information includes the driving place information and the weather information, for example, the detection part, the shooting direction, and the responsible camera can be determined in consideration of the ease of water accumulation in each section of the driving road SR in rainy days. In this way, for example, it is possible to suppress the reduction in the accuracy of the position estimation of the vehicle 100 due to the image of the vehicle 100 reflected by the water being included in the detection image. In addition, the environmental information may be information obtained by sensors mounted on the remotely controlled vehicle 100 or various sensors installed around the driving road SR, indicating the presence or absence of obstacles around the vehicle 100, and the ambient temperature and humidity of the place where the vehicle 100 is driving.

[0082] (D3) In each of the above embodiments, the responsible camera is determined from the plurality of cameras 300 based on the environmental information, but it may not be determined in this way. For example, it may be configured to be able to change the angle or position of the camera 300, and the angle or position of one camera 300 may be changed based on the environmental information so that the determined detection part can be photographed, or the vehicle 100 can be photographed in the determined shooting direction. In this case, the number of cameras 300 provided in the system 10 may be only one. In addition, in this case, the system 10 may not include the second determination unit 240.

[0083] (D4) In each of the above-mentioned embodiments, for example, the first portion Dp1 may be the right side of the vehicle 100, and the second portion Dp2 may be the left side. In addition, as the detection portion, a pair of portions located at positions symmetrical to each other in the vehicle width direction, such as the first portion Dp1 and the second portion Dp2, may not be used. For example, the left rear corner and the right front corner of the vehicle 100 may be used as the detection portion. In addition, instead of using the left side or the right side of the vehicle 100 as the detection portion, a portion of the vehicle 100 that is different from the left side and the right side may be used. In addition, as the detection portion, more than three portions may be used. In addition, in the case where the detection portion is not determined by the first determination unit 230, the number of portions used as the detection portion may be only one.

[0084] (D5) In each of the above-mentioned embodiments, the third determination unit 245 may not determine the program used in the estimation process based on the determined image conditions. In this case, for example, the program used in the estimation process may be determined based on environmental conditions. In addition, for example, the same program may be used in the estimation process even when the image conditions are different. In this case, the estimation unit 250 and the analysis unit 251 may change the contents of the estimation process and the analysis process by changing the parameters used in the estimation process and the analysis process based on the image conditions. In addition, the third determination unit 245 may not determine the contents of the estimation process based on the image conditions. In this case, the estimation unit 250 may, for example, input the detection image into the same machine learning model regardless of the image conditions, thereby estimating the position of the vehicle 100. In addition, in this case, the system 10 may not have the third determination unit 245.

[0085] (D6) In the above-mentioned embodiments, the third determination unit 245 determines to execute the second process in the estimation process when the second part Dp2 is used as the detection part. In contrast, the third determination unit 245 may not determine to execute the second process in the above-mentioned case. In this case, the analysis unit 251 may detect the second part Dp2 by actually detecting the second part Dp2 in the detection image that is not mirror-inverted, for example.

[0086] (D7) In each of the above embodiments, the vehicle 100 only needs to have a structure capable of moving by unmanned driving, for example, it may be a form having a stand having the structure described below. Specifically, in order for the vehicle 100 to perform the three functions of "driving", "turning" and "stopping" by unmanned driving, it is sufficient to have at least an actuator such as a control device, a drive device, a steering device and a braking device that controls the driving of the vehicle 100. In the case where the vehicle 100 obtains information from the outside for unmanned driving, the vehicle 100 may also have a communication device. That is, on the vehicle 100 capable of moving by unmanned driving, at least a part of the interior parts such as the driver's seat and the instrument panel may not be installed, at least a part of the exterior parts such as the bumper and the fender may not be installed, and the body shell may not be installed. In this case, the remaining parts such as the body shell may be installed on the vehicle 100 during the period until the vehicle 100 is shipped from the factory, or the remaining parts such as the body shell may be installed on the vehicle 100 after the vehicle 100 is shipped from the factory in a state where the remaining parts such as the body shell are not installed on the vehicle 100. Each component can be installed from any direction such as the upper side, lower side, front side, rear side, right side or left side of the vehicle 100, and can be installed from the same direction or from different directions. In addition, the position of the stand can also be determined in the same way as the vehicle 100 in the first embodiment.

[0087] (D8) In the above-mentioned embodiments, in the system 10, various functional units such as the image acquisition unit 210, the environment information acquisition unit 220, the first determination unit 230, the second determination unit 240, the third determination unit 245, the estimation unit 250, and the command generation unit 260 may be provided by the vehicle 100. In this case, as described in the third embodiment, the image acquisition unit 210, the environment information acquisition unit 220, the first determination unit 230, the second determination unit 240, the third determination unit 245, the estimation unit 250, and the command generation unit 260 may all be provided by the vehicle 100, or some of these functional units may be provided by the vehicle 100. In addition, in the system 10, some or all of these functional units may be provided by devices outside the server 200 and the vehicle 100.

[0088] (D9) In each of the above embodiments, the system 10 is configured as a system including the camera 300, the estimation unit 250, and the command generation unit 260, but it may not be configured in this way. For example, the system 10 may be configured as a system that determines at least one of the detection part and the shooting direction of the detection image based on the environmental information, and sends the determination result to another system including the camera 300, the estimation unit 250, and the command generation unit 260. In this case, the system 10 may not include part or all of the camera 300, the estimation unit 250, and the command generation unit 260, for example.

[0089] (D10) In the first embodiment, the server 200 performs the processing from the acquisition of the vehicle position information to the generation of the travel control signal. In contrast, at least a part of the processing from the acquisition of the vehicle position information to the generation of the travel control signal may be performed by the vehicle 100. For example, the following methods (1) to (3) may be used.

[0090] (1) The server 200 may obtain vehicle position information, determine a target position to which the vehicle 100 should go next, and generate a path from the current position of the vehicle 100 indicated in the obtained vehicle position information to the target position. The server 200 may generate a path to the target position between the current position and the destination, or may generate a path to the destination. The server 200 may transmit the generated path to the vehicle 100. The vehicle 100 may generate a travel control signal so that the vehicle 100 travels on the path received from the server 200, and control the actuator of the vehicle 100 using the generated travel control signal.

[0091] (2) The server 200 may also obtain vehicle position information and transmit the obtained vehicle position information to the vehicle 100. The vehicle 100 may also determine a target position to which the vehicle 100 should go next, generate a path from the current position of the vehicle 100 indicated by the received vehicle position information to the target position, generate a travel control signal so that the vehicle 100 travels on the generated path, and control the actuator of the vehicle 100 using the generated travel control signal.

[0092] (3) In the above-mentioned methods (1) and (2), the vehicle 100 may be equipped with an internal sensor, and the detection result outputted from the internal sensor may be used in at least one of the generation of the route and the generation of the driving control signal. The internal sensor is a sensor mounted on the vehicle 100. Specifically, the internal sensor may include, for example, a camera, LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyroscope sensor, etc. For example, in the above-mentioned method (1), the server 200 may also obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating the route. In the above-mentioned method (1), the vehicle 100 may also obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the driving control signal when generating the driving control signal. In the above-mentioned method (2), the vehicle 100 may also obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating the route. In the above-mentioned aspect (2), the vehicle 100 may obtain the detection result of the internal sensor and reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.

[0093] (D11) In the above embodiment in which the vehicle 100 can travel by autonomous control, the vehicle 100 may be equipped with an internal sensor, and a detection result outputted from the internal sensor may be used in at least one of the generation of the route and the generation of the travel control signal. For example, the vehicle 100 may obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating the route. The vehicle 100 may obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.

[0094] (D12) In the above-mentioned embodiment in which the vehicle 100 can travel by autonomous control, it is also possible that an internal sensor is mounted on the vehicle 100, the vehicle 100 obtains vehicle position information using the detection results of the internal sensor, determines the target position to which the vehicle 100 should go next, generates a path from the current position of the vehicle 100 represented by the obtained vehicle position information to the target position, generates a travel control signal for traveling on the generated path, and uses the generated travel control signal to control the actuator of the vehicle 100. In this case, the vehicle 100 can travel without using the detection results of the external sensor. In addition, the vehicle 100 can also obtain the target arrival time and / or congestion information from outside the vehicle 100, so that the target arrival time and / or congestion information is reflected in at least one of the path and the travel control signal.

[0095] (D13) In the first embodiment described above, the server 200 automatically generates a travel control signal to be sent to the vehicle 100. Alternatively, the server 200 may generate a travel control signal to be sent to the vehicle 100 in accordance with an operation of an external operator located outside the vehicle 100. For example, the external operator may operate an operating device including a display for displaying a captured image output from an external sensor, a steering wheel for remotely operating the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server 200 via wired communication or wireless communication, and the server 200 may generate a travel control signal corresponding to the operation applied to the operating device.

[0096] (D14) The vehicle 100 can be manufactured by combining a plurality of modules. A module refers to a unit composed of one or more components that are aggregated according to the structure and function of the vehicle 100. For example, the rack of the vehicle 100 can be manufactured by combining a front module that constitutes the front part of the rack, a central module that constitutes the central part of the rack, and a rear module that constitutes the rear part of the rack. In addition, the number of modules that constitute the rack is not limited to three, and may be less than two or more than four. In addition, in addition to the rack, a part of the vehicle 100 that is different from the rack may also be modularized, or a part of the vehicle 100 that is different from the rack may be modularized instead of the rack. In addition, various modules may also include any exterior parts such as bumpers and grilles, and any interior parts such as seats or consoles. In addition, not limited to the vehicle 100, any type of mobile body may also be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or fixing tools, or may be manufactured by integrally molding at least a portion of the module into one part by casting. The molding method of integrally molding at least a part of a module into one component is also called Giga-casting or Mega-casting. By using Giga-casting, various parts of a moving body that were previously formed by joining multiple parts can be formed into one component. For example, the front module, the center module, and the rear module mentioned above can also be manufactured using Giga-casting.

[0097] (D15) The transportation of the vehicle 100 by the driving of the unmanned vehicle 100 is also referred to as "self-propelled transportation". In addition, the structure for realizing self-propelled transportation is also referred to as "vehicle remote control autonomous driving transportation system". In addition, the production method of producing the vehicle 100 by self-propelled transportation is also referred to as "self-propelled production". In self-propelled production, for example, in a factory that manufactures the vehicle 100, at least a part of the transportation of the vehicle 100 is realized by self-propelled transportation.

[0098] In the above-mentioned embodiments, part or all of the functions and processes implemented in software may also be implemented in hardware. In addition, part or all of the functions and processes implemented in hardware may also be implemented in software. As hardware for implementing the various functions in the above-mentioned embodiments, various circuits such as integrated circuits or discrete circuits may also be used.

[0099] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various configurations within the scope of its main purpose. For example, the technical features in the embodiments corresponding to the technical features in each method described in the invention content section can be appropriately replaced or combined in order to solve part or all of the above-mentioned problems, or to achieve part or all of the above-mentioned effects. In addition, as long as the technical feature is not described as a necessary part in this specification, it can be appropriately deleted. [Explanation of Reference Numerals]

[0100] 10: System 100, 100A, 100B: Vehicle, 110: Vehicle control device, 111: Processor, 112: Memory, 113: Input / output interface, 114: Internal bus, 115, 115v: Vehicle control unit, 116: Position information acquisition unit, 120: Actuator group, 130: Communication device, 140: GNSS receiver, 200: Control device (server), 201: Processor, 202: Memory, 203: Input / output interface, 204: Internal bus , 205: communication device, 210: image acquisition unit, 220: environmental information acquisition unit, 230: first determination unit, 240: second determination unit, 245: third determination unit, 250: estimation unit, 251: analysis unit, 260: instruction generation unit, 271: condition data, 272: camera data, 280: estimation program, 281: first analysis program, 282: second analysis program, 300, 300A, 300B, 300C, 300D: camera, 400: process management device.

Claims

1. A system comprising: A mobile body that moves by unmanned driving using a predetermined detection portion included in a detection image in which the mobile body is captured; an image acquisition unit that acquires the detection image; an environment information acquisition unit that acquires environment information indicating an environment in which the moving object moves by the unmanned driving and which may affect the acquired detection image; and A first determination unit determines at least one of a part of the moving object used as the detection part and a shooting direction of the detection image based on the acquired environmental information.

2. The system according to claim 1, wherein: The environmental information includes at least one of information indicating a time when the mobile object moves by the unmanned driving and information indicating a place where the mobile object moves by the unmanned driving.

3. The system according to claim 1, wherein: have: a plurality of cameras for photographing the moving object; and a second determination unit, which determines, based on the environmental information, the camera among the cameras that can capture the detection image in a manner that satisfies the condition related to at least one of the determined part and the shooting direction as the camera responsible for capturing the detection image, The image acquisition unit acquires the detection image from the responsible camera.

4. The system according to claim 1, wherein: The first determination unit determines at least the site to be used as the detection site, The first determination unit selectively determines the portion to be either the left portion or the right portion of the moving object based on the environmental information.

5. The system according to claim 4, wherein: The first determination unit selectively determines the location to be either a left rear corner or a right rear corner of the moving object based on the environmental information.

6. The system according to any one of claims 1 to 5, wherein: A third determination unit is provided, which determines the content of the estimation processing based on the determined part and at least one of the shooting direction. The estimation processing is a processing of estimating the position of the moving object using the detection part in the detection image, and the position of the moving object is used in generating the control instruction for the unmanned driving.

7. The system according to claim 6, wherein: The first determination unit determines at least the site to be used as the detection site, The first determination unit selectively determines the portion to be one of a first portion and a second portion located at positions symmetrical to each other in a width direction of the moving body, based on the environmental information. The third determination unit determines to perform the following processing in the estimation processing, when the second part is used as the detection part: generating a reverse image obtained by mirror-reversing the detection image, and detecting the second part in the reverse image, The third determination unit determines, when the first portion is used as the detection site, to execute, in the estimation process, a process of detecting the first portion in the detection image that is not mirror-inverted.

8. The system according to claim 6, wherein: The third determination unit determines the content of the estimation process by determining a program used in the estimation process based on the at least one of the determined part and the imaging direction.

9. A control device comprising: an image acquisition unit that acquires a detection image in which a moving object is captured, wherein a predetermined detection portion included in the detection image is used for unmanned driving of the moving object; an environment information acquisition unit that acquires environment information indicating an environment in which the moving object moves by the unmanned driving and which may affect the acquired detection image; and A first determination unit determines at least one of a part of the moving object used as the detection part and a shooting direction of the detection image based on the acquired environmental information.

10. A control method, comprising: A step of acquiring a detection image in which a moving object is captured, wherein a predetermined detection portion included in the detection image is used for unmanned driving of the moving object; a step of acquiring environmental information indicating an environment in which the moving object moves by the unmanned driving and which may affect the acquired detection image; and A step of determining at least one of a portion of the moving object used as the detection portion and a shooting direction of the detection image based on the acquired environmental information.

Images