Mobile Control System

Abstract

To provide a mobile body control system capable of tracking a moving object by quickly detecting the object by a plurality of cameras.SOLUTION: Each camera edge terminal includes: a control unit; a camera unit; and a communication unit that communicates with other camera edge terminals and a mobile body. The control unit includes: object detection means for detecting an object in an imaging area to acquire information on a type, a position, and a velocity vector of the object; probability distribution calculation means for obtaining a probability distribution of an estimated position of the object from information on a position and a velocity vector of the object about the object moving to outside of the imaging area; information sharing means for sharing information on a type of the object and the probability distribution of the estimated position with other camera edge terminals; and tracking means for determining whether or not the moving object detected in the imaging area is the object from information on the type of the object and the probability distribution of the estimated position when there is possibility that the object is positioned in the imaging area with the probability distribution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a mobile object control system for controlling mobile objects such as vehicles and drones with autonomous driving functions. [Background technology]

[0002] In recent years, technological developments related to the autonomous driving of moving objects such as cars and drones have progressed and are beginning to be put into practical use. Generally, such moving objects are equipped with various cameras and sensors, which allow them to move autonomously while obtaining information about surrounding obstacles.

[0003] To further enhance the safety of the movement of a mobile body, there is a method of assisting the autonomous movement of the mobile body by obtaining information about the mobile body and obstacles around it using cameras and sensors located outside the mobile body and transmitting that information to the mobile body. For example, Patent Document 1 describes a system in which sensors are installed on the ceiling of the target area in which an autonomous vehicle travels. This system uses sensors to detect the positions of the autonomous vehicle and objects to be avoided, and based on this, generates a planned route that the vehicle can travel and notifies the vehicle of this. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-177328 Summary of the Invention [Problem to be solved by the invention]

[0005] In the system of Patent Document 1, the entire target area is within the detection range of the sensor. However, as the range of movement of a moving object becomes wider, it becomes difficult to install cameras and sensors to cover the entire area. In such cases, cameras and sensors are placed at intervals, resulting in ranges that can be detected by the cameras and sensors and ranges that cannot. Furthermore, even if the range of movement of a moving object is narrow, blind spots that block detection will occur in complex locations or places with many obstructions, so ranges that can be detected by the cameras and sensors and ranges that cannot be detected will still be generated. Therefore, when an object such as a moving object detected by a camera or the like moves out of the detection range of that camera or the like, it must be redetected by another camera or the like to track the object. However, if there is an erroneous detection or a delay in this redetection of the object, the risk of the moving object colliding with an obstacle increases.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a mobile object control system for controlling mobile objects such as vehicles and drones with autonomous driving functions, which has multiple cameras and can quickly detect and track moving objects using each camera. [Means for solving the problem]

[0007] The present invention is a mobile body control system that includes multiple camera edge terminals and transmits information about moving objects obtained by the camera edge terminals to a mobile body with an automatic driving function, wherein each camera edge terminal has a control unit that controls itself, a camera unit that photographs a predetermined area, and a communication unit that communicates with the other camera edge terminals and the mobile body, and the control unit has an object detection means that detects the object within the photographing area based on the photographing data from the camera unit and acquires information about the type, position, and velocity vector of the object, a probability distribution calculation means that, for the object moving outside the photographing area of ​​the camera unit, calculates a probability distribution of the estimated position of the object based on information about the position and velocity vector of the object, an information sharing means that shares information about the probability distribution of the type and estimated position of the object with the other camera edge terminals, and a tracking means that determines whether a moving object detected within the photographing area is the object based on information about the probability distribution of the object's type and estimated position when there is a possibility that the object is located within the photographing area of ​​the camera unit in the probability distribution. Note that the term "object" includes both moving objects and obstacles other than moving objects. The communication unit may communicate directly with other camera edge terminals or moving objects, or may communicate via other devices such as a server. Furthermore, while the control unit includes each means, this does not necessarily mean that the means of the control unit of one camera edge terminal function consecutively, but also means that the means of the control units of multiple camera edge terminals function consecutively across the board. For example, after the object detection means, probability distribution calculation means, and information sharing means of the control unit of one camera edge terminal function, the tracking means of the control unit of another camera edge terminal may function.

[0008] In addition, in the present invention, the probability distribution calculation means may include a first probability distribution calculation means that calculates a first probability distribution regarding the position in the shooting area of ​​the camera unit of any of the camera edge terminals in which the object will appear, and a second probability distribution calculation means that calculates a second probability distribution regarding the position in the shooting area of ​​the camera unit of the camera edge terminal that is determined to have the highest probability in the first probability distribution in which the object will appear.

[0009] In addition, in the present invention, the tracking means may include a match rate acquisition means for calculating a match rate between the object and the target based on information about the type of the object, a probability acquisition means for acquiring an object existence probability, which is a probability in the second probability distribution of the position where the object is detected, and a determination means for determining that the object is the target when the match rate and the object existence probability are each greater than a predetermined value. Note that the match rate between the object and the target may be calculated based on shape or color, or may be calculated based on a marker or the like that displays identification information on the object. Furthermore, the predetermined values ​​for the match rate and the object existence probability are not limited to a single set of predetermined values, but may also include multiple sets of predetermined values ​​or be expressed by one or more equations.

[0010] In addition, the present invention may be such that the control unit has a first probability distribution update means that updates the first probability distribution by Bayesian estimation based on the posterior probability of the object being present in a photographing area of ​​the camera unit of the camera edge terminal that is determined to have the highest probability in the first probability distribution when the object is not found in that photographing area.

[0011] In addition, the present invention may be such that the control unit includes a second probability distribution update means that updates the second probability distribution by Bayesian estimation based on the posterior probability of the object being present at a position that is determined to have the highest probability in the second probability distribution when the object is not found at that position. [Effects of the Invention]

[0012] According to the present invention, whether an object is a target object is determined based on the probability distribution of the estimated position of the target object, thereby improving the accuracy of target object detection and reducing false detections and delays. Furthermore, by transmitting information about the target object thus obtained to a mobile object, the mobile object can avoid collisions with obstacles with ample time to maneuver. Therefore, even when the mobile object has a wide range of movement, or when the range of movement of the mobile object is in a complicated location or a location with many obstructions, the mobile object can move more safely.

[0013] Furthermore, if the probability distribution calculation means calculates a first probability distribution of which camera edge terminal's camera unit's shooting area the object will appear in, and a second probability distribution of which position the object will appear in the shooting area of ​​the camera unit of the camera edge terminal that is determined to have the highest probability in the first probability distribution, it is possible to efficiently determine which camera edge terminal from among multiple camera edge terminals should attempt to detect the object, and to efficiently detect the object within the shooting area of ​​that camera edge terminal, thereby improving the accuracy of object detection.

[0014] Furthermore, if the tracking means determines that an object is a target object based on the matching rate between the object and the target and the target existence probability, the accuracy of target detection can be improved by using two indicators.

[0015] Furthermore, if the first probability distribution and the second probability distribution are updated by Bayesian estimation, the reliability of the probability distributions will be higher, and the accuracy of target object detection will be improved. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram of a mobile object control system according to the present invention. [Figure 2] FIG. 1 is a schematic diagram of a camera edge terminal. [Figure 3] 10 is a flowchart of a control program. [Figure 4] FIG. 10 is an explanatory diagram of a first probability distribution. [Figure 5] FIG. 10 is an explanatory diagram of a second probability distribution. DETAILED DESCRIPTION OF THE INVENTION

[0017] The following describes the specific details of the mobile object control system of the present invention. This mobile object control system is used to control various mobile objects, but as one embodiment, the case of controlling a vehicle (hereinafter simply referred to as a vehicle) with an automatic driving function will be described. The vehicle is assumed to travel on a road. Furthermore, although obstacles that may impede the vehicle's travel are assumed to include people, animals, automobiles, bicycles, etc., here, the obstacle is assumed to be a separate automobile. The vehicle is equipped with various cameras and sensors, which allow it to move autonomously (autonomously drive) while obtaining information about surrounding obstacles. The mobile object control system is equipped with a camera edge terminal, and assists the autonomous driving of the vehicle by transmitting information about moving objects obtained by the camera edge terminal to the vehicle. Objects include both the vehicle itself, which is a moving object, and other obstacles.

[0018] As shown in FIG. 1, this mobile object control system includes multiple camera edge terminals 1. Utility poles P are erected in a row at predetermined intervals along the side of a road R on which a vehicle C and an obstacle B travel, and the camera edge terminals 1 are attached to the tops of the utility poles P. In the illustrated example, one camera edge terminal 1 is attached to one utility pole P, but multiple camera edge terminals 1 may be attached to one utility pole P at different heights and orientations. Each camera edge terminal 1 has the same configuration.

[0019] As shown in FIG. 2, the camera edge terminal 1 has a control unit 2 that controls itself, a camera unit 3 that captures images of a predetermined area, and a communication unit 4 that communicates with other camera edge terminals 1 and a vehicle C. The camera unit 3, control unit 2, and communication unit 4 are attached to a utility pole P in this order from top to bottom, and the camera unit 3 and communication unit 4 are each connected to the control unit 2. The connection between the control unit 2 and the camera unit 3 and the connection between the control unit 2 and the communication unit 4 may be wired or wireless. Note that in the illustrated example, the control unit 2, camera unit 3, and communication unit 4 are each independent, but they may also be housed in a single housing.

[0020] The control unit 2 consists of hardware (computer) such as a PLC that executes a control program for controlling the camera edge terminal 1 (camera unit 3 and communication unit 4), and includes components such as a CPU that executes the program's instructions in sequence, and a storage device that stores the program, data required for executing the program, and processing results, etc. It can be said that the control program causes the hardware to function as the control unit 2.

[0021] The camera unit 3 takes pictures based on commands from the control unit 2. The taken images are stored in a storage device of the control unit 2. The camera unit 3 is fixed to the utility pole P, and the photographing area (angle of view) that can be photographed by the camera unit 3 is also fixed.

[0022] The communication unit 4 communicates with other camera edge terminals 1 and the vehicle C based on instructions from the control unit 2. The communication unit 4 and the communication units 4 of other camera edge terminals 1 are connected via the Internet through a server 40. The communication unit 4 and the vehicle C are also directly connected via Wi-Fi.

[0023] Next, a specific flow of controlling the vehicle C (mobile body) using the mobile body control system of the present invention configured as described above will be explained. As described above, even when controlling the vehicle C, the vehicle C basically drives autonomously, and the present system assists in this. To achieve this, the control unit 2 controls the camera edge terminal 1, various steps are executed by a control program, and the control unit 2 functions as various means to operate the camera edge terminal 1 in various ways. In other words, the control unit 2 has various means, and these various means operate the camera edge terminal 1 in various ways.

[0024] First, as shown in the flowchart of FIG. 3, a control program executes an object detection step S1 in a camera edge terminal 1, and the control unit 2 functions as an object detection means. The object detection means detects an object in a photographed area based on photographed data captured by the camera unit 3 and acquires information about the object's type, position, and velocity vector. More specifically, the object detection means activates the camera unit 3 to capture an image of the photographed area and detects a moving object in the photographed area as an object based on the acquired photographed data. That is, at this initial stage, the system does not search for a specific object, but rather identifies the detected moving object as the object. Note that methods for detecting moving objects from photographed data are well known, and therefore, their description will be omitted. Information about the object's type is basically information about its shape and color. Furthermore, if the object is a vehicle C, a marker representing identification information, such as a character, graphic, or two-dimensional code, may be attached to the vehicle C, and information about the type (individual information) of the vehicle C may be acquired by recognizing the marker in the photographed data. Furthermore, if the object is an obstacle B other than the vehicle C, the specific identity of the object may be estimated by comparing information about its shape and color with information about the shape and color of each expected type of obstacle B stored in advance in a storage device. The object detection means then tracks the detected object while it is moving within the shooting area of ​​the camera unit 3, and acquires the latest information about its position and velocity vector. The acquired information about the type, position, and velocity vector of the object is stored in the storage device of the control unit 2.

[0025] Next, the control program executes a probability distribution calculation step S2, and the control unit 2 functions as a probability distribution calculation means. When an object detected by the object detection means moves outside the shooting area of ​​the camera unit 3, the probability distribution calculation means calculates a probability distribution of an estimated position of the object based on information about the object's position and velocity vector. More specifically, the probability distribution calculation step S2 includes a position estimation step S21, a first probability distribution calculation step S22, and a second probability distribution calculation step S23. In other words, the probability distribution calculation means includes a position estimation means, a first probability distribution calculation means, and a second probability distribution calculation means.

[0026] First, a position estimation step S21 is executed by the control program, and the control unit 2 functions as a position estimation means. The position estimation means estimates how the object moves outside the imaging area from the history of the object's position and velocity vector, i.e., its movement trajectory, acquired by the object detection means. Then, an estimated position (estimated trajectory) is obtained each time a predetermined time has elapsed since the object left the imaging area. Regarding the method of estimating the object's movement, for example, if the object moves in a straight line within the imaging area, it is estimated that there is a high probability that it will continue to move in the same straight line outside the imaging area; if the object moves on an arc of a certain curvature within the imaging area, it is estimated that there is a high probability that it will continue to move on the same arc outside the imaging area. Furthermore, if the object moves at a constant speed within the imaging area, it is estimated that there is a high probability that it will continue to move at the same constant speed outside the imaging area; and if the object decelerates within the imaging area, it is estimated that there is a high probability that it will continue to decelerate outside the imaging area. However, these estimation methods are merely examples and may be appropriately set depending on conditions such as the surrounding environment. Furthermore, estimation based on the object's position and velocity vector history may also be supplemented with estimation based on the object's type. That is, estimation based on the object's position and velocity vector history may be corrected based on information stored in advance in a storage device about the characteristics of each type of object, such as how the object typically moves. The estimated position is stored in the storage device of the control unit 2.

[0027] Next, the control program executes a first probability distribution calculation step S22, and the control unit 2 functions as a first probability distribution calculation means. The first probability distribution calculation means calculates a first probability distribution regarding the image capture area of ​​the camera unit 3 of which camera edge terminal 1 the object will appear in. In other words, the first probability distribution is calculated based on the estimated position of the object calculated by the position estimation means, and is the probability that the object will appear in each of the image capture areas of each camera edge terminal 1 where the object may arrive. Specifically, the probability that the object will appear is higher in image capture areas closer to the estimated position. More specifically, how the probability is determined for the estimated position is set appropriately depending on conditions such as the surrounding environment. The calculated first probability distribution is stored in the storage device of the control unit 2.

[0028] Here, a specific example of the first probability distribution calculation means and the first probability distribution will be described with reference to FIG. 4. Here, it is assumed that there are four camera edge terminals (first camera edge terminal 1a, second camera edge terminal 1b, third camera edge terminal 1c, and fourth camera edge terminal 1d), and that a vehicle C as an object has been detected within the photographing area of ​​the first camera edge terminal 1a. As described above, when vehicle C moves out of the photographing area of ​​the first camera edge terminal 1a, the position estimation means calculates an estimated position (estimated trajectory) of vehicle C every predetermined time based on information about the vehicle C's position and velocity vector. Here, the time after the first predetermined time has elapsed (the time when the estimated position was first calculated) is set to t=0. Furthermore, the unit of time is set to seconds, and estimated positions (estimated positions at t=0, 1, 2, 3, ...) are calculated every second. Then, based on the estimated position at t=0, the first probability distribution calculation means calculates the probability that vehicle C will appear in the photographing area of ​​each camera edge terminal. However, it is assumed that vehicle C will always appear in one of the photographing areas of the four camera edge terminals. Therefore, the sum of the probabilities of vehicle C appearing in the image capture area of ​​each camera edge terminal is 1. In the example shown in FIG. 4, vehicle C is moving on a straight line, and its estimated position is also located on the same straight line, so the probability of vehicle C appearing in the image capture area of ​​the second camera edge terminal 1b, which overlaps with its estimated position, is highest. Here, the probability of vehicle C appearing in the image capture area of ​​the first camera edge terminal 1a is 0.1, the probability of vehicle C appearing in the image capture area of ​​the second camera edge terminal 1b is 0.45, the probability of vehicle C appearing in the image capture area of ​​the third camera edge terminal 1c is 0.15, and the probability of vehicle C appearing in the image capture area of ​​the fourth camera edge terminal 1d is 0.3. Since vehicle C may make a U-turn and return, the probability of vehicle C appearing again in the image capture area of ​​the first camera edge terminal 1a is not 0. In this way, the first probability distribution at t=0 is obtained.

[0029] Next, the control program executes a second probability distribution calculation step S23, and the control unit 2 functions as a second probability distribution calculation means. The second probability distribution calculation means calculates a second probability distribution regarding the position in the photographing area of ​​the camera unit 3 of the camera edge terminal 1 that is determined to have the highest probability in the first probability distribution at which the object will appear. That is, the second probability distribution is calculated by dividing the photographing area into a plurality of cells in a grid and calculating the probability that the object will appear in each cell in the photographing area based on the estimated position of the object calculated by the position estimation means. Specifically, the probability that the object will appear is higher in cells closer to the estimated position. More specifically, how the probability is determined for the estimated position is set appropriately depending on conditions such as the surrounding environment. The calculated second probability distribution is stored in the storage device of the control unit 2.

[0030] Here, a specific example of the second probability distribution calculation means and the second probability distribution will be shown with reference to FIG. 5. Following on from the example of the first probability distribution described above, the probability of vehicle C appearing at a position in the image capture area of ​​the camera unit 3 of the second camera edge terminal 1b, which is determined to have the highest probability, is calculated. To do this, the image capture area is divided into 10 cells in a grid. Then, based on the estimated position at t=0, the second probability distribution calculation means calculates the probability of vehicle C appearing in each cell. However, here, it is assumed that vehicle C will always appear in one of the 10 cells. Therefore, the sum of the probabilities of vehicle C appearing in each cell is 1. As with the first probability distribution, the probability of vehicle C appearing in the cell overlapping the estimated position is highest, and in this case, the probability of vehicle C appearing in that cell is 0.3, while the probability is lower in cells farther from the estimated position. In this way, the second probability distribution at t=0 is calculated.

[0031] Next, the control program executes an information sharing step S3, and the control unit 2 functions as an information sharing unit. The information sharing unit shares information about the object type, estimated position, and its probability distribution (first probability distribution and second probability distribution) with the other camera edge terminals 1. To do this, the information sharing unit issues a command to the communication unit 4 to upload the information obtained by the object detection unit and the probability distribution calculation unit from the storage device to the server 40. Then, in the other camera edge terminals 1, the control program also executes the information sharing step S3, and the control unit 2 functions as an information sharing unit. The information sharing unit of the other camera edge terminals 1 issues a command to the communication unit 4 to download the information from the server 40 to the storage device. In this way, this information is shared between the camera edge terminals 1. Then, subsequent steps in the control program are executed not in the original camera edge terminal 1 but in the other camera edge terminals 1 to which the information has been downloaded. In FIG. 3, the portion surrounded by a dashed line indicates the portion executed in the original camera edge terminal 1, and the portion surrounded by a dashed line indicates the portion executed in the other camera edge terminals 1.

[0032] Next, a tracking step S4 is executed by the control program, and the control unit 2 functions as a tracking means. The tracking means determines whether a moving object detected in the photographing area of ​​the camera unit 3 of the camera edge terminal 1, which is determined to have the highest probability in the first probability distribution, is the target object based on information about the probability distribution of the type of target object and the estimated position. More specifically, the tracking step S4 includes a redetection step S41, a match rate acquisition step S42, a probability acquisition step S43, and a determination step S44. In other words, the tracking means includes a redetection means, a match rate acquisition means, a probability acquisition means, and a determination means.

[0033] First, a redetection step S41 is executed by the control program, and the control unit 2 functions as a redetection means. The redetection means activates the camera unit 3 to capture an image of the image capture area, and detects a moving object in the image capture area based on the obtained image capture data. The redetection means then acquires information about the type, position, and velocity vector of the object. Information about the object type is basically information about its shape and color. Furthermore, if the object has a marker made up of characters, figures, two-dimensional codes, or the like that represent identification information, the marker is recognized in the image capture data and this information is acquired. The acquired information about the object type, position, and velocity vector is stored in the storage device of the control unit 2.

[0034] Subsequently, when a moving object is detected by the re-detection means, the control program executes a match rate acquisition step S42, and the control unit 2 functions as a match rate acquisition means. The match rate acquisition means calculates the match rate between the object and the target based on information about the type of the object. That is, the match rate acquisition means compares the information about the type of object acquired by the re-detection means with the information about the type of object acquired by the target detection means. Any method may be used to calculate the match rate, and examples of methods based on shape or color include a method of comparing the shooting data (images) of the object and the target to calculate the percentage of matching pixels. Other methods based on shape or color include a method of comparing shapes or colors extracted from the shooting data. Furthermore, if the object or target has a marker indicating identification information, a method of recognizing the identification information from the respective shooting data and comparing them may also be used. The calculated match rate is stored in the storage device of the control unit 2.

[0035] Subsequently, the control program executes a probability acquisition step S43, and the control unit 2 functions as a probability acquisition means. The probability acquisition means acquires the object existence probability, which is the probability in the second probability distribution of the position (cell) where the object is detected. The acquired object existence probability is stored in the storage device of the control unit 2.

[0036] Next, the control program executes a determination step S44, and the control unit 2 functions as a determination means. The determination means determines that the object is a target object if the match rate and the object existence probability are each greater than a predetermined value. The higher the match rate, the more likely the object is a target object, and the higher the object existence probability, the more likely the object is a target object. The determination means makes its determination based on both of these criteria. However, the predetermined values ​​for the match rate and the object existence probability may be a set of predetermined values, multiple sets of predetermined values, or may be expressed by one or more formulas. Note that if either the match rate or the object existence probability is very large, the object may be determined to be a target object regardless of the magnitude of the other value. This means that the predetermined value for the other is set to 0. The determination result is stored in the storage device of the control unit 2.

[0037] Next, if the determination means determines that the object is a target object, the control program executes information transmission step S5, and the control unit 2 functions as information transmission means. The information transmission means issues a command to the communication unit 4 to transmit information about the type, position, and velocity vector of the object to vehicle C. Vehicle C uses the transmitted information to assist autonomous driving. That is, if the object is vehicle C, vehicle C recognizes its own position and velocity vector, but can confirm accuracy and correct its own position and velocity vector as necessary by comparing them with the position and velocity vector obtained from the external camera edge terminal 1. Also, if the object is obstacle B, vehicle C can obtain information about surrounding obstacle B using its own cameras and sensors, but can obtain information about obstacle B before it is discovered by vehicle C, and can compare the information obtained by vehicle C with external information to confirm accuracy and correct it as necessary. This completes the processing of the control program when an object is discovered, i.e., when an object is detected by the redetection means and the determination means determines that the object is a target object.

[0038] On the other hand, if the object is not found, i.e., if the redetection means does not detect the object, or if the determination means determines that the object is not the target, the control program performs a different process. As described above, the tracking means searches for the object throughout the entire image capture area of ​​the camera unit 3 of the camera edge terminal 1, which is determined to have the highest probability in the first probability distribution. Therefore, if the object is not found, it means that the object is not found in the image capture area of ​​the camera unit 3 of the camera edge terminal 1, which is determined to have the highest probability in the first probability distribution, and that the object is not found in the cell (position) which is determined to have the highest probability in the second probability distribution. (In the second probability distribution, since the object was not found in all the target cells, it naturally means that the object was not found in the cell which is determined to have the highest probability.) In this case, as shown in FIG. 3, the control program returns to the process before the tracking step S4. At this time, the first and second probability distributions are updated by Bayesian estimation. First, the theory of Bayesian estimation and the update of the probability distributions based on this will be explained.

[0039] Let p be the probability that an object exists in a certain area, and q be the probability that an object will be found when searching when it exists in a certain area. Each event and probability is defined as follows: Event G: An object exists in a certain area Event G C :There is no object in a certain area P(G): The probability that an object exists in a certain area P(G)=p P(G C ): Probability that an object does not exist in a region ·P(G C )=1-p Event F: Object is discovered Event F C :Object not found P(F|G): The probability of finding an object when searching in a certain area P(F|G)=q P(F C|G): Probability of not finding an object when searching in a certain area ·P(F C |G)=1-q In this case, if the object is not found after searching a certain area (event F C ), the posterior probability p' when an object exists (event G) is expressed by the following equation (1) according to Bayes' theorem.

number

[0040] When there are multiple areas to be searched for an object, the probability that the object exists in each area is first calculated based on various conditions, and this is called the initial probability distribution. Then, if the area with the highest probability of the object being present is searched and the object is not found, the probability that the object exists in that area (posterior probability p') is calculated using the above formula (1). The posterior probability of the object being present in other areas is calculated by dividing the remaining probability 1-p' of the posterior probability p' by the probability at the time of the search (prior probability). In this way, a new probability distribution is obtained using Bayesian estimation based on the posterior probability p'. Obtaining this new probability distribution is called updating the probability distribution. The prior probability used in the first update is the initial probability distribution. Then, based on the updated probability distribution, the area with the highest probability of the object being present is searched in the same way. If the object is not found, the probability distribution is updated again, and this process is repeated. The prior probability used in the second and subsequent updates is the probability distribution that was updated immediately before.

[0041] Based on this theory, the following processing is performed. If the redetection means does not detect an object or the determination means determines that the object is not a target object, the control program first executes an initial setting update step S6, and the control unit 2 functions as the initial setting update means. The initial setting update means updates the initial setting probability distribution when the first and second probability distributions have been updated a predetermined number of times or more. That is, as described above, the initial setting probability distribution is sequentially updated using Bayesian estimation, and a target is searched for based on the updated probability distribution. However, if the target is not found despite repeated searches, it is assumed that the target has shifted over time from the estimated position on which the initial setting probability distribution is based, and a probability distribution based on the estimated position after that time has passed is set as the new initial setting probability distribution. Specifically, as described above, the position estimation means calculates estimated positions at t = 0, 1, 2, 3, ..., and the initial initial setting probability distribution is based on the estimated position at t = 0. As described below, the first and second probability distributions are updated every 0.1 seconds. Here, when the updating of the first probability distribution and the second probability distribution has been repeated 10 times and the time reaches t=1 and the target object has still not been found, the initial setting update means sets the first probability distribution and the second probability distribution based on the estimated position at t=1 as the initial setting probability distribution for updating each of them. Then, thereafter, unless the target object is found, the initial setting probability distribution is updated to the probability distribution based on the estimated position at t=2, the probability distribution based on the estimated position at t=3, and so on, every time the updating of the first probability distribution and the second probability distribution is repeated 10 times.

[0042] Next, the control program executes a first probability distribution update step S7, and the control unit 2 functions as a first probability distribution update means. The first probability distribution update means updates the first probability distribution by Bayesian estimation based on the posterior probability of an object being present in a photographing area of ​​the camera unit 3 of the camera edge terminal 1, which is determined to have the highest probability in the first probability distribution, when the object is not found in that photographing area. The area in the above theory here refers to the photographing area of ​​the camera unit 3 of each camera edge terminal 1. Furthermore, p, which is the probability that an object exists in a certain photographing area, is a value in the first probability distribution (initialized probability distribution) previously calculated by the first probability distribution calculation means. Furthermore, q, which is the probability that an object will be found when searching for an object when it exists in a certain photographing area, is a value determined in advance depending on the performance of each camera unit 3. The higher the performance of the camera unit 3, the higher the probability of discovery q, and the lower the performance of the camera unit 3, the lower the probability of discovery q. The probability q for each photographing area independently takes a value between 0 and 1.

[0043] Under these conditions, the first probability distribution update means uses equation (1) to calculate the posterior probability p' that an object exists in the photographing area of ​​the camera unit 3 of the camera edge terminal 1 that was determined to have the highest probability in the previously calculated first probability distribution when no object is found in that photographing area. The first probability distribution update means also calculates the posterior probabilities for the other camera edge terminals 1. In this case, the posterior probability that an object exists in the photographing area of ​​the camera unit 3 of the other camera edge terminals 1 (the sum of the posterior probabilities for the other camera edge terminals 1) is 1-p', and the posterior probabilities for each of the other camera edge terminals 1 are prorated according to the previously calculated first probability distribution. In this way, the first probability distribution is updated by Bayesian estimation.

[0044] Here, referring to FIG. 4, a specific example will be shown in which the first probability distribution updating means updates the first probability distribution using Bayesian estimation. Following the previous example of the first probability distribution, it is assumed that a vehicle C (object) was not found in the image capture area of ​​the camera unit 3 of the second camera edge terminal 1b, which was determined to have the highest probability. p, the probability that a vehicle C exists in a certain image capture area at t = 0, is calculated as described above. Here, the probability distribution is updated every 0.1 seconds. However, it is assumed that the object does not move during that 0.1 second. Furthermore, regarding the probability q, which is the probability that a vehicle C will be found when searching for it when it is present in a certain image capture area, as shown in FIG. 4, the probability is 0.35 for the image capture area of ​​the first camera edge terminal 1a, 0.25 for the image capture area of ​​the second camera edge terminal 1b, 0.25 for the image capture area of ​​the third camera edge terminal 1c, and 0.15 for the image capture area of ​​the fourth camera edge terminal 1d. Under these conditions, the first probability distribution update means calculates the posterior probability that vehicle C exists within the photographing area of ​​the camera unit 3 of the second camera edge terminal 1b using equation (1). Here, by substituting p = 0.45 and q = 0.25 into equation (1), the posterior probability p' = 0.3803 is calculated. The first probability distribution update means also calculates the posterior probabilities for the other camera edge terminals 1. The posterior probabilities for the other camera edge terminals 1 are calculated by proportionally dividing the posterior probability (1-p') that vehicle C exists within the photographing area of ​​the camera unit 3 of camera edge terminals 1 other than the second camera edge terminal 1b in the same ratio as the first probability distribution at t = 0. For example, the posterior probability for the first camera edge terminal 1a is calculated as (1-0.3803) × 0.1 / (0.1 + 0.15 + 0.3) = 0.1126. The posterior probabilities for the third camera edge terminal 1c and the fourth camera edge terminal are calculated in a similar manner. In this way, the first probability distribution at t=0.1 is obtained, that is, the first probability distribution is updated. In this example, even at t=0.1, the probability that vehicle C will be found within the photographing area of ​​the camera unit 3 of the second camera edge terminal 1b is the highest.

[0045] Next, the control program executes a second probability distribution update step S8, and the control unit 2 functions as a second probability distribution update means. The second probability distribution update means updates the second probability distribution by Bayesian estimation based on the posterior probability of an object being present in a cell (position) when an object is not found in the cell (position) with the highest probability in the second probability distribution. Here, the area in the above theory refers to the cells within the shooting area of ​​the camera unit 3 of the camera edge terminal 1. Furthermore, p, which is the probability that an object exists in a certain cell, is a value in the second probability distribution (initialized probability distribution) previously calculated by the second probability distribution calculation means. Furthermore, q, which is the probability that an object will be found when searching when it exists in a certain cell, is a predetermined value determined by the performance of the camera unit 3. If the performance of the camera unit 3 is high, the probability q of discovery will be high, and if the performance of the camera unit 3 is low, the probability q of discovery will be low. However, since each cell here is a division of the shooting area of ​​one camera unit 3, the probability q for all cells is the same value between 0 and 1.

[0046] Under these conditions, the second probability distribution update means uses equation (1) to calculate the posterior probability p' of the presence of an object in a cell that is determined to have the highest probability in the previously calculated second probability distribution when no object is found in that cell. The second probability distribution update means also calculates the posterior probabilities for the other cells. In this case, the posterior probability of the presence of an object in the other cells (the sum of the posterior probabilities for the other cells) is 1-p', and the posterior probabilities for each of the other cells are prorated according to the previously calculated second probability distribution. In this way, the second probability distribution is updated by Bayesian estimation.

[0047] Once the first and second probability distributions are updated in this manner, the subsequent processing of the control program proceeds in the same manner as in the previous case, and this is repeated until the object is discovered. For example, in the example shown in FIG. 4, the first probability distribution for t=0.1 is shown followed by the first probability distribution for t=0.2 and t=0.3, indicating that the object was discovered at t=0.3. The second probability distribution for t=0.2 and t=0.3 is calculated in the same manner as for t=0.1. At t=0 and t=0.1, the probability of discovering vehicle C (object) within the image capture area of ​​the camera unit 3 of the second camera edge terminal 1b was highest. However, from t=0.2 onward, the probability of discovering vehicle C within the image capture area of ​​the camera unit 3 of the fourth camera edge terminal 1d became highest, and that image capture area became the search target. Finally, at t=0.3, vehicle C was discovered within the image capture area of ​​the camera unit 3 of the fourth camera edge terminal 1d.

[0048] The processing of the control program explained up to this point is that an object is first detected within the photographing area of ​​the camera unit 3 of a certain camera edge terminal 1, and after that object moves out of that photographing area, it is again found within the photographing area of ​​the camera unit 3 of one of the camera edge terminals 1. As the object moves, the camera edge terminal 1 that performs the processing is switched, and this series of flows is repeated. If the object is a moving object (vehicle C), the movement of the moving object is tracked until it reaches its destination and the information is transmitted to the moving object itself, and if the object is obstacle B, the movement of obstacle B is tracked until it moves away from the moving object by a predetermined distance or more and the information is transmitted to the moving object.

[0049] According to the above-described embodiment of the mobile object control system of the present invention configured as described above, whether or not an object is a target object is determined based on the probability distribution of the estimated position of the target object, thereby improving the accuracy of target object detection and reducing false detections and delays. Furthermore, by transmitting the information about the target object thus obtained to the mobile object (vehicle C), the mobile object can avoid a collision with obstacle B with ample time to maneuver. Therefore, even when the mobile object has a wide range of movement, or when the range of movement of the mobile object is in a complicated location or a location with many obstructions, the mobile object can move more safely. Furthermore, since the probability distribution calculation means calculates a first probability distribution of which camera edge terminal 1's camera unit 3's shooting area the object will appear in, and a second probability distribution of which position the object will appear in the shooting area of ​​the camera unit 3 of the camera edge terminal 1 that is determined to have the highest probability in the first probability distribution, it is possible to efficiently determine which camera edge terminal 1 from multiple camera edge terminals 1 should attempt to detect the object, and to efficiently detect the object within the shooting area of ​​that camera edge terminal 1, thereby improving the accuracy of object detection. Furthermore, since the tracking means determines that an object is a target object based on the matching rate between the object and the target and the target existence probability, the accuracy of target detection is improved by using two indicators. Furthermore, since the first probability distribution and the second probability distribution are updated by Bayesian estimation, the reliability of the probability distributions is increased, and the accuracy of object detection is improved.

[0050] The present invention is not limited to the above-described embodiments. For example, the probability distribution calculation means is not limited to calculating the first probability distribution and the second probability distribution, and may calculate only one of them. Furthermore, the tracking means is not limited to determining that an object is a target based on the matching rate between the object and the target and the target existence probability, but may determine based only on the matching rate. Furthermore, the method for updating the first probability distribution and the second probability distribution is not limited to the Bayesian estimation method, and other known methods can also be used.

[0051] The camera edge terminal can be installed in various locations other than utility poles. For example, the camera edge terminal may be attached to a drone, which may be hovered to capture images of a predetermined area, or the drone may be moved to follow an object and capture images.

[0052] Furthermore, the control unit of each camera edge terminal has each means, but the means of the control unit of one camera edge terminal may function continuously, or the means of the control units of multiple camera edge terminals may function continuously across the board, and it is possible to set as appropriate which control unit of which camera edge terminal functions as which means. Furthermore, instead of each camera edge terminal having a control unit, a server on the cloud may have a control unit that controls each camera edge terminal, or both the camera edge terminal and the server on the cloud may have a control unit, and processing may be shared between the camera edge terminal and the server.

[0053] The communication unit may be directly connected to other camera edge terminals or mobile bodies via Wi-Fi or the like, or may be connected via a network such as the Internet via another device such as a server. In communication between the camera edge terminal and the mobile body, not only may the camera edge terminal transmit information such as the position and velocity vector of an object to the mobile body, but also, conversely, the mobile body may transmit information such as the position and velocity vector of the mobile body itself to the camera edge terminal, and this information may be used for calculating a probability distribution. In addition to the information described in the above embodiment, various information may be transmitted at various times between the camera edge terminal and other camera edge terminals or mobile bodies.

[0054] Furthermore, when the probability distribution calculation means initially calculates the probability distribution, it may calculate the probability distribution based on a predetermined method from the beginning, rather than first calculating an estimated position and then calculating the probability distribution based on that. Furthermore, q, which is the probability that an object will be found when searching when it exists in a certain area, may be determined based on the performance of the camera unit, or may be determined based on various environmental conditions of the camera unit's shooting area. Furthermore, the tracking means may search not only the photographing area of ​​the camera unit of the camera edge terminal that is determined to have the highest probability of the object being present in the first probability distribution, but also photographing areas of the camera units of other camera edge terminals that are determined to have the possibility of the object being present therein. In this case, the control units of multiple camera edge terminals simultaneously function as tracking means.

[0055] Furthermore, in the above embodiment, only information about the type, position, and velocity vector of the object is transmitted from the camera edge terminal to the moving body, but the camera edge terminal may also have a path calculation means for calculating the path of the moving body based on this information about the object, and transmit the calculated path to the moving body. Furthermore, if the target is a flying object such as a drone or bird, its position and velocity vector are acquired as three-dimensional information. The photographed area is also a three-dimensional space, and when calculating the second probability distribution, the photographed area is divided into three-dimensional cells.

[0056] The above embodiment is based on the premise that the unit time when the first probability distribution update means and the second probability distribution update means update the probability distributions is very short, and that the target object is considered not to be moving during that time. In contrast, if the target object is moving faster and within the unit time, updating the probability distribution using Bayesian estimation based on the posterior probability will not be possible because the target object has already moved by that time. Therefore, in this case, the probability distribution is calculated based on the estimated position of the target object, which is calculated from the target object's position and velocity vector information, rather than based on the posterior probability, each time a search is performed. [Explanation of symbols]

[0057] 1. Camera edge terminal 2. Control Unit 3 Camera section 4. Communications Department

Claims

1. A mobile body control system including a plurality of camera edge terminals, which transmits information about a moving object obtained by the camera edge terminals to a mobile body having an automatic driving function, Each of the camera edge terminals has a control unit for controlling itself, a camera unit for photographing a predetermined area, and a communication unit for communicating with other camera edge terminals and the mobile body, The control unit an object detection means for detecting the object in the photographing area based on photographing data taken by the camera unit and acquiring information on the type, position and velocity vector of the object; a probability distribution calculation means for calculating a probability distribution of an estimated position of the object based on information on the position and velocity vector of the object when the object moves outside the photographing area of ​​the camera unit; an information sharing means for sharing information about the probability distribution of the type and estimated position of the object with other camera edge terminals; a tracking means for determining whether a moving object detected within the photographing area is the target object based on information about the probability distribution of the type and estimated position of the target object when the probability distribution indicates that the target object is located within the photographing area of ​​the camera unit; A mobile object control system comprising:

2. The probability distribution calculation means a first probability distribution calculation means for calculating a first probability distribution regarding whether the object will appear in the photographing area of ​​the camera unit of any of the camera edge terminals; a second probability distribution calculation means for calculating a second probability distribution regarding the position at which the object will appear within the photographing area of ​​the camera unit of the camera edge terminal that is determined to have the highest probability in the first probability distribution; 2. The mobile object control system according to claim 1, further comprising:

3. The tracking means a matching rate acquisition means for acquiring a matching rate between the object and the target based on information about the type of the target; a probability acquisition means for acquiring an object presence probability, which is a probability in the second probability distribution of the position where the object is detected; a determination means for determining that the object is the target object when the matching rate and the target object existence probability are greater than predetermined values; 3. The mobile object control system according to claim 2, further comprising:

4. The control unit The mobile object control system according to claim 2 or 3, characterized in that it has a first probability distribution update means that updates the first probability distribution by Bayesian estimation based on the posterior probability of the object being present in the shooting area of ​​the camera unit of the camera edge terminal when the object is not found in that shooting area, which is determined to have the highest probability in the first probability distribution.

5. The control unit A mobile object control system according to claim 2, 3 or 4, characterized in that it has a second probability distribution update means that updates the second probability distribution by Bayesian estimation based on the posterior probability of the object being present at a position that is deemed to have the highest probability in the second probability distribution when the object is not found at that position.

Images