Positioning System

Abstract

To provide a position measurement system which can highly accurately measure the position of each measurement object in an area where the plurality of measurement objects exist.SOLUTION: A position measurement system comprises: a plurality of pieces of imaging means which image an own position detection measurement object having an own position detection part and another measurement object having no own position detection part; own position measurement means which obtains the own position of the own position detection measurement object on the basis of the own position information from the own position detection part; image analysis means which estimates the positions of the own position detection measurement object and the other measurement object for each photographed image of the plurality of pieces of imaging means; position determination means which compares the own position of the own position detection measurement object measured by the own position measurement means with the estimated position of the own position detection measurement object for each photographed image obtained by the image analysis means, selects and determines one estimated position of the other measurement object for each photographed image according to the comparison result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a position measuring system for a moving measurement object, and more particularly to a position measuring system for measuring the position of a measurement object by using an imaging means. [Background technology]

[0002] In recent years, with the decline in the working population, there are growing expectations for the automation and autonomy of work systems to resolve labor shortages and improve productivity.For example, in a work area such as a factory where workers and moving measurement objects such as autonomous vehicles and robots coexist, it is possible to build a work system that balances safety and productivity by measuring the positions of the measurement objects and using a technique such as projection mapping to present the movable range (workable area, safe area, etc.) within which the measurement objects can safely move.

[0003] To smoothly manage such a system, it is necessary to constantly measure the position of the measurement object with high accuracy. For example, when there is an autonomously moving measurement object (vehicle, robot, etc.), in order to safely move the measurement object, it is common to mount sensors such as a camera or LiDAR (Light Detection and Ranging) on ​​the measurement object to detect structures or obstacles (e.g., work machines, pillars, walls, fixtures, etc.) around the measurement object and measure its positional relationship with the structures and obstacles.

[0004] However, it is difficult to measure many structures and obstacles in the work area using only the sensors mounted on the object to be measured, due to blind spots caused by structures and obstacles. Therefore, by measuring the work area using fixed sensors such as cameras and LiDAR around the work area and integrating the results with those of the sensors mounted on the object to be measured, it becomes possible to measure the positions of many structures and obstacles with high accuracy.

[0005] However, in various types of work sites, the measurement results of many sensors are not necessarily accurate due to various disturbances (e.g., disturbances of the optical system), and it is necessary to select the measurement data detected by each sensor according to the environmental conditions of the work site. For example, JP 2020-52600 A (Patent Document 1) discloses the following.

[0006] Patent Document 1 shows that camera images of objects (subjects to be photographed) are acquired from multiple different viewpoints, the position of each object is estimated, the degree of occlusion between objects is calculated for each camera based on the viewpoint of each camera and the position of each object, a camera to be used to identify each object is selected based on this degree of occlusion, and each object is identified based on the camera images from the camera selected for each object. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2020-52600 A Summary of the Invention [Problem to be solved by the invention]

[0008] As described above, in the field of position measurement systems, there is a strong demand for constantly measuring the position of a measurement target with high accuracy. However, in Patent Document 1, a camera image is selected according to the degree of occlusion, and no consideration is given to improving the measurement accuracy of the object position using the camera image.

[0009] An object of the present invention is to provide a position measurement system capable of measuring the position of each measurement object with high accuracy in an area where a plurality of measurement objects exist. [Means for solving the problem]

[0010] A first feature of the present invention is that it comprises a plurality of imaging means for photographing a self-position detection measurement object equipped with a self-position detection unit, and other measurement objects not equipped with a self-position detection unit, a self-position measuring means for determining the self-position of the self-position detection measurement object based on self-position information from the self-position detection unit, an image analysis means for estimating the positions of the self-position detection measurement object and the other measurement objects for each image captured by the plurality of imaging means, and a position determination means for comparing the self-position of the self-position detection measurement object measured by the self-position measuring means with the estimated position of the self-position detection measurement object for each image captured by the image analysis means, and selecting and determining one of the estimated positions of the other measurement objects for each image captured in accordance with the comparison result.

[0011] A second feature of the present invention is that it comprises a plurality of imaging means for photographing a self-position detection measurement object equipped with a self-position detection unit, and other measurement objects not equipped with a self-position detection unit, a self-position measurement means for determining the self-position of the self-position detection measurement object based on self-position information from the self-position detection unit, a measurement object position estimation means for estimating the positions of the self-position detection measurement object and the other measurement objects from the captured images captured by each imaging means, a measurement error estimation means for comparing the self-position of the self-position detection measurement object measured by the self-position measurement means with the estimated position of the self-position detection measurement object estimated by the measurement object position estimation means to determine a measurement error and estimate the measurement error of the other measurement objects based on this measurement error, and a position determination means for selecting other measurement objects with smaller measurement errors estimated by the measurement error estimation means and determining the estimated positions of the other measurement objects. Effect of the Invention

[0012] According to the present invention, the position of another measurement target object that does not have a self-position detection unit can be accurately obtained, so that the accuracy of position estimation can be improved. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a functional block diagram showing functions of a position measurement system according to a first embodiment of the present invention. [Diagram 2] 2 is a configuration diagram of a vehicle information acquisition unit shown in FIG. 1. [Diagram 3] FIG. 2 is an explanatory diagram illustrating a function of a self-location information acquisition unit; [Figure 4] 2 is a configuration diagram illustrating the configuration of an image analysis unit shown in FIG. 1. [Diagram 5] 5 is an explanatory diagram illustrating functions of an object detection unit and a position measurement unit shown in FIG. 4. [Figure 6] 5 is a configuration diagram of a measurement error estimating unit shown in FIG. 4. [Figure 7A] 7 is an explanatory diagram for explaining a vehicle measurement error calculation unit and a worker measurement error estimating unit in FIG. 6. [Figure 7B] 7 is an explanatory diagram illustrating an example of measurement errors by a vehicle measurement error calculation unit and a worker measurement error estimating unit in FIG. 6. [Figure 8] 2 is an explanatory diagram illustrating a work area presenting unit shown in FIG. 1. [Figure 9] 2 is a configuration diagram of a work area inside / outside determination unit shown in FIG. 1. [Figure 10] FIG. 5 is a functional block diagram showing functions of a position measurement system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiment, and various modifications and application examples within the technical concept of the present invention are also included within its scope. EXAMPLES

[0015] 1 is a functional block diagram showing the functions of the first embodiment of the present invention. In this embodiment, a case where multiple cameras are used as a position estimation means will be described, but the position estimation means is not limited to this, and other sensors such as a stereo camera or a distance sensor such as LiDAR can also be used, and further, multiple different types of sensors can be combined.

[0016] The position measurement system 1 shown in Fig. 1 is a system in which a plurality of cameras (imaging means in claims / hereinafter referred to as infrastructure cameras) 3a, 3b are installed in a work area in which vehicles (self-position detection measurement object in claims) 2b capable of acquiring self-position information and workers (other measurement objects in claims) 2a coexist, and the positions of the vehicles 2b and workers 2a are measured. Note that the number of infrastructure cameras is not limited to two. In the following description, the term "measurement object" may be used in addition to the vehicles 2b and workers 2a.

[0017] Here, the vehicle 2b, which can acquire its own position information, can accurately determine its own position by its own position detection unit, for example, a GPS sensor position detection function or an image pattern recognition function. In addition, the positions of other measurement objects, such as the worker 2a, whose own position information cannot be acquired, are estimated based on the images captured by the infrastructure cameras 3a and 3b. Of course, the positions of the measurement objects for which the own position is detected are also estimated based on the images captured by the infrastructure cameras 3a and 3b. This is used to calculate the measurement error, which will be described later.

[0018] Using the positions of each measurement object estimated by this position measurement system 1, workable areas (specific areas such as safe areas, dangerous areas, etc.) can be displayed in the work area by a map display device 4 using a projection mapping technique. For this reason, accurate position information of the measurement objects is required. Note that projection mapping is just one example, and the method can be applied to other work systems as well.

[0019] Incidentally, the accurate position of the measurement target (vehicle 2b) whose own position information can be acquired can basically be obtained, but the position of the other measurement target (worker 2a) whose own position information cannot be acquired is estimated from the images captured by the infrastructure cameras 3a and 3b, and the measurement error differs depending on which image captured by the infrastructure cameras 3a and 3b is used to estimate the position. This embodiment is characterized in that the position of the other measurement target (worker 2a) whose own position information cannot be acquired is determined by selecting an image with a small measurement error.

[0020] In Fig. 1, the position measurement system 1 is composed of functional blocks such as a vehicle information acquisition unit 5, an image analysis unit 6, a measurement error estimation unit 7, a position information selection and determination unit 8, a work area presentation unit 9, and a work area inside / outside determination unit 10. The measurement error estimation unit 7 and the position information selection and determination unit 8 together constitute a position determination unit 18. Each functional block is realized in a computer having an arithmetic unit, a main memory device, and an external memory device. Next, an overview of each functional block shown in Fig. 1 will be described.

[0021] The vehicle information acquisition unit 5 has a function of acquiring the vehicle 2's own position information and, if an external sensor is installed, the sensor's measurement information. The image analysis unit 6 has a function of measuring the positions of the worker (other measurement object) 2a and the vehicle (self-position detection measurement object) 2b by analyzing the captured images acquired from the infrastructure cameras 3a and 3b.

[0022] The measurement error estimation unit 7 has the function of comparing the own position of the vehicle (measurement object for detecting its own position) 2b acquired from the vehicle information acquisition unit 5 with the estimated position of the vehicle (measurement object for detecting its own position) 2b acquired from the image analysis unit 6, and estimating the measurement error of the vehicle (measurement object for detecting its own position) 2b by the infrastructure cameras 3a, 3b.

[0023] The position information selection and determination unit 8 has the function of using the estimated measurement error to select and determine the estimated position to be ultimately used in the position measurement system 1 from among the estimated positions of the worker (other measurement object) 2a obtained by the image analysis unit 6.

[0024] The work area presenting unit 9 has a function of analyzing the position information of the vehicle (self-position detection measurement object) 2b and the worker (other measurement object) 2a acquired by the position information selecting and determining unit 8, and presenting the movable range (specific areas such as workable area, safe area, etc.) of the vehicle (self-position detection measurement object) 2b and the worker (other measurement object) 2a in the work area by the map display device 4. The work area inside / outside determining unit 10 has a function of determining whether the vehicle (self-position detection measurement object) 2b and the worker (other measurement object) 2a are present in the presented specific area, and providing feedback to the position information selecting and determining unit 8.

[0025] Next, the functions of the functional blocks such as the vehicle information acquisition unit 5, image analysis unit 6, measurement error estimation unit 7, position information selection and determination unit 8, work area presentation unit 9, and work area inside / outside determination unit 10 will be described in detail.

[0026] 2 shows functional blocks of the vehicle information acquisition unit 5. The vehicle information acquisition unit 5 includes a self-location information acquisition unit 11 that acquires self-location information by a GPS sensor or the like installed in the vehicle 2, and an external sensor information acquisition unit 12 that acquires measurement information of an external sensor mounted in the vehicle 2. The functions of the self-location information acquisition unit 11 and the external sensor information acquisition unit 12 will be described below.

[0027] The self-location information acquisition unit 11 will be described with reference to Fig. 3. In Fig. 3, the location information of the vehicle 2, infrastructure cameras 3a and 3b, and map display device 4 used in the location measurement system 1 is pre-calibrated with respect to a work area map 15 to which the location measurement system 1 is applied, and the location information is handled in a common coordinate system 16 with "Og" as the origin.

[0028] The local position information acquired by the local position information acquisition unit 11 includes the position information of the origin "Om" of the vehicle coordinate system 17 in the common coordinate system 16 shown in Figure 3, and the inclination "θ" of the vehicle coordinate system 17 relative to the common coordinate system 16 indicating the orientation of the vehicle.

[0029] This information can be obtained using a GPS sensor, or by installing a LiDAR or camera on the vehicle 2b and using technologies such as SLAM (Simultaneous Localization and Mapping), but there is no particular limitation as long as the method can obtain the vehicle 2b's position information with high accuracy.

[0030] When the vehicle 2b is equipped with an external sensor such as a camera or LiDAR, the external sensor information acquisition unit 12 can acquire shape information of structures around the vehicle 2b from the external sensor. For example, when a camera is installed, the external sensor information acquisition unit 12 can acquire position information by analyzing the image captured by the camera to obtain shape information of the surroundings, similar to the image analysis unit 6 of the infrastructure cameras 3a and 3b. The position measurement method by the image analysis unit 6 will be described later. When a distance sensor such as LiDAR is installed, the measured three-dimensional point cloud information can be analyzed to acquire position information of structures having a certain height or more.

[0031] In addition, when the position information from the external sensor is position information in the vehicle coordinate system 17 with "Om" shown in Fig. 3 as the origin, it can be converted into position information on the common coordinate system 16 using the vehicle's own position information. In addition, when an external sensor is not installed, the configuration may not include this functional block. Furthermore, when an external sensor is installed, there is no particular limitation on the type of sensor or the measurement algorithm as long as it can measure the position information of structures existing around the vehicle 2b.

[0032] 4 shows functional blocks of the image analysis unit 6. The image analysis unit 6 includes an image acquisition unit 20 that acquires captured images from the infrastructure cameras 3a and 3b, an object detection unit 21 that extracts the area of ​​a measurement target in the position measurement system 1 that exists in the captured image by applying an image recognition algorithm or the like to each captured image, and a position measurement unit 22 that acquires the position information of the measurement target based on the area information of the measurement target in the extracted image. The object detection unit 21 and the position measurement unit 22 will be described in detail below.

[0033] The object detection unit 21 and the position measurement unit 22 will be described with reference to Fig. 5. In Fig. 5, a captured image 30 is an image acquired by the image acquisition unit 20. An image coordinate system 31 is a coordinate system with its origin at "Oi" in the upper left of the image. A worker 32a and a vehicle 32b, which are objects to be measured, are surrounded by detection frames 33a and 33b that are areas including the objects to be measured estimated by the object detection unit 21 from within the image.

[0034] Image coordinates 34a, 34b of the detection frames 33a, 33b are set at the center of the bottom ends of the detection frames 33a, 33b. Camera parameters 35 are the camera parameters of the infrastructure cameras 3a, 3b used to acquire the captured images. The world coordinate system 36 is a coordinate system whose origin is "Ow" converted by the position measurement unit 22 using the image coordinate system 31 and the camera parameters 35.

[0035] The world coordinates 37a and 37b are world coordinates corresponding to the image coordinates 34a and 34b. The coordinates 38a and 38b are obtained by converting the world coordinates 37a and 37b into coordinate values ​​in the common coordinate system 16 and plotting them on the working area map 15. The position information 39 indicates the position information output from the position measurement unit 22.

[0036] The object detection unit 21 distinguishes and detects the measurement object (the worker 2a or the vehicle 2b) present in the captured image by utilizing (pattern matching) a dictionary capable of detecting the measurement object (the worker 2a or the vehicle 2b) from the image generated in advance from learning data by machine learning or the like. The algorithm for generating the dictionary may be a general one such as a convolutional neural network or AdaBoost, and is not particularly limited. In addition, other than this example, any method may be used as long as it can distinguish and detect the measurement object from the captured image and estimate its circumscribing rectangle (bounding box).

[0037] The position measurement unit 22 converts the image coordinates 34a, 34b of the detection frames 33a, 33b of the object to be measured acquired by the object detection unit 21 into position information in a world coordinate system 36 and a common coordinate system 16 of the object to be measured. The conversion from image coordinates to world coordinates utilizes a general method using camera parameters 35.

[0038] Incidentally, as a method for deriving the camera parameters 35, general camera calibration may be used, and the camera parameters 35 may be estimated by a method of manually giving in advance world coordinates 36 corresponding to the image coordinates 31. As a method for converting the world coordinates 37a, 37b into the common coordinates 38a, 38b, any general method such as performing calibration in advance between the infrastructure cameras 3a, 3b and the work area map 15 and performing conversion using a perspective transformation matrix may be used, and is not particularly limited.

[0039] The position measurement unit 22 acquires position information 39 of the measurement object (worker 2a or vehicle 2b) by the method described above. As the position information, three pieces of information are acquired: class information, score, and position of each of the measurement objects 32a, 32b. The class information is the identification result of the measurement object (worker 2a or vehicle 2b) by the object detection unit 21, the score is an index indicating the probability that the measurement object (worker 2a or vehicle 2b) exists within the rectangle of the detection frames 33a, 33b, and the position is the two-dimensional coordinate value of the measurement object (worker 2a or vehicle 2b) in the common coordinate system 16.

[0040] 6 shows functional blocks of the measurement error estimation unit 7. The measurement error estimation unit 7 includes a vehicle measurement error calculation unit 40 that compares information on the vehicle 2b's own position acquired from the vehicle information acquisition unit 5 with information on the estimated position of the vehicle 2b estimated by the image analysis unit 6 to calculate the estimated measurement error of the vehicle 2b, and a worker measurement error estimation unit 41 that estimates the measurement error of the estimated position of the worker 2a estimated by the image analysis unit 6 based on the calculated vehicle measurement error information. The vehicle measurement error calculation unit 40 and the worker measurement error estimation unit 41 will be described in detail below.

[0041] 7A and 7B, the vehicle measurement error calculation unit 40 and the worker measurement estimation calculation unit 41 will be described. In FIG. 7A, the vehicle 2b's own position 50 is the own position coordinate (x 3 g, y 3 The estimated position 51 of the vehicle 2b is the estimated position coordinates (x 2 g, y 2 g).

[0042] The measurement error (σ2) 52 is a measurement error calculated by comparing the vehicle's own position 50 with the estimated position 51 by the image analysis unit 6. In FIG. 7A, the area of ​​the measurement error is indicated by a dashed line. The estimated position 53 of the worker 2a is calculated by subtracting the position information (x 1 g, y 1 g). Measurement error (σ1) 54 is the measurement error of the worker 2a estimated from the measurement error 52 of the vehicle 2b. In Fig. 7A, the area of ​​the measurement error is indicated by a dashed line. Output information 55 is finally output from the measurement error estimation unit 7, as shown in Fig. 7B.

[0043] The vehicle measurement error calculation unit 40 judges whether or not the own position 50 of the vehicle 2b exists in the error region of the estimated position 51 of the vehicle 2b output by the image analysis unit 6. The judgment method is not particularly limited, and may be a method using the Euclidean distance between the coordinates of the estimated position 51 and the own position 50 of the vehicle 2b.

[0044] Then, when the estimated position 51 of the vehicle 2b by the image analysis unit 6 does not exist, the measurement error is deemed unmeasurable and the function of the position information selection and determination unit 8 is executed. On the other hand, when the estimated position 51 of the vehicle 2b by the image analysis unit 6 exists, a circular area having a radius equal to the Euclidean distance from the vehicle 2b's own position information 50 is determined as the measurement error 52.

[0045] Once the measurement error 52 is obtained, the worker measurement error estimation unit 41 then estimates a worker position measurement error 54 from the measurement error 52 and the estimated position 53 of the worker 2a estimated by the image analysis unit 6. One estimation method is to use information on the positional relationships between the infrastructure cameras 3a, 3b and each of the measurement targets.

[0046] Normally, in position measurement methods using cameras, the measurement accuracy of a measurement target that is located at a position farther away from the camera tends to decrease. For this reason, the positions of the infrastructure cameras 3a and 3b in the common coordinate system 16 are calculated in advance, and the circular area of ​​the measurement error 54 can be estimated by a method in which the measurement error 52 is linearly increased or decreased according to the Euclidean distance between the positions of the infrastructure cameras 3a and 3b and the estimated positions 51 and 53 of the respective measurement targets (worker 2a and vehicle 2b).

[0047] Furthermore, when estimating the circular area of ​​the measurement error, instead of using the Euclidean distance in the common coordinate system 16 between the positions of the infrastructure cameras 3a and 3b and each of the measurement objects (worker 2a, vehicle 2b) as in this embodiment, a method of utilizing the y coordinate value in the image coordinate system 31, or a method of estimating the distance estimation error for each pixel of the y coordinate value in the image coordinate system 31 from information on the angle of view and resolution and utilizing this as the increase / decrease range of the measurement error 52, may be used. Furthermore, when estimating the circular area of ​​the measurement error, a process of correcting the increase / decrease in the measurement error according to the size of the measurement object may be added, taking into account class information.

[0048] Then, the measurement error estimation unit 7 adds the information on the measurement errors 52 and 54 obtained by the above-mentioned method to the position information output from the image analysis unit 6 as "σ2" and "σ1", and further adds the information on the own position (x 3 g, y 3 By adding g), the information shown in the output information 55 is input to the position information selection and determination unit 8 in the subsequent stage.

[0049] The location information selection and determination unit 8 receives the images captured by each of the infrastructure cameras 3a, 3b as input, and determines the estimated position of the measurement object (worker 2) to be ultimately used in the position measurement system 1 from the estimated position and measurement error for each measurement object (worker 2) output from the image analysis unit 6 and the measurement error estimation unit 7.

[0050] The method of determining this will be described below. The measurement error can be obtained by the method described above.

[0051] First, the system extracts the estimated position of the measurement object (in this case, the worker 2a) on the common coordinate system 16 measured by two different infrastructure cameras 3a and 3b. Next, the system calculates the mutual Euclidean distance from the two estimated positions, and if the calculated Euclidean distance is equal to or less than a predetermined distance threshold and the classes are the same, the measurement objects are determined to be the same object (in this case, the worker 2a).

[0052] Then, the measurement error (σ1) of the estimated position of the measurement target (worker 2a) determined to be the same target obtained for each infrastructure camera 3a, 3b is compared, and the one with the smaller measurement error (σ1) is selected as the estimated position with higher reliability. By executing these processes for the measurement target (worker 2a) in the images captured by all infrastructure cameras 3a, 3b, the finally selected estimated position is output as the estimated position with higher reliability.

[0053] In this embodiment, the vehicle 2b's own position information is always treated as "true", and the measurement error is estimated from this to select the final estimated position of the worker 2a. However, it is also possible to estimate the reliability of the vehicle 2b's own position information, and if the reliability is low, calculate the Euclidean distance between the two estimated positions, and if the calculated Euclidean distance is less than a predetermined distance threshold and the classes are the same, use the average value of the measurement errors.

[0054] Here, as a method for calculating the reliability of the vehicle's own position information, for example, when using position information from a GPS sensor, there is a method in which, by analyzing captured images or embedding information about structures in advance in a work area map, if the vehicle 2 is moving under a roof or near various structures, the detection sensitivity of the GPS sensor decreases and the reliability is estimated to be low.

[0055] In addition, a method may be used in which the positions of the infrastructure cameras 3a and 3b in the common coordinate system 16 are determined in advance, the estimated position of the measurement object close to the infrastructure cameras 3a and 3b is determined to have high reliability, and the estimated position of the infrastructure camera close to the measurement object is selected.

[0056] Also, the final estimated position may be selected by using information other than the measurement error estimated from the vehicle 2b's own position information. For example, in a situation where a shadow of a measurement target is cast by sunlight or the like, if a shadow is included in the detection frames 33a, 33b detected by the image analysis unit 6, the measurement accuracy decreases. Therefore, a method may be used that considers information such as the shooting time and the positional relationship between the sun and the infrastructure cameras 3a, 3b, and selects the estimated position of the captured image of the infrastructure cameras 3a, 3b that is expected to have less measurement error and less influence of the shadow.

[0057] In addition, a method may be used in which final position information is selected taking into consideration information on the density between measurement objects estimated from the estimated position of each measurement object in the common coordinate system 16 by the image analysis unit 6, information on the position of the measurement objects in the common coordinate system 16, trajectory information of the measurement objects is obtained by analyzing the information, and posture information and behavior information of the measurement objects recognized by applying an image processing algorithm to the captured image.

[0058] Furthermore, in addition to this example, information that is thought to affect the accuracy of the estimated position of the measurement object and that can be obtained by analyzing the estimated position of each measurement object in the common coordinate system 16 or a captured image may be utilized.

[0059] Next, the work area display unit 9 will be described with reference to Fig. 8. In Fig. 8, divided areas 60 are formed on the work area map 15 by dividing the entire area into fixed sizes in advance. Then, a workable area 61 for the worker 2a is displayed on the actual work area surface by the map display device 4, and a workable area 62 for the vehicle 2b is displayed on the actual work area surface by the map display device 4. These workable areas 61, 62 are determined based on the estimated positions of the worker 2a and the vehicle 2b.

[0060] In the work area presentation unit 9, based on the information on the estimated position output from the position information selection and determination unit 8, in a situation where measurement objects such as workers 2a and vehicles 2b are mixed within the work area, the workable area is displayed (projection mapping) on ​​the actual work area surface by the map display device 4 in order to display the workable area for the workers 2a and the vehicles 2b.

[0061] When the worker 2a deviates from the available work area, an alarm can be issued, and when the vehicle 2b deviates from the available work area, a stop instruction can be transmitted. Next, specific processing will be described.

[0062] First, the entire work area is divided into small divided areas 60, such as 1m x 1m. Next, a divided area 60 including the position (one point) of the measurement object (worker 2a, vehicle 2b) is selected, and the selected divided area 60 and the surrounding divided areas 60 are determined to be part of the workable area according to the size of the class information of the measurement object (worker 2a, vehicle 2b).

[0063] Then, by comparing the estimated positions of each measurement object (worker 2a, vehicle 2b) with those measured in the past, the moving direction and speed information of the measurement object (worker 2a, vehicle 2b) in the future are estimated, and the direction in which the workable area is to be expanded according to the moving direction and the number of divided regions 60 by which the work area is to be expanded according to the speed information are determined. Finally, the determined workable area is displayed on the actual work area surface by the map display device 4, which has been calibrated in advance with the common coordinate system 16.

[0064] The method of determining the workable area is not limited to this example, and is not particularly limited as long as it is an algorithm that can predict the behavior of the measurement target (worker 2a, vehicle 2b). In addition, the map display device 4 is used in this embodiment, but it is not limited to the map display device 4, and the workable area may be displayed using a display or a light-emitting body such as a light embedded in advance on the work area surface. In addition, the worker 2a may carry a display device, and the workable area may be displayed on the display device.

[0065] Furthermore, the worker 2a may wear a head-mounted display (so-called VR goggles) and a virtual workable area may be displayed on the display. By using such a head-mounted display with an augmented reality function that superimposes the workable area on the surrounding environment, the worker 2a can obtain information equivalent to that displayed on the work area surface with a similar sensation, even if nothing is displayed on the work area surface. This makes it possible to suppress a decrease in work efficiency compared to having the worker 2a display on a portable display device.

[0066] 9 shows functional blocks of the work area inside / outside determination unit 10. The work area inside / outside determination unit 10 includes a workable area recognition unit 70 that recognizes the workable area presented by the map display device 4 from the captured images by analyzing the images captured by the respective infrastructure cameras 3a, 3b, an object area extraction unit 71 that extracts an object area including a measurement target (worker 2a, vehicle 2b) based on the difference between the captured images of the respective infrastructure cameras 3a, 3b and a background image stored in advance, and an object area determination unit 72 that determines whether the object area exists within the workable area. The workable area recognition unit 70, the object area extraction unit 71, and the object area determination unit 72 will be described below.

[0067] The workable area recognition unit 70 recognizes the region of the workable area from the captured image. Recognition methods include a method of extracting the workable area from the captured image based on color information of the divided area presented in advance by the map display device 4, and a method of roughly estimating the position of the workable area on the captured image based on information on the position and size of the workable area determined by the work area presentation unit 9, and then extracting it by utilizing color information, etc.

[0068] The object region extraction unit 71 holds background images captured in advance by the infrastructure cameras 3a and 3b, and obtains a rectangular object region in which the measurement object (worker 2a, vehicle 2b) exists by taking the difference between the input new captured image and the background image and performing binarization processing. Then, based on the position information of the measurement object from the infrastructure cameras 3a and 3b and the rectangular information of the detection frames 33a and 33b output by the image analysis unit 6, it is recognized which object region the measurement object (worker 2a, vehicle 2b) is included in.

[0069] In addition, when extracting an object area, in addition to the method of acquiring a background image, frame difference processing or the like may be used, and there is no particular limitation as long as the method is capable of extracting an area containing the measurement target from the captured image.

[0070] The object determination unit 72 calculates "1" if the bottom end portion of the object region extracted from the images captured by the infrastructure cameras 3a, 3b is included within the workable area, and "0" if it is not. Then, if "1" is calculated for all captured images, it determines that the object region is included within the workable area, and feeds back this result to the position information selection and determination unit 8. The position information selection and determination unit 8 holds the fed back determination result, and uses this determination result when finally determining position information for the next frame and onwards.

[0071] For example, in a situation where the estimated position of a certain measurement object (worker 2a, vehicle 2b) by the same infrastructure camera is consecutively selected as the final estimated position, if the measurement object (worker 2a, vehicle 2b) is not present within the workable area by the work area inside / outside determination unit 10, one method is to use the estimated position by the infrastructure cameras 3a, 3b with the next smallest measurement error after the corresponding infrastructure cameras 3a, 3b as the final estimated position.

[0072] In addition, when the object determination unit 72 calculates "0" when the lower end portion of the object area is not included within the workable area, it may calculate information such as the extent to which the lower end portion of the object area extends beyond the workable area, and feed this back to the position information selection and determination unit 8 as a correction amount.

[0073] Then, the position information selection and determination unit 8, which has received feedback of this correction amount, may output an estimated position corrected by the above-mentioned correction amount when using the estimated position by the relevant infrastructure camera 3a, 3b as the final estimated position from the next frame onwards, or may use a method in which the estimated positions of each infrastructure camera 3a, 3b input to the position information selection and determination unit 8 are corrected using the above-mentioned correction amount over several frames, and then select which estimated position to use as the final estimated position.

[0074] According to the position measurement system of this embodiment, the system is configured to include a plurality of imaging means for photographing a self-position detection measurement object equipped with a self-position detection unit, and other measurement objects not equipped with a self-position detection unit, a self-position measurement means for determining the self-position of the self-position detection measurement object based on self-position information from the self-position detection unit, an image analysis means for estimating the positions of the self-position detection measurement object and the other measurement objects for each image captured by the plurality of imaging means, and a position determination means for comparing the self-position of the self-position detection measurement object measured by the self-position measurement means with the estimated position of the self-position detection measurement object for each image captured by the image analysis means, and selecting and determining one of the estimated positions of the other measurement objects for each image captured in accordance with the comparison result.

[0075] According to this, the position of a measurement object that does not have a self-position detection unit can be accurately obtained, so that the accuracy of position estimation can be improved.

[0076] Furthermore, as is clear from the above description, in this embodiment, even if the measurement accuracy of the infrastructure camera is reduced due to the influence of shadows from sunlight or obstruction by structures, the influence of camera parameters and calibration information with a common coordinate system, etc., highly accurate position information can be efficiently selected by selecting an image captured by the infrastructure camera. In addition, it is also possible to correct the measurement error of the estimated position output from the position information determination unit by using information on the workable area of ​​the map display device.

[0077] In the present embodiment, a case has been described in which an infrastructure camera is used to obtain the estimated position of a measurement object, but for example, LiDAR may also be used. EXAMPLES

[0078] Next, a second embodiment of the present invention will be described below with reference to Fig. 10, which shows a functional block diagram of the second embodiment.

[0079] A position measurement system 80 shown in FIG. 10 is based on the first embodiment, to which a new correction function has been added. The system is characterized by using information obtained from sensors mounted on the vehicle 2b or a map display device 4 that displays workable areas to select a highly accurate estimated position from among the estimated positions of the individual infrastructure cameras, and by correcting calibration information between the vehicle, infrastructure camera, and map display device as necessary.

[0080] In FIG. 10, a vehicle information acquisition unit 5, an image analysis unit 6, a measurement error estimation unit 7, a position information selection and determination unit 8, a work area presentation unit 9, and a work area inside / outside determination unit 10 have the same functions as in the first embodiment.

[0081] Furthermore, in this embodiment, a calibration accuracy determination unit 81 and a configuration execution unit 82 are newly added. The calibration accuracy determination unit 81 has a function of storing information output from the work area inside / outside determination unit 10 as to whether or not the measurement target exists within the workable area (the above-mentioned information of "1" or "0" from the work area inside / outside determination unit 10) for a certain period of time, and determining whether or not the calibration accuracy of any of the vehicle 2b, the infrastructure cameras 3a, 3b, and the map display device 4 is sufficient. In addition, the calibration execution unit 82 has a function of executing calibration again when the calibration accuracy determination unit 81 determines that the calibration accuracy is insufficient. The calibration accuracy determination unit 81 and the calibration execution unit 82 will be described below.

[0082] As a method for determining whether the calibration accuracy is sufficient by the calibration accuracy determination unit 81, for example, when there are no structures around the vehicle 2b and the reliability of the vehicle's position information from the GPS sensor is high, if the vehicle 2b is not always located within the workable area, it can be determined that the accuracy of the calibration information between the map display device 4 and the common coordinate system 16 is insufficient.

[0083] Similarly, in a situation where the reliability of the vehicle 2b's own position information is high, if the vehicle 2b is determined to be within the workable area but the worker 2a is not always present within the workable area, it can be determined that the accuracy of the calibration information between the infrastructure cameras 3a, 3b and the common coordinate system 16 is insufficient.

[0084] Furthermore, by employing a similar method, it is also possible to determine the accuracy of calibration information between the external sensor mounted on the vehicle 2b and the common coordinate system 16.

[0085] Furthermore, when the calibration accuracy determination unit 81 determines the calibration accuracy, it may adopt a method in which, without using the work area information from the map display device 4, if a measurement error always occurs between the vehicle's own position and the estimated position of the vehicle 2b by the infrastructure cameras 3a, 3b under conditions in which the reliability of the vehicle's own position information is high, the calibration accuracy between the infrastructure cameras 3a, 3b and the common coordinate system 16 is determined to be insufficient.

[0086] Next, when the calibration accuracy determination unit 81 determines that the calibration accuracy is insufficient, the calibration execution unit 82 automatically corrects the calibration information or notifies the user to perform recalibration. As an automatic calibration method, an image of the vehicle 2b capable of acquiring its own position information traveling within a work area within a range in which the reliability of the own position information does not decrease can be captured, and the position measurement system can be calibrated.

[0087] For example, when performing automatic calibration of the infrastructure cameras 3a, 3b and the common coordinate system 16, one method is to store information on the vehicle 2b's own position and information on the estimated position by the infrastructure cameras 3a, 3b at the same time, estimate camera parameters that minimize the error in the position information in the common coordinate system 16, and perform calibration.

[0088] In addition, when performing automatic calibration of the map display device 4 and the common coordinate system 16, there is a method of estimating the parameters of the angle of view and distortion of the map display device 4 so that the bottom end position of the object area of ​​the vehicle 2b falls within the midpoint of the divided area of ​​the workable area when performing an inside / outside determination of the workable area of ​​the vehicle 2b, and then performing calibration.

[0089] In this embodiment, using information obtained by sensors mounted on the vehicle or a map display device that displays the workable area, a highly accurate estimated position can be selected from the estimated positions of each infrastructure camera, and calibration information between the vehicle, infrastructure camera, and map display device can be corrected as necessary.

[0090] The present invention is not limited to the above-mentioned embodiments, but includes various modified examples. The above-mentioned embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace other configurations with respect to the configuration of each embodiment. [Explanation of symbols]

[0091] 1...position measurement system, 2a...worker (other measurement object), 2b...vehicle (measurement object for detecting own position), 3a, 3b...infrastructure camera, 4...map display device, 5...vehicle information acquisition unit, 6...image analysis unit, 7...measurement error estimation unit, 8...position information determination unit, 9...work area presentation unit, 10...work area inside / outside determination unit, 11...own position information acquisition unit, 12...external sensor information acquisition unit, 20...image acquisition unit, 21...object detection unit, 22...position measurement unit, 40...vehicle measurement error calculation unit, 41...worker measurement error estimation unit, 50...vehicle's own position, 51...estimated position of vehicle, 52...vehicle measurement error, 53...estimated position of worker, 54...worker measurement error, 60...divided area, 61...worker's workable area, 62...vehicle's workable area.

Claims

1. 1. A position measuring system including a position measuring unit for measuring a position of a measurement object from a captured image, The position measuring means includes: A plurality of imaging means for capturing images of a measurement target object having a self-position detection unit and other measurement targets not having a self-position detection unit; a self-position measuring means for determining a self-position of the self-position detection measurement object based on self-position information from the self-position detection unit; an image analysis means for determining an estimated position of the object to be measured for detecting the self position and the other object to be measured for each image captured by the plurality of imaging means; a position determining means for comparing the self-position of the self-position detection measurement object measured by the self-position measuring means with the estimated position of the self-position detection measurement object for each captured image obtained by the image analyzing means, and selecting and determining one of the estimated positions of the other measurement objects for each captured image in accordance with a result of this comparison. A position measurement system comprising:

2. 2. The position measurement system according to claim 1, The position determination means includes a self-position measurement error estimation means for comparing the self-position of the self-position detection measurement object obtained by the self-position measurement means with the estimated position of the self-position detection measurement object obtained by the image analysis means to calculate a self-position measurement error of the captured image. A position measurement system comprising:

3. 3. The position measurement system according to claim 2, The position determination means includes a measurement error estimation means for estimating a measurement error of the other measurement object using information on the self-position measurement error obtained by the self-position measurement error estimation means and information on a distance between the other measurement object and the imaging means. A position measurement system comprising:

4. 4. The position measurement system according to claim 3, The position determining means includes a position information selecting and determining means for selecting the other measurement object having the smallest error among the measurement errors of the other measurement objects of the plurality of captured images obtained by the measurement error estimating means for the measurement object, and determining the estimated position of the other measurement object. A position measurement system comprising:

5. 5. The position measurement system according to claim 4, The position measuring means includes: a map display means for dividing the work area into a plurality of divided regions and displaying the divided regions on a surface of the work area; a work area presenting means for presenting the divided area in which the self-position detection measurement object and the other measurement object can work as a workable area to the map display means based on the self-position of the self-position detection measurement object and the estimated positions of the other measurement objects, and displaying the divided area on the work area surface. A position measurement system comprising:

6. 6. The position measurement system according to claim 5, The position measuring means further includes a work area inside / outside determining means for determining whether the self-position detection measurement object or the measurement object is present in the workable area and feeding back the determination result to the position information selection determining means. A position measurement system comprising:

7. 7. The position measurement system according to claim 6, The position measuring means includes: a calibration accuracy determination means for determining a calibration accuracy between the object to be measured for self-position detection, the map display means, and the imaging means, using at least the determination information of the work area inside / outside determination means; A calibration execution means is provided for executing recalibration when the calibration accuracy is insufficient. A position measurement system comprising:

8. 2. The position measurement system according to claim 1, The object to be measured for detecting its own position includes a sensor for detecting its own position information indicating its current position and orientation, and a sensor for detecting shape information of structures and obstacles in the vicinity of the object to be measured for detecting its own position. A position measurement system comprising:

9. 2. The position measurement system according to claim 1, The image analysis means detects the self-position detection measurement object and the other measurement objects in the captured image to set a detection frame, and obtains the estimated positions of the self-position detection measurement object and the other measurement objects from the positions of the detection frame. A position measurement system comprising:

10. 2. The position measurement system according to claim 1, Furthermore, the position determination means selects the estimated position of the other measurement object using one or more pieces of information: image capture time information, information on the positional relationship between the sun and the image capture means, information on the attitude of the other measurement object, information on the density of the surroundings of the other measurement object, and information on obstruction of the other measurement object by an obstacle. A position measurement system comprising:

11. 1. A position measuring system including a position measuring unit for measuring a position of a measurement object from a captured image, The position measuring means includes: A plurality of imaging means for photographing a measurement target object having a self-position detection unit and another measurement target object not having a self-position detection unit; a self-position measuring means for determining a self-position of the self-position detection measurement object based on self-position information from the self-position detection unit; a measurement object position estimation means for determining an estimated position of the measurement object and the other measurement objects from the captured images captured by each of the imaging means; a measurement error estimation means for comparing the self-position of the self-position detection measurement object measured by the self-position measurement means with the estimated position of the self-position detection measurement object estimated by the measurement object position estimation means to obtain a measurement error, and estimating a measurement error of the other measurement object based on the measurement error; a position determining means for selecting the other measurement object having a smaller measurement error estimated by the measurement error estimating means and determining the estimated position of the other measurement object; A position measurement system comprising:

Images