Position estimation system, position estimation method, and program

Abstract

To provide a position estimation system, a position estimation method, and a program that can estimate the position of a moving object without installing an installation on the moving object.SOLUTION: A position estimation system 100 includes: a detection unit 55 that generates a first mask image to which is added a mask region where a moving object in an image captured by an imaging unit is masked; a perspective transformation unit 57 that perspective-transforms the first mask image; and a position calculation unit 58 that calculates a third coordinate point P3 by correcting a coordinate point P1 of a first bounding rectangle R1 set in a mask region M1 in a first mask image 15 using a second coordinate point P2 of the mask region M1 in a second mask image 17 obtained by perspective-transforming the first mask image 15.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a position estimation system, a position estimation method, and a program, and more particularly to a position estimation system, a position estimation method, and a program for a moving body such as a vehicle. [Background technology]

[0002] For example, Patent Document 1 discloses a technology for detecting the position of a vehicle.

[0003] Patent document 1 discloses a vehicle position detection device comprising: a transmitting / receiving means installed at two predetermined positions on a vehicle; four reply means installed at at least three fixed points in the parking space of the vehicle; and a control means connected to each of the transmitting / receiving means, wherein each of the transmitting / receiving means communicates with two different reply means to detect the distance between them, and the control means calculates the vehicle position based on the distance detected by each of the transmitting / receiving means. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2001-51720 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 requires installation of objects in both the vehicle and the parking space, and therefore has the problem that it is not possible to estimate the position of a moving object when, for example, the moving object or space in which installation objects cannot be installed is being targeted.

[0006] The present disclosure has been made to solve such problems, and aims to provide a position estimation system, a position estimation method, and a program that are capable of estimating the position of a moving body without installing an object on the moving body. [Means for solving the problem]

[0007] The position estimation system according to one embodiment includes an imaging unit that captures an image of an imaging area including a target moving object, and a Picture a detection unit that generates a first mask image to which a mask area that masks a moving object in the image is added; a perspective transformation unit that performs perspective transformation on the first mask image; and a position calculation unit that sets a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, sets a vertex indicating the same position as the first coordinate point among vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transformation of the first mask image as a second coordinate point, and calculates a third coordinate point that indicates the position of the moving object in the image coordinate system by correcting the first coordinate point using the second coordinate point. ,of Yes.

[0008] A position estimation method according to one embodiment includes a step of capturing an image of an imaging area including a moving object of interest by an imaging unit; Picture the first mask image is added with a mask area masking a moving object in the image; the first mask image is subjected to perspective transformation; a first coordinate point is a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image, a second coordinate point is a vertex of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transformation of the first mask image, the second coordinate point is a vertex indicating the same position as the first coordinate point, and the second coordinate point is used to correct the first coordinate point to calculate a third coordinate point indicating the position of the moving object in the image coordinate system. ,of Yes.

[0009] In addition, a program according to an embodiment is a program executed by a computer, the program comprising: Picturea process of performing perspective transformation on the first mask image, and a process of setting a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, setting a vertex indicating the same position as the first coordinate point among vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transformation of the first mask image as a second coordinate point, and correcting the first coordinate point using the second coordinate point to calculate a third coordinate point indicating the position of the moving object in the image coordinate system. ,of Yes. Effect of the Invention

[0010] The present disclosure makes it possible to provide a position estimation system, a position estimation method, and a program capable of estimating the position of a moving body without installing an object on the moving body. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an overview of a location estimation system according to a first embodiment. [Diagram 2] 1 is a flowchart illustrating an example of a position estimation method using the position estimation system. [Diagram 3] 1A to 1C are schematic diagrams showing examples of various images. [Figure 4] FIG. 11 is a first diagram illustrating step S8. [Diagram 5] FIG. 2 is a second diagram illustrating step S8. [Figure 6] FIG. 11 is a diagram illustrating step S9. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] First embodiment Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the present disclosure is not limited to the following embodiments. In addition, the following description and drawings are appropriately simplified for clarity of explanation.

[0013] Hereinafter, a specific example of a moving body whose position is estimated by the position estimation system 100 according to the present embodiment will be described, in which the moving body is a vehicle 1 such as an automobile, but the position estimation system 100 can be applied to position estimation of various moving bodies other than vehicles. The moving body may be, for example, a road-traveling robot.

[0014] In the following description, the direction parallel to the traveling direction of the vehicle 1 is referred to as the front-rear direction, the horizontal direction perpendicular to the front-rear direction is referred to as the left-right direction, and the direction perpendicular to the front-rear direction and the left-right direction is referred to as the up-down direction. Within the front-rear direction, the traveling direction of the vehicle 1 is referred to as the front, and the opposite direction is referred to as the rear. The black arrow in the figure indicates the traveling direction.

[0015] An overview of a position estimation system 100 according to the present embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of the position estimation system according to the first embodiment. As shown in Fig. 1, the position estimation system 100 includes a camera 10, a display device 30, and an information processing device 40, and estimates the position of a target vehicle 1.

[0016] The vehicle 1 moves (travels) along the traveling direction on a flat road surface 71 on which grid lines 70 are drawn in a mesh pattern along the Xg axis parallel to the traveling direction and the Yg axis perpendicular to the traveling direction. The Xg axis and the Yg axis are each a coordinate axis of the global coordinate system. Such grid lines 70 may be omitted.

[0017] The position estimation system 100 employs the position of a positioning point set for a specific portion of the vehicle 1 as the position of the vehicle 1. In this embodiment, the positioning point of the vehicle 1 is the left rear vehicle end 1e of the vehicle 1.

[0018] The camera 10 is a specific example of an imaging unit. The camera 10 is connected to the information processing device 40 wirelessly or by wire so as to be able to communicate with the information processing device 40. The camera 10 has an imaging element, such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor, and an optical system. In this embodiment, the camera 10 is fixed outside the vehicle 1 at a position where the camera 10 can capture an image of the vehicle 1 moving on a road surface 71 from above the left rear of the vehicle 1. The camera 10 captures an image of an imaging area including the vehicle 1 at a predetermined imaging period. In this embodiment, a case where the camera 10 captures an image of an imaging area including the vehicles 1 and 2 will be described as an example.

[0019] The camera 10 generates an image showing the imaging area. The image is a two-dimensional image formed by a collection of pixels arranged on the XcYc plane, and includes at least two-dimensional data of the vehicle 1. The image is preferably a color image, but may be a gray image. For example, every time the camera 10 generates an image, it outputs to the information processing device 40 the image acquisition time when the image was acquired together with the generated image.

[0020] The display device 30 displays various information to the user. The display device 30 is configured, for example, by a liquid crystal display (LCD) or an organic electroluminescence display (OLED). The display device 30 is connected to the information processing device 40 wirelessly or via wires so as to be able to communicate with the information processing device 40.

[0021] The information processing device 40 has a hardware configuration of a typical computer, for example, including a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory), storage devices such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), an input / output I / F for connecting peripheral devices such as the display device 30, and a communication I / F for communicating with devices outside the device.

[0022] The information processing device 40 processes the image captured by the camera 10 to estimate (calculate) the position of the vehicle 1. The information processing device 40 has an image input unit 51, a distortion correction unit 52, a rotation processing unit 53, a crop processing unit 54, a detection unit 55, a deletion unit 56, a perspective transformation unit 57, a position calculation unit 58, a position correction unit 59, and a storage unit 60. Image input unit 51, Distortion correction unit 52, Rotation processing unit 53, A cropping unit 54, a deletion unit 56, position correction section 59; And the storage unit 60 may be omitted as appropriate.

[0023] The image input unit 51 accepts input of an image from the camera 10. The distortion correction unit 52 corrects distortion of the image. The rotation processing unit 53 rotates the image so that the vector direction of the vehicle 1 faces a predetermined direction. When the vehicle 1 moves a distance exceeding a predetermined threshold, the crop processing unit deletes from the image a moved area corresponding to the distance moved by the vehicle 1, and cuts out an unmoved area including the moving object.

[0024] The detection unit 55 generates a first mask image 15 to which a mask region M1 that masks the vehicle 1 in the image is added. The perspective transformation unit 57 performs perspective transformation on the first mask image 15.

[0025] The position calculation unit 58 first acquires, as a first coordinate point P1, a specified vertex of a first circumscribing rectangle R1 set in a mask area M1 in the first mask image 15. In addition, the position calculation unit 58 acquires, as a second coordinate point P2, a vertex indicating the same position as the first coordinate point P1 among the vertices of a second circumscribing rectangle R2 set in a mask area M1 in a second mask image 17 obtained by perspective transforming the first mask image 15. Then, the position calculation unit 58 corrects the first coordinate point P1 using the second coordinate point P2 to calculate a third coordinate point P3 indicating the position of the vehicle 1 in the image coordinate system.

[0026] The position correction unit 59 calculates the position of the vehicle 1 in the global coordinate system by correcting the third coordinate point P3 by applying the position of the camera 10 in the global coordinate system.

[0027] The storage unit 60 is configured, for example, by a memory provided in the information processing device 40. The storage unit 60 stores various programs executed by the information processing device 40, various data such as images obtained from the camera 10, various parameters, and the like. The programs stored in the storage unit 60 include at least a part of a program in which the processing of the position estimation method according to the present embodiment is implemented. The storage unit 60 also stores a two-dimensional map of a global coordinate system including the image capture coordinate point Pc and the fixed coordinate point Pf used for position estimation, information indicating the height H of the camera 10, information indicating the height h of the vehicle end 1e, and further calculation results obtained during position estimation.

[0028] A user can estimate the position of the vehicle 1 by using the above-described position estimation system 100. A position estimation method using the position estimation system 100 will now be described with reference to Figs. 2 and 3. Fig. 2 is a flow chart for explaining an example of a position estimation method using the position estimation system. Fig. 3 is a schematic diagram showing examples of various images. In the following description, images obtained by each process before the detection unit 55 adds the mask region M1 are referred to as images 11 to 14, respectively.

[0029] First, in step S1, the image input unit 51 accepts input of the image 11 from the camera 10. The image 11 acquired from the camera 10 may include two-dimensional data of a plurality of vehicles including the vehicle 1. The image 11 illustrated in FIG. 2 includes two-dimensional data of each of the vehicles 1 and 2. If such processing by the image input unit 51 is not required, step S1 can be omitted.

[0030] In step S2, the distortion correction unit 52 corrects the distortion of the image 11. In this step, it is possible to generate the image 12 in which the distortion of the image 11 has been corrected using the distortion correction parameters stored in advance in the storage unit 60. The distortion correction parameters are parameters relating to the position information of the grid lines 70 obtained by calibration, for example. Note that any parameters other than the grid lines 70 may be adopted. If such processing by the distortion correction unit 52 is not required, step S2 can be omitted.

[0031] In step S3, the rotation processing unit 53 rotates the image 12 so that the vector direction of the vehicle 1 faces a predetermined direction. In this step, for example, an image 13 is generated by rotating the image 12 around the center of gravity of the vehicle 1 in the image 12 as the center of rotation so that the vector direction, which is the traveling direction of the vehicle 1, faces upward on the screen of the display device 30 that displays the image 12. If such processing by the rotation processing unit 53 is not required, step S3 can be omitted.

[0032] Here, for example, the optical flow method or the like can be used to express the movement of the characteristic point of the vehicle 1 as a vector. The vector quantity and vector direction of the vehicle 1 are estimated based on the change in position of the characteristic point (center of gravity C) of the vehicle 1, which is appropriately set on the image 11, between image frames.

[0033] In step S4, when the vehicle 1 has moved a distance exceeding a predetermined threshold, the cropping processing unit 54 deletes the moved area corresponding to the distance moved by the vehicle 1 from the image 13, and generates an image 14 in which the unmoved area including the vehicle 1 is cut out. The cropping processing unit 54 generates an image 14 by using the estimated vector quantity. Car Both 1s know the distance traveled.

[0034] 3 is the area indicated by the dashed line in the image 13. This improves the accuracy of image processing for the vehicle 1 that is far away from the camera 10. If such processing by the crop processing unit 54 is not required, step S4 can be omitted. Also, the processing by the crop processing unit 54 in step S4 may be performed before the processing by the rotation processing unit 53 in step S3.

[0035] In step S5, the detection unit 55 generates a first mask image 15 to which a mask region M1 that masks the vehicle 1 in the image 14 is added. In this step, the vehicle 1 in the image 14 is masked using a trained model. For example, by inputting the image 14 to a classifier that has been trained in advance by machine learning to mask vehicles in an input image, it is possible to output a first mask image 15 to which a mask region M1 indicating the region of the vehicle 1 is added to the image 14.

[0036] As such a classifier, a DNN (Deep Neural Network) having a CNN (Convolutional Neural Network) structure that realizes semantic segmentation and instance segmentation can be used. Note that the classifier may be a neural network for object detection, or may be a classifier according to a machine learning model other than a neural network.

[0037] An example of such a classifier is a DNN that performs instance segmentation, such as YOLACT++. When such a DNN is used, it is possible to classify a plurality of vehicles included in the image 14 and generate a first mask image 15 in which each of the vehicles 1 and 2 is masked. In this embodiment, the first mask image 15 is generated by adding mask areas M1 and M2 indicating the areas of the masked vehicles 1 and 2 to the image 14. This makes it possible to suppress a decrease in position estimation accuracy due to the diversity of the background, and also makes it possible to select the target vehicle 1 when the image 14 includes a plurality of vehicles.

[0038] The first mask image 15 may be a binarized image obtained by adding mask regions M1 and M2 to the image 14 and then performing binarization processing. Note that in step S5, the image 13 may be treated as the processing target instead of the image 14, and the first mask image 15 may be generated by adding the mask regions M1 and M2 to the image 13.

[0039] In step S6, when the first mask image 15 includes a plurality of vehicles, the deletion unit 56 deletes the masked non-target vehicles 2 from the first mask image 15. In this step, of the mask regions M1 and M2 generated in step S5, the deletion unit 56 deletes the mask region M2 that exists outside the recognition target region from the first mask image 15. The recognition target region is a predetermined region in which the vehicle 1 moves in the first mask image 15, and is stored in advance in the storage unit 60. For example, the recognition target region corresponding to the region within the grid lines 70 is set.

[0040] This makes it possible to eliminate the influence of vehicle 2, which is not a target for position estimation, when multiple vehicles are captured, thereby improving the accuracy of position estimation of vehicle 1. Note that, when the process by deletion unit 56 is not required, such as when vehicle 2 does not exist, step S6 may be omitted.

[0041] In step S7, the perspective transformation unit 57 performs perspective transformation on the first mask image 15. In step S7, the first mask image 15 generated in step S5 may be processed, or the first mask image 15 from which the vehicle 2 has been deleted in step S6 may be processed.

[0042] In this step, using perspective transformation parameters pre-stored in storage unit 60, first mask image 15 is perspective transformed into a bird's-eye image seen from a viewpoint above vehicle 1 (for example, directly above vehicle 1) approximately perpendicular to road surface 71.

[0043] As a result, the camera coordinate system having coordinate axes indicated by Xc and Yc with the focal point of the camera 10 as its origin is transformed into an image coordinate system having coordinate axes indicated by Xi and Yi with a point on the image plane projected by the perspective transformation as its origin. The perspective transformation parameters are parameters related to the position information and internal parameters of the camera 10 obtained by, for example, calibration.

[0044] In step S8, the position calculation unit 58 first acquires, as a first coordinate point P1, a specified vertex of a first circumscribing rectangle R1 set in a mask area M1 in the first mask image 15. In addition, the position calculation unit 58 acquires, as a second coordinate point P2, a vertex indicating the same position as the first coordinate point P1 among the vertices of a second circumscribing rectangle R2 set in a mask area M1 in a second mask image 17 obtained by perspective transforming the first mask image 15. Then, the position calculation unit 58 corrects the first coordinate point P1 using the second coordinate point P2 to calculate a third coordinate point P3 indicating the position (estimated position) of the vehicle 1 in the image coordinate system.

[0045] Step S8 will now be described in detail with reference to Figures 4 and 5. Figure 4 is a first diagram illustrating step S8. Figure 5 is a second diagram illustrating step S8. As shown in Figure 4, in step S8, a third coordinate point P3 used to calculate the position of the vehicle 1 is calculated.

[0046] To calculate the third coordinate point P3, the first mask image 15 to which the first circumscribing rectangle R1 set for the mask area M1 in the first mask image 15 is added is subjected to perspective transformation to obtain a first coordinate point P1 of the first circumscribing rectangle R1 deformed by the perspective transformation. In addition, a second coordinate point P2 of the second circumscribing rectangle R2 set for the mask area M1 in the second mask image 17 is obtained.

[0047] The third coordinate point P3 is calculated using the coordinates (Xi1, Yi1), (Xi2, Yi2) of the first coordinate point P1 and the second coordinate point P2 acquired in this manner. The third coordinate point P3 can be calculated by replacing the coordinates (Xi1, Yi1) of the first coordinate point P1 with the coordinates (Xi2, Yi2) of the second coordinate point P2 as necessary. Specifically, if Xi1>Xi2, Xi1 is replaced with Xi2. Also, if Yi1>Yi2, Yi1 is replaced with Yi2. The coordinate point P3 illustrated in FIG. 4 has coordinates (Xi2, Yi1). Such a correction improves the accuracy of estimating the position of the vehicle 1.

[0048] Here, a method for acquiring the first coordinate point P1 and the second coordinate point P2 will be described. First, in order to acquire the first coordinate point P1, a base coordinate point P0 is acquired from the first circumscribing rectangle R1 before the perspective transformation. FIG. 5 shows a method for acquiring the base coordinate point P0.

[0049] 5, first, a circumscribing rectangle R0 is set for the mask area M1 in the first mask image 15. Next, the first mask image 15 is rotated by a required amount around the center of gravity C of the mask area M1 as the rotation center so that the vector direction of the movement vector V of the vehicle 1 corresponding to the mask area M1 in the first mask image 15 faces a predetermined direction (upward on the screen).

[0050] After setting a first circumscribing rectangle R1 (first circumscribing rectangle R1 whose long side is parallel to the vector direction of the movement vector V) for the mask area M1 in the first mask image 15 rotated in this manner, the first mask image 15 to which the first circumscribing rectangle R1 has been added is rotated in the reverse direction by the above-mentioned rotation amount, with the center of gravity C of the mask area M1 as the rotation center. A base coordinate point P0 of the first circumscribing rectangle R1 in the first mask image 15 rotated in the reverse direction in this manner is obtained. The base coordinate point P0 obtained here is one of the four vertices of the first circumscribing rectangle R1, and the vertex having the coordinates closest to the vehicle edge 1e is selected.

[0051] 4, perspective transformation is performed on the first mask image 15 after reverse rotation to obtain the coordinates (Xi1, Yi1) of the first coordinate point P1 of the first circumscribing rectangle R1 transformed by the perspective transformation. The first coordinate point P1 obtained here is a coordinate point corresponding to the base coordinate point P0.

[0052] 4, in order to obtain the second coordinate point P2, a second circumscribing rectangle R2 is set for the mask region M1 in the second mask image 17, and the coordinates (Xi2, Yi2) of the second coordinate point P2 of the second circumscribing rectangle R2 are obtained. The second coordinate point P2 obtained here is one of the four vertices of the second circumscribing rectangle R2, and the vertex having the coordinates that are closest to the vehicle edge 1e is selected. That is, the first coordinate point P1 and the second coordinate point P2 are coordinate points that indicate the same position, and therefore have a correlation with each other.

[0053] In step S9, the position correction unit 59 calculates the position of the vehicle 1 in the global coordinate system by correcting the third coordinate point P3 by applying the position of the camera 10 in the global coordinate system. Details of step S9 will now be described with reference to Fig. 6. Fig. 6 is a diagram for explaining step S9.

[0054] The global coordinate system shown in Fig. 6 is a coordinate system having coordinate axes indicated by Xg and Yg with a fixed coordinate point Pf indicating an arbitrary reference position on the road surface 71 as the origin. An imaging coordinate point Pc indicates the position of the camera 10 in the global coordinate system. The fixed coordinate point Pf and the imaging coordinate point Pc are stored in advance in the storage unit 60. In this step, a vehicle coordinate point Pv indicating the position of the vehicle 1 is calculated.

[0055] As shown in the left side view of Figure 6, if the observation distance between the position of camera 10 and the position of vehicle 1 is Do, the observation error is ΔD, the height [m] of camera 10 from road surface 71 is H, and the height [m] of vehicle end 1e from road surface 71 is h, then the observation error ΔD can be expressed by the following equation (1). ΔD=h / H×Do...Equation (1) That is, the greater the observation distance Do, the greater the observation error ΔD. Note that the height H of the camera 10 and the height h of the vehicle end 1e are stored in the storage unit 60 in advance.

[0056] Next, if the actual distance between the position of the camera 10 and the position of the vehicle end 1e is D, the actual distance D can be expressed by the following formula (2). D=Do×(1-h / H)...Equation (2) That is, the actual distance D is determined by the observation distance Do, the height H of the camera 10, and the height h of the vehicle end 1e.

[0057] Then, as shown in the top view of Figure 6, if the estimated distance between the reference position and the estimated position of vehicle 1 is Dp and the actual distance between the reference position and the position of vehicle 1 is Dt, the actual distance Dt can be expressed by the following equation (3). Dt = Dp × (1-h / H) Equation (3) Here, the estimated distance Dp can be calculated using the actual distance Dc obtained from the coordinate points Pf and Pc, and the coordinate points P3 and Pf.

[0058] Therefore, the vehicle coordinate point Pv can be calculated using the actual distance Dt obtained by correcting the estimated distance Dp using the formula (3) and the fixed coordinate point Pf. The vehicle coordinate point Pv calculated in this manner corresponds to the position of the vehicle 1 in the real space. However, since the height H increases as the installation position of camera 10 is increased, the correction value indicated by h / H in formula (3) can be decreased so as to approach zero. In this way, when the processing by position correction unit 59 is not required, such as when camera 10 is installed at such a high position that correction is not necessary, step S9 may be omitted.

[0059] As described above, the position estimation system 100 according to this embodiment includes an imaging unit (camera 10) that captures an image of an imaging area including a target moving object (vehicle 1), a rotation processing unit 53 that rotates the image captured by the imaging unit so that the vector direction of the moving object faces a predetermined direction, a detection unit 55 that generates a first mask image 15 to which a mask area M1 that masks the moving object in the image is added, a perspective transformation unit 57 that performs perspective transformation on the first mask image 15, and a perspective transformation unit 58 that transforms designated vertices of a first circumscribed rectangle R1 set in the mask area M1 in the first mask image 15. The image processing device includes a position calculation unit 58 that calculates a third coordinate point P3 indicating the position of the moving object in the image coordinate system by correcting the first coordinate point P1 using the second coordinate point P2, and a position correction unit 59 that calculates the position of the moving object in the global coordinate system by correcting the third coordinate point P3 by applying the position of the imaging unit in the global coordinate system. The position calculation unit 58 determines the first coordinate point P1 as a first coordinate point, and the second coordinate point P2 as a vertex of a second circumscribed rectangle R2 set in a mask area M1 in a second mask image 17 obtained by perspective transforming the first mask image 15, which indicates the same position as the first coordinate point P1.

[0060] According to such a configuration, it is possible to provide a system capable of estimating the position of a moving body without installing on the moving body any installation objects such as sensors, markers, transceivers, etc., used for estimating the position of the moving body. In addition, it is not necessary to mount the position estimation system 100 on the moving body, and it is more versatile than other systems that require the moving body to be equipped with at least some functions for storing and processing data.

[0061] Moreover, the position estimation system 100 further includes a distortion correction unit 52 that corrects image distortion. With this configuration, the accuracy of estimating the position of a moving object is improved.

[0062] Moreover, the position estimation system 100 further includes a cropping processor 54 that, when the moving object moves a distance exceeding a predetermined threshold, deletes from the image a moved area corresponding to the distance moved by the moving object, and cuts out an unmoved area including the moving object. With this configuration, the accuracy of estimating the position of the moving object is improved.

[0063] Moreover, the position estimation system 100 further includes a deletion unit 56 that, when a plurality of moving objects are included in the first mask image 15, deletes the non-target moving objects masked by the detection unit 55 from the first mask image 15. With this configuration, the accuracy of estimating the position of the moving object is improved.

[0064] The moving body is a vehicle 1. Here, as a system capable of measuring the position of a moving body such as a vehicle in an outdoor space, for example, a system using a GPS (Global Positioning System) can be used. On the other hand, in an indoor space where there is a shielding effect and data from a GPS satellite is difficult to reach, there is a problem that the accuracy of the position calculated by the system using the GPS decreases. In contrast, according to this embodiment, the position of the vehicle 1 can be estimated even in an environment where the use of GPS is undesirable. Therefore, the position estimation system 100 described above can be suitably used for estimating the position of the vehicle 1 that is present in an indoor space such as a vehicle factory.

[0065] Furthermore, the position estimation method according to this embodiment includes the steps of capturing an image of an imaging area including a target moving object by an imaging unit, rotating the image captured by the imaging unit so that the vector direction of the moving object faces a predetermined direction, generating a first mask image 15 to which a mask area M1 that masks the moving object in the image is added, performing perspective transformation on the first mask image 15, and performing perspective transformation on the first mask image 15. the first coordinate point P1 being a first coordinate point P1, the second coordinate point P2 being a vertex of a second circumscribing rectangle R2 set in a mask area M1 in a second mask image 17 obtained by perspective transforming the first mask image 15, and correcting the first coordinate point P1 using the second coordinate point P2 to calculate a third coordinate point P3 indicating the position of the moving body in the image coordinate system; and calculating the position of the moving body in the global coordinate system by correcting the third coordinate point P3 by applying the position of the imaging unit in the global coordinate system.

[0066] According to this configuration, it is possible to provide a method capable of estimating the position of a moving body without installing any installation objects such as sensors, markers, transceivers, etc., on the moving body for use in estimating the position of the moving body.

[0067] Furthermore, the program according to this embodiment is a program to be executed by a computer, and includes a process of rotating an image captured of an imaging area including a target moving object so that the vector direction of the moving object faces a predetermined direction; a process of generating a first mask image 15 to which a mask area M1 that masks the moving object in the image is added; a process of perspective transforming the first mask image 15; a process of perspective transforming the first mask image 15; a process of setting a specified vertex of a first circumscribing rectangle R1 set in the mask area M1 in the first mask image 15 as a first coordinate point P1, setting a vertex that indicates the same position as the first coordinate point P1 among the vertices of a second circumscribing rectangle R2 set in the mask area M1 in a second mask image 17 obtained by perspective transforming the first mask image 15 as a second coordinate point P2, and correcting the first coordinate point P1 using the second coordinate point P2 to calculate a third coordinate point P3 that indicates the position of the moving object in the image coordinate system; and a process of calculating the position of the moving object in the global coordinate system by correcting the third coordinate point P3 by applying the position of the imaging unit in the global coordinate system.

[0068] According to this configuration, it is possible to provide a program capable of estimating the position of a moving body without installing any installation objects, such as sensors, markers, transceivers, etc., used for estimating the position of the moving body on the moving body.

[0069] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present disclosure.

[0070] For example, in the above embodiment, an example of estimating the position of a moving vehicle 1 has been described, but it is also possible to estimate the position of a stationary vehicle 1. When estimating the position of a stationary vehicle 1, for example, the system start position may be used instead of the vector direction of the moving vehicle 1. motionThereafter, the position of the stationary vehicle 1 can be estimated using the initial vector direction of the vehicle 1 estimated from the first acquired image 11 .

[0071] In the above embodiment, an example has been described in which the information processing device 40 processes two-dimensional images, but the information processing device 40 may process three-dimensional images captured by a stereo camera or a TOF (Time Of Flight) camera.

[0072] Furthermore, the information processing device 40 may be realized by a device contained in one housing, or each unit of the information processing device 40 may be realized by cloud computing configured by one or more computers, for example.

[0073] In addition, in the above embodiment, an example was given in which the image input unit 51 accepts input of the image 11 from the camera 10, but it may also be possible to accept input of an image 11 that has been stored in advance in, for example, a memory unit 60.

[0074] In addition, the present disclosure can be realized by, for example, making a processor execute a computer program to perform the processing performed by the information processing device 40. The program can be stored using various types of non-transitory computer readable media and supplied to the computer. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs). The program may be supplied to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can supply the program to the computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths. [Explanation of symbols]

[0075] 1, 2 Vehicle 1e Vehicle end 10 Camera 11, 12, 13, 14 Images 15 First mask image 17 Second mask image 20 Input device 30 Display device 40 Information processing device 51 Image input unit 52 Distortion correction unit 53 Rotation processing unit 54 crop processing section 55 detection section 56 deletion section 57 perspective transformation section 58 Position calculation section 59 Position correction section 60 Storage section 70 Grid line 71 Road surface 100 Position estimation system C Center of gravity M1, M2 Mask area P0 Base coordinate point P1 First coordinate point P2 Second coordinate point P3 Third coordinate point Pf Fixed coordinate point Pc Image coordinate point Pv Vehicle coordinate point R0 Circumscribing rectangle R1 First circumscribing rectangle R2 Second bounding rectangle V Movement vector

Claims

1. An imaging unit that captures an image of an imaging area including a target moving object; a detection unit that generates a first mask image to which a mask region that masks the moving object in the image captured by the imaging unit is added; a perspective transformation unit that performs perspective transformation on the first mask image; a position calculation unit which calculates a third coordinate point indicating a position of the moving object in an image coordinate system by setting a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, setting a vertex indicating the same position as the first coordinate point among the vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transforming the first mask image as a second coordinate point, and replacing the X coordinate of the first coordinate point with the X coordinate of the second coordinate point if an X coordinate of the first coordinate point is greater than an X coordinate of the second coordinate point, and replacing the Y coordinate of the first coordinate point with the Y coordinate of the second coordinate point if a Y coordinate of the first coordinate point is greater than an Y coordinate of the second coordinate point; A position estimation system having:

2. The position estimation system according to claim 1 , further comprising a distortion correction unit that corrects distortion of the image.

3. An imaging unit that captures an image of an imaging area including a target moving object; a rotation processing unit that rotates the image captured by the imaging unit so that the vector direction of the moving object faces a predetermined direction; a detection unit that generates a first mask image to which a mask region that masks the moving object in the image is added; a perspective transformation unit that performs perspective transformation on the first mask image; a position calculation unit that sets a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, sets a vertex indicating the same position as the first coordinate point among the vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transforming the first mask image as a second coordinate point, and corrects the first coordinate point using the second coordinate point to calculate a third coordinate point indicating the position of the moving object in an image coordinate system; A position estimation system having:

4. An imaging unit that captures an image of an imaging area including a target moving object; a cropping processing unit that, when the moving object in the image captured by the imaging unit has moved a distance exceeding a predetermined threshold, deletes a moved area corresponding to the distance moved by the moving object from the image, and cuts out an unmoved area including the moving object; a detection unit that generates a first mask image to which a mask region that masks the moving object in the image is added; a perspective transformation unit that performs perspective transformation on the first mask image; a position calculation unit that sets a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, sets a vertex indicating the same position as the first coordinate point among the vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transforming the first mask image as a second coordinate point, and corrects the first coordinate point using the second coordinate point to calculate a third coordinate point indicating the position of the moving object in an image coordinate system; A position estimation system having:

5. The position estimation system according to claim 1 , further comprising a deletion unit that, when the first mask image includes a plurality of moving objects, deletes non-target moving objects masked by the detection unit from the first mask image.

6. The position estimation system according to claim 1 , further comprising a position correction unit that calculates the position of the moving object in the global coordinate system by correcting the third coordinate point by applying a position of the imaging unit in a global coordinate system.

7. The position estimation system according to claim 1 , wherein the moving body is a vehicle.

8. A position estimation method executed by a computer, comprising: capturing an image of an imaging region including a target moving object by an imaging unit; generating a first mask image to which a mask region that masks the moving object in the image captured by the imaging unit is added; Perspective transforming the first mask image; a step of setting a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, setting a vertex indicating the same position as the first coordinate point among the vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transforming the first mask image as a second coordinate point, and if an X coordinate of the first coordinate point is greater than an X coordinate of the second coordinate point, replacing the X coordinate of the first coordinate point with the X coordinate of the second coordinate point, and if a Y coordinate of the first coordinate point is greater than an Y coordinate of the second coordinate point, replacing the Y coordinate of the first coordinate point with the Y coordinate of the second coordinate point, thereby calculating a third coordinate point indicating a position of the moving object in an image coordinate system; A location estimation method comprising:

9. A program executed by a computer, A process of generating a first mask image in which a mask area is added by masking the moving object in an image captured in an imaging area including the moving object; A process of performing a perspective transformation on the first mask image; a process of calculating a third coordinate point indicating a position of the moving object in an image coordinate system by setting a specified vertex of a first circumscribing rectangle set in the mask area in the first mask image as a first coordinate point, setting a vertex indicating the same position as the first coordinate point among the vertices of a second circumscribing rectangle set in the mask area in a second mask image obtained by perspective transforming the first mask image as a second coordinate point, and replacing the X coordinate of the first coordinate point with the X coordinate of the second coordinate point if an X coordinate of the first coordinate point is greater than an X coordinate of the second coordinate point, and replacing the Y coordinate of the first coordinate point with the Y coordinate of the second coordinate point if a Y coordinate of the first coordinate point is greater than an Y coordinate of the second coordinate point; A program having:

Images