Positioning device, moving body, and positioning method

Abstract

This positioning device comprises: a reception unit that receives image data from a camera mounted on a moving body; a marker-positioning unit that calculates the position of the moving body and the orientation of the camera on the basis of a marker image included in the image data; a feature-point-positioning unit that, when the marker image is not included in the image data, calculates the position of the moving body and the orientation of the camera on the basis of the past position of the moving body and a feature point of the image data; a calculation unit that calculates a movement vector of the moving body on the basis of the position of the moving body; and a determination unit that, on the basis of the orientation of the camera and the movement vector, determines whether the moving body is advancing or retreating.

Description

Positioning device, mobile body, and positioning method

[0001] The present disclosure relates to a positioning device, a mobile object, and a positioning method.

[0002] There are mobile objects that transport luggage indoors, such as in a logistics warehouse, while determining their position. Patent Literature 1 discloses a luggage location management device that determines the location of a vehicle using positioning technologies such as a global positioning system (GPS), a wireless local area network (LAN) positioning, and infrared positioning.

[0003] Non-Patent Document 1 discloses a positioning method called Visual-SLAM (Visual Simultaneous Localization and Mapping). In Visual-SLAM, a mobile object equipped with a camera moves while capturing images of its surroundings, and the amount of movement of the mobile object is calculated based on the amount of movement of feature points in multiple captured images. This makes it possible for Visual-SLAM to estimate the current position of the mobile object and generate a map based on the trajectory of the mobile object. Calculating the position, direction, etc. of a mobile object based on multiple captured images is called visual odometry. As a function of this visual odometry, the amount of movement of a mobile object may be calculated based on the amount of movement of feature points in multiple captured images.

[0004] Also known as a positioning method is a positioning method using markers. For example, a positioning device stores image data of markers attached (sticked) to objects such as pillars or walls of a building, and map information including the installation positions of the markers. The marker information and the installation positions are associated with each other.

[0005] The positioning device calculates the relative position of the moving object with respect to the marker based on the marker image captured by the camera. The positioning device refers to map information to acquire the installation position of the marker captured by the camera, and calculates the position of the moving object on the map of the map information based on the acquired installation position and the calculated relative position of the moving object with respect to the marker.

[0006] JP 2011-219229 A International Publication No. 2020 / 137315

[0007] R. Mur-Artal, et al., "ORB-SLAM2: an Open-Source SLAM Systemfor Monocular, Stereo and RGB-D Cameras", IEEE Transactions on Robotics, Volume: 33, Issue: 5, Oct. 2017

[0008] In conventional positioning methods, the position (movement trajectory) of a moving object on a map of map information is obtained, but it is not possible to know whether the moving object is moving forward or backward.

[0009] Non-limiting examples of the present disclosure contribute to providing a positioning device, a moving body, and a positioning method that determine the forward and backward movement of a moving body in positioning the moving body.

[0010] A positioning device according to one embodiment of the present disclosure includes a receiving unit that receives image data from a camera mounted on a moving body; a marker positioning unit that calculates the position of the moving body and the attitude of the camera based on a marker image included in the image data; a feature point positioning unit that, if the marker image is not included in the image data, calculates the position of the moving body and the attitude of the camera based on a past position of the moving body and feature points of the image data; a calculation unit that calculates a movement vector of the moving body based on the position of the moving body; and a determination unit that determines whether the moving body is moving forward or backward based on the attitude of the camera and the movement vector.

[0011] A moving body according to one embodiment of the present disclosure includes a positioning device having a camera, a receiving unit that receives image data from the camera, a marker positioning unit that calculates the position of the moving body and the attitude of the camera based on a marker image included in the image data, a feature point positioning unit that calculates the position of the moving body and the attitude of the camera based on a past position of the moving body and feature points of the image data if the marker image is not included in the image data, a calculation unit that calculates a movement vector of the moving body based on the position of the moving body, and a determination unit that determines whether the moving body is moving forward or backward based on the attitude of the camera and the movement vector.

[0012] In a positioning method according to one embodiment of the present disclosure, a positioning device receives image data from a camera mounted on a moving body, calculates the position of the moving body and the attitude of the camera based on a marker image included in the image data, and if the marker image is not included in the image data, calculates the position of the moving body and the attitude of the camera based on the past position of the moving body and feature points of the image data, calculates a movement vector of the moving body based on the position of the moving body, and determines the forward and backward movement of the moving body based on the attitude of the camera and the movement vector.

[0013] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0014] According to an embodiment of the present disclosure, forward and backward movement can be determined in positioning of a moving body.

[0015] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features.

[0016] FIG. 1 is a diagram showing a moving body equipped with a positioning device according to an embodiment of the present disclosure; FIG. 2 is a diagram showing an area in which the moving body moves, viewed from above; FIG. 3 is a diagram explaining marker positioning; FIG. 4 is a diagram explaining feature point positioning; FIG. 5 is a diagram explaining the principle of determining whether a moving body is moving forward; FIG. 6 is a diagram explaining the principle of determining whether a moving body is moving backward;

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or redundant explanation of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0018] The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0019] 1 is a diagram showing a moving object 2 equipped with a positioning device 1 according to an embodiment of the present disclosure. The moving object 2 may be a vehicle such as a forklift or a truck. The moving object 2 is equipped with the positioning device 1 and a camera 3.

[0020] The positioning device 1 receives image data of an image captured by the camera 3. Based on the image data received from the camera 3, the positioning device 1 measures the position of the moving body 2 by positioning based on image recognition of markers (images of the markers) attached to objects and by positioning based on the amount of movement of feature points in the image, such as visual odometry. That is, the positioning device 1 measures the position of the moving body 2 using two positioning methods (see, for example, Patent Document 2). Note that the position of the positioning device 1 may be regarded as the position of the moving body 2. That is, the positioning device 1 may measure the position of the positioning device 1 and use it as the position of the moving body 2.

[0021] Hereinafter, positioning based on image recognition of markers may be referred to as marker positioning, and positioning based on the amount of movement of feature points within an image may be referred to as feature point positioning. Positioning based on both marker positioning and feature point positioning may be referred to as a hybrid positioning method.

[0022] The camera 3 is, for example, a stereo camera. The camera 3 may be a compound eye camera, as long as it outputs image data that enables the positioning device 1 to calculate the distance to an object. The camera 3 is mounted on the moving body 2 so that the shooting direction (forward or optical axis) of the camera 3 faces forward (forward direction) of the moving body 2.

[0023] FIG. 2 is a diagram showing an area A2a in which the moving object 2 moves, as viewed from above. The square and circle shapes shown in the area A2a represent objects such as luggage stored in a building such as a logistics warehouse, pillars of the building, and walls of the building. Markers A2b to A2e are affixed to the objects as shown in FIG. 2. Note that in FIG. 2, the markers A2b to A2e are drawn on the top surface of the object for ease of understanding, but the markers A2b to A2e may also be affixed to the side of the object so that the camera 3 can easily capture the object. Furthermore, the positions of the objects and markers A2b to A2e shown in FIG. 2 are merely examples and are not limited to the example in FIG. 2.

[0024] Map information (map data) of the area A2a is stored in the positioning device 1. The map information includes marker information such as image data of the markers A2b to A2e and position information indicating the attachment positions of the markers A2b to A2e. The marker information and the attachment positions are associated with each other.

[0025] The map information includes, for example, two-dimensional coordinates, such as x and y coordinates with the origin at the bottom left of the area A2a shown in Fig. 3 (see arrow A2f in Fig. 2).

[0026] The area A2a in which the moving object 2 moves may be indoors or outdoors. The term "attaching" may be rephrased as "installing."

[0027] Marker Positioning Fig. 3 is a diagram for explaining marker positioning. Fig. 3 shows an image A3a captured by the camera 3. The image A3a includes a marker A3b.

[0028] The positioning device 1 receives an image A3a captured by the camera 3. The positioning device 1 calculates the relative position of the moving object 2 with respect to the marker A3b based on the appearance of the marker A3b, such as the angle (tilt), shape, and size of the image of the marker A3b included in the received image A3a. For example, the positioning device 1 calculates the relative position, such as the distance and direction, of the moving object 2 with respect to the marker A3b. Based on the calculated relative position and the position information of the marker A3b included in the map information, the positioning device 1 calculates the position (absolute position) of the moving object 2 on the map of the map information and the attitude of the camera 3 on the map of the map information. The attitude of the camera 3 is, for example, the yaw angle of the camera 3 with respect to the x-axis.

[0029] Feature Point Positioning and Hybrid Positioning There may be cases where the marker A3b is not present in the location within the area A2a where the moving object 2 is moving or in the direction of travel (the shooting direction of the camera 3) of the moving object 2. In this case, the positioning device 1 measures the position of the moving object 2 by feature point positioning.

[0030] Fig. 4 is a diagram illustrating feature point positioning. Fig. 4 shows an image A4a captured by the camera 3. The black squares shown in the image A4a indicate feature points calculated by the positioning device 1. Note that the image A4a does not include markers.

[0031] The positioning device 1 receives an image A4a captured by the camera 3. If the received image A4a does not include a marker image, the positioning device 1 calculates feature points for each frame (or n frames, where n is a positive integer) and calculates the relative movement amount, such as the movement distance and movement direction (rotation angle) of the moving object 2, from the difference (deviation) in the positions of the feature points for each frame.

[0032] For example, the positioning device 1 calculates edges, such as corners of an object, where the shape changes and where the color changes, as feature points from the image A4a. The positioning device 1 calculates the feature points for each frame. The positioning device 1 calculates the relative movement amount of the moving object 2 from the difference in the positions of the feature points for each frame.

[0033] The calculation of the feature points is not limited to the above example. The feature points may be calculated using existing techniques. The feature points may also be referred to as feature amounts.

[0034] After calculating the amount of relative movement, the positioning device 1 calculates the position and orientation of the moving object 2 within the area A2a based on the calculated amount of relative movement and the positioning result of the marker positioning. For example, the positioning device 1 adds the amount of relative movement to the positioning result of the marker positioning to calculate the position (absolute position) of the moving object 2 on the map of the map information and the yaw angle of the camera 3 on the map of the map information.

[0035] The positioning device 1 performs marker positioning when the marker is again included in the image received from the camera 3. When the marker is no longer included in the image received from the camera 3, the positioning device 1 performs feature point positioning based on the position of the moving object 2 measured by marker positioning.

[0036] In this way, the positioning device 1 performs marker positioning when the marker is photographed by the camera 3, and performs feature point positioning when the marker is not photographed by the camera 3. As a result, in the hybrid positioning method, even if the positioning result of the feature point positioning contains an error, the error is corrected by the positioning result of the marker positioning.

[0037] Furthermore, compared to the Visual-SLAM positioning method, the hybrid positioning method has the advantage that it does not require prior acquisition of point cloud data of the location where positioning is to be performed (e.g., area A2a shown in Figure 2), and that processing is lighter.

[0038] Principle of determining whether a moving body is moving forward, backward, or stopped In hybrid measurement, the positioning results obtained are the absolute position of the moving body 2 and the yaw angle of the camera 3. For example, in hybrid positioning, the positioning results obtained are the x and y coordinates of the moving body 2 on the map in the map information, and the yaw angle of the camera 3 relative to the x axis of the shooting direction on the map in the map information.

[0039] The positioning device 1 determines whether the moving body 2 is moving forward, backward, or stopped based on the absolute position of the moving body 2 obtained from hybrid positioning and the yaw angle of the camera 3.

[0040] 5 is a diagram illustrating the principle of determining the forward movement of the moving body 2. As described above, the absolute position of the moving body 2 is obtained as a positioning result of the hybrid positioning. The positioning device 1 calculates a movement vector of the moving body 2 from the two absolute positions of the moving body 2 obtained by the hybrid positioning.

[0041] For example, the positioning device 1 calculates a movement vector indicated by arrow A5c in FIG. 5 from the difference between the most recently obtained absolute position of the moving body 2 (the absolute position at time t indicated by black circle A5a in FIG. 5) and the absolute position one epoch earlier (the absolute position at time t-1 indicated by black circle A5b in FIG. 5).

[0042] The positioning device 1 calculates the azimuth angle θ of the calculated movement vector. For example, the positioning device 1 calculates the azimuth angle θ of the calculated movement vector with respect to the x-axis by taking the arc tangent of the calculated movement vector.

[0043] As described above, the positioning result of the hybrid positioning is the yaw angle of the camera 3. For example, the positioning device 1 can obtain the yaw angle of the camera 3 (e.g., the azimuth angle of the camera 3 with respect to the x-axis) as the positioning result of the hybrid positioning, as indicated by arrow A5d in Fig. 5 .

[0044] 5, if the moving body 2 is moving forward, the direction of the movement vector of the moving body 2 will be oriented in the direction of the yaw angle of the camera 3. In other words, the positioning device 1 can determine that the moving body 2 is moving forward if the movement vector representing the movement of the moving body 2 is oriented in the direction of the yaw angle of the camera 3 obtained from the hybrid positioning.

[0045] 6 is a diagram illustrating the principle of determining whether the moving object 2 is retreating. As described above, the absolute position of the moving object 2 is obtained as a positioning result of the hybrid positioning. As in the description of FIG. 5 , the positioning device 1 calculates a movement vector of the moving object 2 from the two absolute positions of the moving object 2 obtained by the hybrid positioning.

[0046] For example, the positioning device 1 calculates the movement vector shown by the arrow A6c in Figure 6 from the difference between the most recently obtained absolute position of the moving body 2 (the absolute position at time t shown by the black circle A6a in Figure 6) and the absolute position one epoch ago (the absolute position at time t-1 shown by the black circle A6b in Figure 6).

[0047] The positioning device 1 calculates the azimuth angle θ of the calculated movement vector in the same manner as in the description of Fig. 5. For example, the positioning device 1 calculates the azimuth angle θ of the calculated movement vector with respect to the x-axis by taking the arc tangent of the calculated movement vector.

[0048] As described above, the positioning result of the hybrid positioning is the yaw angle of the camera 3. For example, the positioning device 1 can obtain the yaw angle of the camera 3 (e.g., the azimuth angle with respect to the x-axis) as the positioning result of the hybrid positioning, as indicated by the arrow A6d in FIG.

[0049] 6, if the moving body 2 is moving backward, the direction of the movement vector of the moving body 2 will be opposite to the direction of the yaw angle of the camera 3. In other words, if the movement vector representing the movement of the moving body 2 is opposite to the direction of the yaw angle of the camera 3 obtained from the hybrid positioning, the positioning device 1 can determine that the moving body 2 is moving forward.

[0050] 7 is a diagram illustrating the principle of determining whether the moving body 2 has stopped. As described above, the absolute position of the moving body 2 is obtained as a positioning result of the hybrid positioning. As in the description of FIG. 5 , the positioning device 1 calculates a movement vector of the moving body 2 from the two absolute positions of the moving body 2 obtained by the hybrid positioning.

[0051] For example, the positioning device 1 calculates a movement vector from the difference between the most recently obtained absolute position of the moving body 2 (the absolute position at time t shown by the black circle A7a in Figure 7) and the absolute position one epoch ago (the absolute position at time t-1 shown by the black circle A7b in Figure 7).

[0052] If the moving object 2 is stopped, the magnitude of the movement vector of the moving object 2 is 0 (or approximately 0), as shown in Fig. 7. In other words, the positioning device 1 can determine that the moving object 2 is stopped if the magnitude of the movement vector representing the movement of the moving object 2 is 0 (or approximately 0).

[0053] Operation of the Positioning Device Fig. 8 is a flowchart showing an example of the operation of the positioning device 1. The positioning device 1 repeatedly executes the processing of the flowchart shown in Fig. 8, for example, at a fixed cycle. Note that although not shown in the flowchart of Fig. 8, the positioning device 1 repeatedly executes hybrid positioning, for example, at a fixed cycle, and calculates the absolute position (coordinates) of the moving object 2 and the yaw angle of the camera 3.

[0054] The positioning device 1 calculates (obtains) the movement vector of the moving object 2 by calculating the difference between the coordinates of the moving object 2 at time t and the coordinates of the moving object 2 at time t-1, which is one epoch earlier (S1). Note that the coordinates of the moving object 2 are, for example, coordinates on the map in the map information of the area A2a shown in FIG. 2.

[0055] The positioning device 1 calculates the average of N samples of the movement vector calculated in S1 (S2). The value of N (N is a positive integer) may be determined, for example, taking into consideration the response speed and error suppression. For example, increasing the value of N slows the response speed but increases the error suppression. Decreasing the value of N speeds up the response speed but decreases the error suppression.

[0056] The positioning device 1 compares the magnitude of the movement vector calculated in S2 as the average of N samples with a rest determination threshold (S3).

[0057] The positioning device 1 determines whether the magnitude of the movement vector averaged over N samples is equal to or less than the rest determination threshold (S4).

[0058] If the magnitude of the movement vector in the average of N samples is equal to or smaller than the rest determination threshold (YES in S4), the positioning device 1 determines that the moving object 2 is resting (S5).

[0059] On the other hand, if the magnitude of the motion vector in the N sample average is not equal to or less than the rest determination threshold (NO in S4), the positioning device 1 calculates the azimuth angle of the motion vector (S6). For example, the positioning device 1 calculates the azimuth angle of the motion vector by taking the arc tangent of the motion vector calculated in S1.

[0060] The positioning device 1 calculates the difference (absolute value) between the azimuth angle of the movement vector calculated in S6 and the yaw angle of the camera 3 calculated by hybrid positioning (S7).

[0061] The positioning device 1 determines whether the difference between the azimuth angle of the movement vector calculated in S7 and the yaw angle of the camera 3 is equal to or smaller than the forward movement determination threshold (S8).

[0062] If the difference between the azimuth angle of the movement vector and the yaw angle of the camera 3 is equal to or smaller than the forward movement determination threshold (YES in S8), the positioning device 1 determines that the moving body 2 is moving forward (S9).

[0063] On the other hand, if the difference between the azimuth angle of the movement vector and the yaw angle of the camera 3 is not equal to or less than the forward movement determination threshold (NO in S8), the positioning device 1 determines that the moving body 2 is moving backward (S10).

[0064] After performing the stationary determination in S5, the forward movement determination in S9, or the backward movement determination in S10, the positioning device 1 takes a majority vote of the determination results for the three epochs and outputs the result as the final determination result (S11).

[0065] Majority Decision on Determination Results When hybrid positioning is switched from feature point positioning to marker positioning, the absolute position of the moving object 2 is corrected by the marker positioning, and the actual moving direction (forward or backward) of the moving object 2 may not match the determination result of the positioning device 1. For example, even if the moving object 2 is moving forward, the determination result may be backward due to the correction of the absolute position by the marker positioning. Therefore, the positioning device 1 takes a majority decision on the determination results and outputs the result of the majority decision as the final determination result.

[0066] 9 is a diagram illustrating the majority decision of the determination result. In the explanation of FIG. 9, it is assumed that the moving object 2 is moving forward. V1 to V3 shown in FIG. 9 indicate feature point positioning, and M1 indicates marker positioning.

[0067] As shown in Fig. 9, two epochs before time t-2, feature point positioning V2 is executed and the determination result is "advance." One epoch before time t-1, feature point positioning V3 is executed and the determination result is "advance." In the epoch at time t, marker positioning M1 is executed and the determination result is "regress."

[0068] In marker positioning, the accumulated error of the absolute position obtained by feature point positioning before the marker positioning is corrected. Therefore, the determination result obtained by marker positioning M1 may determine that the moving object 2 is moving backward, even though the moving object 2 is actually moving forward, as shown at time t in Figure 9.

[0069] Therefore, the positioning device 1 outputs the majority vote of the determination results for three epochs, including the determination result at time t, the determination result one epoch ago, and the determination result two epochs ago, as the final determination result. For example, in the example of Figure 9, there are two forward determinations and one backward determination, so the positioning device 1 outputs forward as the final determination result.

[0070] When feature point positioning is switched to marker positioning and the marker positioning continues, the positioning result of the first marker positioning may not match the actual moving direction (forward or backward) of the moving body 2 due to correction of the absolute position, but the positioning results of the second and subsequent marker positioning will match the actual moving direction of the moving body 2. Furthermore, when marker positioning is switched to feature point positioning or when feature point positioning continues, the absolute position of the moving body 2 is not corrected in the feature point positioning, so the determination result of the feature point positioning will often match the moving direction of the moving body 2.

[0071] Therefore, when the moving object 2 is moving in one direction, either forward or backward, it is unlikely that two erroneous determination results will be included in the determination results for three consecutive epochs. Therefore, the determination results for which a majority vote is taken only need to be for three epochs. Furthermore, by taking a majority vote for three epochs, the positioning device 1 can shorten the processing time.

[0072] 10 is a block diagram showing the configuration of the positioning device 1. As shown in FIG. 11, the positioning device 1 has a control unit 11, a storage unit 12, and a communication unit 13.

[0073] The control unit 11 controls the entire positioning device 1. The control unit 11 may be configured by a processor such as a CPU (central processing unit), for example.

[0074] The control unit 11 includes a marker positioning unit 11 a, a feature point positioning unit 11 b, a calculation unit 11 c, and a determination unit 11 d. The control unit 11 may realize the functions of the above-mentioned units in accordance with a program (application) stored in the storage unit 12.

[0075] The marker positioning unit 11 a calculates the position of the moving object 2 and the attitude of the camera 3 based on the marker images included in the image data of the camera 3 .

[0076] If the image data of the camera 3 does not contain a marker image, the feature point positioning unit 11b calculates the position of the moving object 2 and the attitude of the camera 3 based on the past position of the moving object 2 and the feature points of the image data.

[0077] The calculation unit 11 c calculates the movement vector of the moving object 2 based on the position of the moving object 2 .

[0078] The determination unit 11d determines whether the moving object 2 is moving forward or backward based on the attitude and movement vector of the camera 3. The determination unit 11d determines whether the moving object 2 is stopping based on the magnitude of the movement vector.

[0079] For example, the determination unit 11d compares the yaw angle indicating the attitude of the camera 3 with the azimuth angle of the movement vector to determine whether the moving body 2 is moving forward or backward.

[0080] For example, the determination unit 11d determines that the moving body 2 is moving forward when the absolute value of the difference between the yaw angle of the camera 3 and the azimuth angle of the movement vector is equal to or less than a threshold. For example, the determination unit 11d determines that the moving body 2 is moving backward when the absolute value of the difference between the yaw angle of the camera 3 and the azimuth angle of the movement vector is greater than a threshold. For example, the determination unit 11d determines that the moving body 2 is stopped when the magnitude of the movement vector is equal to or less than a threshold. For example, the determination unit 11d makes a final determination of whether the moving body 2 is moving forward or backward based on a majority vote of three determination results determining whether the moving body 2 is moving forward or backward.

[0081] The storage unit 12 stores programs for causing the control unit 11 to execute various processes. These programs may include, for example, an operating system (OS) program and an application program. The storage unit 12 also stores various data necessary for processing by the control unit 11. For example, the storage unit 12 stores map information of the area A2a. The storage unit 12 stores movement range information. The storage unit 12 stores no-entry area information. The storage unit 12 may be, for example, a solid state drive (SSD), a random access memory (RAM), a flash memory, a read-only memory (ROM), and / or a hard disk drive (HDD). The determination result of the determination unit 11d may be associated with position information of the moving object 2 and stored in the storage unit 12.

[0082] The communication unit 13 communicates with the camera 3 via, for example, a wireless or wired LAN.

[0083] Summary of the embodiment As explained above, the positioning device 1 receives image data from the camera 3 mounted on the moving object 2. The positioning device 1 calculates the position of the moving object 2 and the attitude of the camera 3 based on marker images included in the image data. If the image data does not include marker images, the positioning device 1 calculates the position of the moving object 2 and the attitude of the camera 3 based on the past position of the moving object 2 and feature points of the image data. The positioning device 1 calculates a movement vector of the moving object 2 based on the position of the moving object 2. The positioning device 1 determines whether the moving object 2 is moving forward or backward based on the attitude of the camera 3 and the azimuth angle of the movement vector. This allows the positioning device 1 to determine whether the moving object 2 is moving forward or backward when positioning the moving object 2.

[0084] For example, when the moving body 2 is moving forward, the azimuth angle of the movement vector of the moving body 2 points in the direction of the yaw angle of the camera 3. When the moving body 2 is moving backward, the azimuth angle of the movement vector of the moving body 2 points in the opposite direction to the yaw angle of the camera 3. Therefore, the positioning device 1 can determine whether the moving body 2 is moving forward or backward, based on the attitude of the camera 3 and the azimuth angle of the movement vector.

[0085] Furthermore, when analyzing the flow of the moving object 2, information such as the forward and backward movement of the moving object 2 is important information. Since the positioning device 1 can determine the forward and backward movement of the moving object 2, the flow analyst does not need to visually determine the forward and backward movement by, for example, playing back a video image from the camera 3, thereby reducing the workload of analyzing the flow of the moving object 2.

[0086] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims. It is understood that such modifications or alterations also fall within the technical scope of the present disclosure. Furthermore, the components in the embodiments may be combined in any manner without departing from the spirit of the present disclosure.

[0087] In the above-described embodiments, the notation "part" used for each component may be replaced with other notations such as "circuitry", "assembly", "device", "unit", or "module". Calculation may be read as computation.

[0088] The present disclosure can be realized by software, hardware, or software in conjunction with hardware. Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may also be called an IC, system LSI, super LSI, or ultra LSI.

[0089] The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0090] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility.

[0091] The present disclosure is useful, for example, for analyzing the movement of moving objects.

[0092] REFERENCE SIGNS LIST 1 Positioning device 2 Mobile object 3 Camera A2b, A3b Marker 11 Control unit 11a Marker positioning unit 11b Feature point positioning unit 11c Calculation unit 11d Determination unit 12 Storage unit 13 Communication unit

Claims

1. A positioning device having: a receiving unit that receives image data from a camera mounted on a moving object; a marker positioning unit that calculates the position of the moving object and the attitude of the camera based on a marker image included in the image data; a feature point positioning unit that calculates the position of the moving object and the attitude of the camera based on a past position of the moving object and feature points of the image data if the marker image is not included in the image data; a calculation unit that calculates a movement vector of the moving object based on the position of the moving object; and a determination unit that determines whether the moving object is moving forward or backward based on the attitude of the camera and the movement vector.

2. The positioning device according to claim 1, wherein the determination unit compares a yaw angle indicating the attitude of the camera with an azimuth angle of the movement vector to determine whether the moving body is moving forward or backward.

3. The positioning device according to claim 2, wherein the determination unit determines that the moving body is moving forward when an absolute value of a difference between the yaw angle of the camera and the azimuth angle of the movement vector is equal to or smaller than a threshold value.

4. The positioning device according to claim 2, wherein the determination unit determines that the moving body is moving backward when an absolute value of a difference between the yaw angle of the camera and the azimuth angle of the movement vector is greater than a threshold value.

5. The positioning device according to claim 1, wherein the determination unit determines that the moving object is at a standstill when the magnitude of the movement vector is equal to or smaller than a threshold value.

6. The positioning device according to claim 1, wherein the determination unit makes a final determination of the forward movement or backward movement of the moving object based on a majority vote of three determination results determining the forward movement or backward movement of the moving object.

7. A moving body comprising: a camera; a receiving unit that receives image data from the camera; a marker positioning unit that calculates a position of the moving body and an attitude of the camera based on a marker image included in the image data; and if the marker image is not included in the image data, a feature point positioning unit that calculates a position of the moving body and an attitude of the camera based on a past position of the moving body and feature points of the image data; a calculation unit that calculates a movement vector of the moving body based on the position of the moving body; and a determination unit that determines forward and backward movement of the moving body based on the attitude of the camera and the movement vector.

8. A positioning method in which a positioning device receives image data from a camera mounted on a moving object, calculates the position of the moving object and the attitude of the camera based on a marker image contained in the image data, and if the image data does not contain the marker image, calculates the position of the moving object and the attitude of the camera based on the past position of the moving object and feature points of the image data, calculates a movement vector of the moving object based on the position of the moving object, and determines forward and backward movement of the moving object based on the attitude of the camera and the movement vector.

Images