Observation information processing method, observation information processing device, and recording medium

Abstract

To provide an observation information processing method that can suppress an error in tracking in EOT (extended object tracking) when the size of a target object as seen from an observing subject changes.SOLUTION: An observation information processing method of the present invention comprises: an acquisition step of acquiring first observation information and second observation information; an adjusting step of generating a third contour so that the size of a first contour included in a first image and the size of a second contour included in a second image are the same size; and a projection step of projecting a first point of interest onto the third contour as a second point of interest so that a relative position of the first point of interest on a first side on the first contour is equal to a relative position of a second point of interest on a second side of the third contour, and projecting the second point of interest onto the second contour as a third point of interest so that the relative position of the second point of interest on the second side on the third contour is equal to a relative position of a third point of interest on a third side of the second contour.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to an observation information processing method, an observation information processing device, and a recording medium. [Background technology]

[0002] In object recognition using LiDAR (Laser Imaging Detection and Ranging), a method called EOT (Extended Object Tracking) is used to predict the shape of a target object from a group of observation points and track the target object. Patent Document 1 discloses a tracking device that suppresses tracking loss of a target object in EOT. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 075581 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 cannot prevent errors from occurring in tracking in EOT when the size of the target object as seen by the observer changes due to a change in the relative positional relationship between the observer and the target, such as when a target object that was partially hidden behind an object emerges from the shadow, or when the target object approaches a stationary observer.

[0005] Therefore, the present invention provides an observation information processing method, etc., that can suppress errors in tracking in EOT even when the size of the target object as seen by the observing subject changes due to a change in the relative positional relationship between the observing subject and the target object. [Means for solving the problem]

[0006] An observation information processing method according to one aspect of the present invention is an observation information processing method using extended object tracking, and includes an acquisition step of acquiring first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is a subject observing the target object; an adjustment step of generating a third contour based on the first contour and the second contour so that a size of a first contour, which is a contour of the target object estimated from the first observation information, and a size of a second contour, which is a contour estimated from the second observation information, are the same; a projection step of projecting the first point of interest on the first contour as the second point of interest onto the third contour so that a relative position of the first point of interest on a first side, which is the side of the first contour on which the first point of interest is located, is equal to a relative position of the second point of interest on a second side, which is the side of the third contour on which the second point of interest is located, and a projection step of projecting the second point of interest on the third contour as the third point of interest onto the second contour so that a relative position of the second point of interest on a second side, which is the side of the third contour on which the second point of interest is located, is equal to a relative position of the third point of interest on a third side, which is the side of the second contour on which the third point of interest is located.

[0007] An observation information processing device according to one aspect of the present invention is an observation information processing method using extended object tracking, comprising: an acquisition step of acquiring first observation information (observation information) acquired at a first time point and second observation information (observation information) acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is a subject observing the target object; and a step of adjusting the first contour and the second contour so that a size of the first contour, which is the contour of the target object estimated from the first observation information, is the same as a size of the second contour, which is the contour estimated from the second observation information. The method includes an adjustment step of generating the second contour into a third contour, and a projection step of projecting the first focus point on the first contour onto the third contour as the second focus point so that the relative position of the first focus point on the first side, which is the side of the first contour on which the first focus point is located, is equal to the relative position of the second focus point on the second side, which is the side of the third contour on which the second focus point is located, and in the projection step, the third contour onto which the second focus point is projected is changed to the size of the second contour while maintaining the relative position of the second focus point on the third contour.

[0008] A recording medium according to one aspect of the present invention is a non-transitory computer-readable recording medium on which a program for causing a computer to execute an observation information processing method according to one aspect of the present invention is recorded. [Effects of the Invention]

[0009] An observation information processing method according to one aspect of the present invention can suppress errors in tracking in EOT even when the size of the target object as seen by the observer changes. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an observation information processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of an observation body equipped with a LiDAR. [Figure 3] FIG. 3 is a diagram showing the concept of contour prediction in EOT. [Figure 4] FIG. 4 is a diagram showing an example of prediction of the contour of a target object in EOT. [Figure 5] FIG. 5 is a diagram illustrating an example of a visible range of a target object by LiDAR. [Figure 6] FIG. 6 is a diagram showing an example of change in the shape of the contour of a target object in EOT. [Figure 7] FIG. 7 is a diagram showing an example of projection of a point of interest in a conventional EOT and an example of projection of a point of interest in an ideal EOT. [Figure 8] FIG. 8 is a diagram showing a method for projecting a point of interest in a conventional EOT and a method for projecting a point of interest according to the embodiment. [Figure 9] FIG. 9 is a diagram showing another method of projecting a point of interest according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of projection of a point of interest in EOT according to the embodiment. [Figure 11] FIG. 11 is a flowchart showing an observation information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0012] The embodiments described below each illustrate a preferred specific example of the present invention. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept of the present invention will be described as optional components that constitute a more preferred embodiment. Note that identical components will be assigned the same reference numerals, and descriptions thereof may be omitted.

[0013] (Embodiment) Below, we will explain an observation information processing method, observation information processing device, and recording medium that can suppress errors in tracking in EOT even when the size of the target object as seen by the observing subject changes due to a change in the relative positional relationship between the observing subject and the target, even though the size of the target object does not actually change.

[0014] [Outline of observation information processing device] First, an observation information processing device 1 that realizes an observation information processing method according to an embodiment will be described. Fig. 1 is a block diagram of the observation information processing device 1 according to an embodiment.

[0015] The observation information processing device 1 includes a control unit 11 and a memory unit 12. The control unit 11 is realized by a processor such as a CPU (Central Processing Unit) and a memory, etc. The memory unit 12 is realized by a memory such as a ROM, a RAM, or a hard disk. In the observation information processing device 1, the control unit 11 executes a program stored in the memory unit 12, thereby functioning as an acquisition unit 11a, an adjustment unit 11b, and a projection unit 11c. The operations of the acquisition unit 11a, the adjustment unit 11b, and the projection unit 11c included in the control unit 11, which will be described below, are realized by the control unit 11 using the memory and the processor.

[0016] The acquisition unit 11 acquires data acquired by a LiDAR. The LiDAR measures the distance between the target object and an observation body equipped with LiDAR equipment by irradiating the target object with laser light and acquiring the laser light reflected from the target object. The data acquired by the LiDAR includes first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point. Here, the first observation information and the second observation information include observation points obtained by observing the target object to be tracked in EOT from an observation body that is the subject observing the target object. An "observation point" is a point on the target object obtained by observing the target object. Here, the acquisition unit 11 acquires data from a LiDAR, but is not limited thereto. Data may be acquired from any device that can measure the distance to the target object, such as a laser sensor other than a LiDAR, an ultrasonic sensor, or a camera.

[0017] Adjustment unit 12 generates third contours 300ac and 300bc (see FIG. 8 ; details will be described later) by changing the sizes of first contour 300a and second contour 300b (see FIG. 8 ; details will be described later) so that the size of first contour 300a (see FIG. 8 ; details will be described later), which is the contour of the target object estimated from the observation points included in the first observation information, is the same as the size of second contour 300b (see FIG. 8 ; details will be described later), which is the contour estimated from the observation points included in the second image. In other words, adjustment unit 12 changes the sizes of first contour 300a and second contour 300b to the sizes of third contours 300ac and 300bc that are different from the sizes of first contour 300a and second contour 300b. Adjustment unit 12 may change the size of first contour 300a to the size of second contour 300b, or may change the size of second contour 300b to the size of first contour 300a. Furthermore, the size of the first contour 300a and the second contour 300b may be the same as the size of the third contours 300ac and 300bc.

[0018] The adjustment unit 12 may estimate the first contour 300a and the second contour 300b from the first image and the second image acquired by the acquisition unit 11. The estimation may be performed using a general EOT method.

[0019] The projection unit 13 projects the first point of interest as a second point of interest onto the third contour 300ac so that the relative position of the first point of interest on the first side, which is the side of the first contour 300a on which the first point of interest is located, is equal to the relative position of the second point of interest on the second side, which is the side of the third contour 300ac on which the second point of interest is located. The projection unit 13 then projects the second point of interest as a third point of interest onto the second contour 300b so that the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located. The "point of interest" refers to a point located at a characteristic location on the target object that is useful for reproducing the contour. The characteristic point is preferably selected from an area other than the blind spot area, and in order to improve the accuracy of contour reproduction, it is preferable to select at least one of the nearest point closest to the observation body, one or more edge points of the visible area visible from the observation body, the midpoint between the nearest point and one or more edge points of the visible area, or a point near said midpoint. In particular, when only one point is used as the characteristic point, the nearest point is preferable.

[0020] Note that "projecting" means determining the position of a point of interest on one contour that corresponds to a point of interest on another contour.

[0021] Specifically, the projection unit 13 may project the first point of interest as the second point of interest onto the third contour 300ac so that the ratio of the length from the end of the first side, which is the side of the first contour 300a on which the first point of interest is located, to the length of the first side is equal to the ratio of the length from the end of the second side, which is the side of the third contour 300ac on which the second point of interest is located, to the length of the second side.The projection unit 13 may then project the second point of interest as the third point of interest onto the second contour 300b so that the ratio of the length from the end of the second side, which is the side of the third contour 300ac on which the second point of interest is located, to the length of the second side is equal to the ratio of the length from the end of the third side, which is the side of the second contour 300b on which the third point of interest is located, to the length of the third side.

[0022] The storage unit 15 stores data acquired or generated by the acquisition unit 11, the adjustment unit 12, and the projection unit 13. For example, the storage unit 15 may store data of the first image and the second image acquired by the acquisition unit 11, data of the third contour 300ac and the like generated by the adjustment unit 12, and data of the second attention point on the third contour 300ac and the third attention point on the second contour 300b projected by the projection unit 13.

[0023] The observation information processing device 1 acquires data observed by the observation body 100 and performs observation information processing. FIG. 2 is a diagram showing an example of an observation body 100 equipped with a LiDAR. The observation body 100 is, for example, a four-wheeled automobile. The observation body 100 is equipped with a LiDAR on its upper surface. While traveling, the observation body 100 uses the LiDAR to measure the distance to the target object being observed.

[0024] The observation body 100 is not limited to a four-wheeled vehicle as shown in FIG. 2. It may be a small four-wheeled vehicle or a large four-wheeled vehicle such as a truck. The observation body 100 may also be a two-wheeled vehicle, a three-wheeled vehicle, or a robot. Alternatively, the observation body 100 may be a ship. The observation body 100 does not have to be a moving object, but may also be an object fixed in one place, such as a support pole.

[0025] [Object Recognition by EOT] Next, object recognition using EOT will be described. Fig. 3 is a diagram showing the concept of predicting a contour 300 in EOT. In EOT, an observation body 100 (not shown) observes a target object 200 and acquires observation points on the target object 200. Then, the observation information processing device 1 predicts (i.e., generates) a contour 300 of the target object 200 from the observation points on the target object 200. Note that the contour 300 is a contour in a bird's-eye view of the target object 200 viewed from above. For example, the observation information processing device 1 predicts a contour of the target object 200 as shown in Fig. 4.

[0026] FIG. 4 is a diagram showing an example of prediction of a contour 300 (300a, 300b) of a target object 200 (not shown) in EOT. The observed object 100 is, for example, a four-wheeled vehicle traveling on a road, and the target object 200 is an oncoming vehicle. As the relative position between the observed object 100 and the target object 200 changes, the shape of the target object 200 as seen from the observed object 100 changes. A first contour 300a is the contour of the target object 200 at time t, and the contour of the target object 200 at time t', which is a time after time t, is a second contour 300b. In this way, as the relative position between the observed object 100 and the target object 200 changes, the contour 300 of the target object 200 changes. In EOT, the changing shape of the target object 200 as seen from the observed object 100 is tracked as the contour 300 of the same object.

[0027] FIG. 5 is a diagram showing an example of the visible range of a target object by LiDAR. This figure shows how the visible range of the target object 200 from the observation body 100 changes when the relative position between the observation body 100 and the target object 200 changes. (a) of FIG. 5 shows how only a portion of the target object 200 (the portion depicted by the solid line) can be seen from the observation body 100. (b) of FIG. 5 shows how the entire length of the target object 200 can be seen from the observation body 100 as the distance between the observation body 100 and the target object 200 decreases. Therefore, as shown in (a) and (b) of FIG. 5, when the relative position between the observation body 100 and the target object 200 changes, the shape of the target object 200 as seen from the observation body 100 changes.

[0028] FIG. 6 is a diagram showing an example of a change in the shape of the contour 300 of the target object 200 in EOT. A blind spot area as seen from the observation body 100 (not shown), for example, an area that is hidden by a building or the like, is shown as a hatched area in FIG. 6. As the target object 200 moves from a blind spot area as seen from the observation body 100 to an area that is not a blind spot as seen from the observation body 100, the size of the contour 300 of the target object 200 as seen from the observation body 100 expands as shown in (a) and (b) of FIG. 6. Specifically, in (a) of FIG. 6, a blind spot of about half of the target object 200 occurs, so only a portion (about half) of the target object 200 is visible from the observation body 100. Therefore, the contour 300 of the target object 200 becomes about half the contour of the target object 200. Then, for example, a point of interest 400 is placed at the upper left corner of the contour 300. In (b) of Figure 6, the range of the target object 200 visible from the observed body 100 expands, and the shape of the contour 300 becomes larger than in (a) of Figure 6. In this case, the position of the attention point 400 ((a) of Figure 6) at a predetermined position on the contour 300, which was set on the contour 300 before the size as seen from the observed body 100 changed, must also be set on the contour 300 ((b) of Figure 6) after the size as seen from the observed body 100 has changed, so that it indicates the same feature on the contour, for example, a point on a corner.

[0029] [Problems with conventional EOT] Next, the problems of the conventional EOT will be described. Fig. 7 is a diagram showing an example of the projection of the attention point 400 in the conventional EOT and an example of the projection of the attention point 400 in the ideal EOT. Fig. 7(a)-(b) show an example of the projection of the attention point in the conventional EOT, and Fig. 7(c) shows an example of the projection of the attention point 400 in the ideal EOT.

[0030] The attention point 400 is a point that indicates a characteristic location on the contour 300 (not shown), and it is desirable that the attention point 400 be projected onto the contour 300 after it has changed as seen from the observed body 100 so that it indicates the same characteristic location on the contour 300 as seen from the observed body 100, even if the shape of the contour 300 as seen from the observed body 100 changes.

[0031] However, in the past, for example, the position of the center line CL of the first contour 300a at time t (as shown in FIG. 7(a)) was aligned with the position of the center line CL' of the second contour 300b at time t' (as shown in FIG. 7(b)), and the point of interest on the first contour 300a was projected onto the second contour 300b at the same position as on the first contour 300a. Specifically, the distance from the center line CL of the point of interest 400 on the first contour 300a is the same as the distance from the center line CL' of the point of interest 400 on the second contour 300b. In other words, in the past, the point of interest was projected onto a position from the center line CL' of the second contour 300b that is the same distance as the distance from the center line CL of the first contour 300a to the point of interest. As a result, the position of the point of interest 400, which should be projected onto a characteristic location (here, the upper left corner) as shown in FIG. 7(c), was conventionally projected onto the position shown in FIG. 7(b). In this case, the estimated shape changes depending on the visible area seen from the observation object, so when reproducing the contour, the target object may be reproduced smaller than its actual size or the shape may be reproduced incorrectly.

[0032] 7(b) is not located in a characteristic part of the target object 200 (here, the upper left corner), and therefore is not suitable as a point of interest that is useful for reproducing a visible outline of the target object 200. For this reason, in the conventional observation information processing device, the point of interest is not located in a characteristic part of the target object 200, which causes problems such as an inaccurate outline reproduction, and the target object cannot be stably tracked.

[0033] [Point of interest projection method] Therefore, in the observation information processing method according to the embodiment, the attention point 400 is projected by the method described below. The observation information processing method according to the embodiment is realized by an observation information processing device 1 according to the embodiment. Fig. 8 is a diagram showing a method of projecting the attention point 400 in a conventional EOT (Fig. 8(a)) and a method of projecting the attention point according to the embodiment (Fig. 8(b)).

[0034] Fig. 8(a) shows a method for projecting a point of interest in a conventional EOT. As shown in Fig. 8(a), in the conventional method for projecting a point of interest in an EOT, the center line of the first contour 300a at time t and the center line of the second contour 300b at time t' are aligned, and the position of the point of interest on the first contour 300a at time t is directly projected onto the second contour 300b at time t'.

[0035] In contrast, the projection method of the attention point 400 according to the embodiment shown in Fig. 8(b) is processed as follows: Here, the first contour 300a shown in Fig. 8(b) is an observed contour which is an observed value at the current time t', and the second contour 300b is a predicted contour at the current time t' predicted from an estimated contour (described later) which is an estimated value at a time t prior to the current time t'.

[0036] First, the adjustment unit 12 normalizes the size of the first contour 300a at time t and the size of the second contour 300b at time t' to generate the third contour 300ac and the third contour 300bc, respectively ((b-1) to (b-2) in FIG. 8). Here, "normalizing" means setting the sizes of the first contour 300a and the second contour 300b to a predetermined size. In this embodiment, the third contours 300ac and 300bc are set to squares with a 1:1 ratio of height to width. This normalization process allows the projection unit 13 to project a point of interest at a characteristic location on the first contour 300a onto the corresponding characteristic location on the second contour 300b.

[0037] At this time, the projection unit 13 projects the position of the attention point 400 on the first contour 300a onto the third contour 300ac (i.e., the upper right corner) while maintaining the position of the attention point 400 on the first contour 300a at a characteristic location (here, the upper right corner) ((b-2) of FIG. 8). Here, the third contours 300ac and 300bc are contours generated by changing the sizes of the first contour 300a and the second contour 300b themselves. However, the third contours 300ac and 300bc may be contours generated separately from the first contour 300a and the second contour 300b.

[0038] On the other hand, for the point of interest on the second contour 300b, the projection unit 13 normalizes the second contour 300b and then determines the point at the same position as the point of interest 400 on the third contour 300ac, which is the observed contour, as the position of the point of interest 400 on the third contour 300bc ((b-3) of FIG. 8). Then, while maintaining the relative position of the point of interest 400 on the third contour 300bc, the size of the third contour 300bc is enlarged or reduced to change it to the size before normalization, i.e., the second contour 300b ((b-4) of FIG. 8). Here, the second contour 300b changed to the size before normalization after the normalization process is referred to as an estimated contour.

[0039] When the third contours 300ac and 300bc are generated separately from the first contour 300a and the second contour 300b, the focus point 400 is projected onto the original second contour 300b (i.e., the upper right corner) while maintaining the relative position of the focus point 400 on the third contour 300bc (maintaining the characteristic points).

[0040] In this manner, in this embodiment, the second contour 300b, which is an estimated value at the current time t', is determined from the third contour 300ac ((b-2) of Figure 8), which is obtained by normalizing the first contour 300a, which is the observed value at the current time t', and the second contour 300b (estimated contour) ((b-4) of Figure 8), which is the predicted value at the current time t' predicted from the previous time t, normalized and then returned to its size before normalization.

[0041] In the above embodiment, the second contour 300b is normalized and then the attention point 400 is projected, but the present invention is not limited to this embodiment.

[0042] For example, the projection unit 13 first projects the first contour 300a onto the second contour 300b at the same position as the position of the attention point 400 on the first contour 300a, with the center lines of the first contour 300a and the second contour 300b aligned, as in the conventional case (Figure 8(a)).

[0043] After performing the normalization process described above, the projection unit 13 projects the position of the attention point 400 on the third contour 300ac (i.e., the upper right corner) onto the third contour 300bc (i.e., the upper right corner) ((b-3) of FIG. 8). At this time, the attention point 400 on the third contour 300bc before the projection is deleted.

[0044] This allows the projection unit 13 to determine the point of interest at a position on the predicted contour that reflects the relative position of the point of interest on the observed contour.

[0045] Finally, the adjustment unit 12 changes (i.e., returns) the size of the third contour 300bc to the size of the second contour 300b, and the projection unit 13 projects the attention point 400 on the third contour 300bc onto the attention point 400 on the second contour 300b while maintaining the attention point 400 projected onto the third contour 300bc located at a characteristic location (i.e., the upper right corner) ((b-4) of Figure 8).

[0046] In the above example, a point of interest at a corner of a contour is used as a characteristic location. However, the point of interest may be a point on a side of the contour. In this case, the projection unit 13 projects the point of interest onto the third contour 300bc so that the ratio of the length from the end of the side of the first contour 300a before normalization on which the point of interest is located to the length of the side is equal to the ratio of the length from the end of the side of the third contour 300ac on which the point of interest is located to the length of the side. The projection unit 13 may also project the point of interest onto the second contour 300b on which the point of interest is located to the length of the side of the third contour 300bc so that the ratio of the length from the end of the side of the third contour on which the point of interest is located to the point of interest is equal to the ratio of the length from the end of the side of the second contour 300b on which the point of interest is located to the length of the side of the second contour. As a result, the point of interest on the pre-normalized contour is projected onto the third contour 300bc at the same relative position as the relative position with respect to the pre-normalized contour (that is, while remaining located at the same characteristic point).

[0047] 9 is a diagram illustrating another method for projecting a point of interest according to an embodiment. The projection unit 13 may determine the distance from the center point of the pre-normalized contour to a location on the side of the pre-normalized contour and the angle in a predetermined direction from the center point by enlarging or reducing the pre-normalized contour by a factor by which the pre-normalized contour is enlarged or reduced to the third contour 300ac. The projection unit 13 may then determine the distance from the center point of the pre-normalized contour to a location on the side of the pre-normalized contour and the angle in a predetermined direction from the center point by enlarging or reducing the pre-normalized contour to a location on the side of the pre-normalized contour.

[0048] Specifically, the point of interest 400 on the first contour 300a is projected onto the third contour 300ac at a distance d2 and angle α2 obtained by magnifying the distance d1 from the center point 500 of the first contour 300a to the location of the point of interest 400 on the first contour 300a and the angle α1 in a predetermined direction as viewed from the center point 500 by a magnification at which the first contour 300a is magnified to the third contour 300ac ((a-1) to (a-2) in Figure 9).

[0049] Then, the second contour 300b is reduced to the size of the third contour 300bc, which is the same size as the third contour 300ac, and the attention point 400 on the contour 300ac is projected onto the contour 300bc (FIG. 9(a-2)).Then, the size of the third contour 300bc is enlarged to the size of the second contour 300b, and the attention point 400 is projected onto the second contour 300b at a distance d3 and an angle α3 from the center point 500 (FIG. 9(a-3)).

[0050] That is, first, the distance from center point 500 of first contour 300a to the location of the point of interest on first contour 300a is defined as a first distance d1, and the angle between line L1 extending in a predetermined direction as viewed from center point 500 and line L2 extending in the X-axis direction is defined as a first angle α1. Point of interest 400 on first contour 300a is projected onto a position on third contour 300ac determined by second distance d2 and second angle α2 obtained by changing first distance d1 and first angle α1 by the magnification used to change first contour 300a to third contour 300ac.

[0051] Then, the point of interest 400 is projected onto a position on a third contour 300bc obtained by normalizing the second contour 300b, which is determined by a second angle α2 between a line L1 extending in a predetermined direction as viewed from the center point 500 and a line L2 extending in the X-axis direction. Subsequently, the point of interest 400 is projected onto a position on the second contour 300b determined by a third distance d3 and a third angle α3 obtained by enlarging or reducing the second distance d2 and the second angle α2 by a magnification factor that changes the size of the third contour 300b to that of the second contour 300b.

[0052] As a result, the point of interest on the pre-normalized contour is projected onto the third contour 300bc at the same relative position as that of the pre-normalized contour (i.e., while being located at the same characteristic point), thus determining the position of the point of interest 400 on the contour 300.

[0053] An example of the above-mentioned method for projecting a point of interest will be described. Fig. 10 is a diagram showing an example of projection of a point of interest in EOT according to an embodiment. Here, an example is shown in which the shape of a target object as seen from an observation body at a past time is enlarged to match the shape of the target object as seen from an observation body at a current time.

[0054] FIG. 10(a) shows the shape of the target object 200 as seen from the observation body 100 (not shown) at a past time. Assume that a point of interest 400 is located in the upper left corner of the target object 200. FIG. 10(c) shows the shape of the target object 200 as seen from the observation body 100 at a current time. FIG. 10(b) shows an enlarged view of the shape of the target object 200 as seen from the observation body 100 at a past time (FIG. 10(a)) in accordance with the shape of the target object 200 as seen from the observation body 100 at a current time (FIG. 10(c)). In this case, as shown in FIG. 10(c), the position of the point of interest 400 at the upper left corner of the target object 200 is determined on the enlarged target object 200 in accordance with the change in the shape of the target object 200 as seen from the observation body 100, without changing its relative position on the target object 200. In other words, even when the shape is enlarged in this manner, the point of interest 400 remains located at the same characteristic location. This prevents the relative position of the attention point 400 with respect to the target object 200 from changing, suppresses errors in reproducing the contour of the target object 200 using the attention point 400, and as a result, the target object 200 can be tracked stably.

[0055] Next, the observation information processing method according to the embodiment will be described in detail. Fig. 11 is a flowchart showing the observation information processing method according to the embodiment.

[0056] First, the acquisition unit 11 acquires the first observation information and the second observation information (step S10). Specifically, the acquisition unit 11 acquires, from the storage unit 12 or the like, the observation information generated from data obtained by the LiDAR-equipped observation body 100 observing the target object 200.

[0057] The first observation information is observation information at a first time point, and includes an observation point on the target object 200 as seen from the observing body 100 at the first time point. The second observation information is observation information at a time point different from the first time point, and includes an observation point on the target object 200 as seen from the observing body 100 at the second time point. The observation point is a point on the target object that the observing body 100 recognizes by observing the target object 200. For example, the observation point is a point on the target object 200 where laser light emitted from the LiDAR is reflected on the target object 200.

[0058] Next, adjustment unit 12 generates third contours 300ac and 300bc such that first contour 300a, which is the contour of target object 200 estimated from the observation points included in the first observation information, and second contour 300b, which is the contour of target object 200 estimated from the observation points included in the second observation information, are the same in size (step S11). Specifically, adjustment unit 12 generates first contour 300a, which is the contour 300 of target object 200, from the observation points included in the first observation information, and generates second contour 300b, which is the contour 300 of target object 200, from the observation points included in the second observation information. Then, adjustment unit 12 changes the sizes of first contour 300a and second contour 300b so that the size of generated first contour 300a and the size of second contour 300b are the same, thereby generating third contours 300ac and 300bc. That is, the third contours 300ac and 300bc are contours generated separately from the first contour 300a and the second contour 300b in different sizes. Note that the third contours 300ac and 300bc may be the same size as the first contour 300a or the second contour 300b.

[0059] Next, the projection unit 13 projects the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side of the first contour 300a is equal to the relative position of the second point of interest on the second side of the third contour 300bc corresponding to the first side, i.e., so that the first and second points of interest are located at the same characteristic location on the contour (step S12). Specifically, the projection unit 13 projects the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located is equal to the relative position of the second point of interest on the second side of the third contour 300bc corresponding to the second point of interest on which the second point of interest is located. For example, the projection unit 13 may project the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the ratio of the length from the end of the first side, which is the side of the first contour 300a on which the first point of interest is located, to the length of the first side, and the ratio of the length from the end of the second side, which is the corresponding side of the third contour 300bc on which the second point of interest is located, to the length of the second side, are equal.

[0060] Then, the projection unit 13 projects the second point of interest on the third contour 300bc as the third point of interest onto the second contour 300b so that the relative position of the second point of interest on the second side of the third contour 300bc is equal to the relative position of the third point of interest on the third side of the second contour 300b, that is, so that the second and third points of interest are located at the same characteristic location on the contour (step S13). Specifically, the projection unit 13 projects the second point of interest on the third contour 300bc as the third point of interest onto the second contour 300b so that the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located. For example, the projection unit 13 may project the second point of interest on the third contour 300bc as a third point of interest onto the second contour 300b so that the ratio of the length from the end of the second side, which is the side of the third contour 300bc on which the second point of interest is located, to the length of the second side, and the ratio of the length from the end of the third side, which is the side of the second contour 300b on which the third point of interest is located, to the length of the third side, are equal.

[0061] Furthermore, when the third contours 300ac and 300bc are the same size as the first contour 300a or the second contour 300b, the projection unit 13 may project the first focus point on the first contour 300a as the third focus point onto the second contour 300b so that the relative position of the first focus point on the first side, which is the side of the first contour 300a on which the first focus point is located, is equal to the relative position of the second focus point on the third side, which is the side of the second contour 300b on which the third focus point is located.

[0062] Specifically, the projection unit 13 may project the first point of interest on the first contour 300a as a third point of interest onto the second contour 300b so that the ratio of the length from the end of the first side, which is the side of the first contour 300a on which the first point of interest is located, to the length of the first side, and the ratio of the length from the end of the third side, which is the side of the second contour 300b on which the third point of interest is located, to the length of the third side, are equal.

[0063] Alternatively, the projection unit 13 may project the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the ratio of the angle and distance from the center point of the first contour 300a to the position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located, to the angle and distance from the center point of the third contour 300bc to the position of the second point of interest on the second side of the third contour 300bc on which the second point of interest is located, is equal to the ratio of the lengths of the first side and the second side.

[0064] The projection unit 13 may then project the second point of interest on the third contour 300bc onto the second contour 300b as a third point of interest so that the ratio of the angle and distance from the X-axis from the center point of the third contour 300bc to the position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, to the angle and distance from the center point of the second contour 300b to the position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located, is equal to the ratio of the lengths of the second side and the third side.

[0065] [Reduce the number of observation points used] Next, a reduction in the number of observation points will be described as a technique useful for improving the real-time performance of processing. Of a first number of observation points obtained by observing the target object 200, the observation information processing device 1 uses a second number of observation points that is less than the first number. In other words, the observation information processing device 1 acquires multiple observation points on the target object 200 using LiDAR, but uses a fewer number of observation points than the acquired observation points for predicting the contour 300 in the first observation information and second observation information acquired by the acquisition unit 11.

[0066] This allows the observation information processing device 1 to reduce the processing load and shorten the processing time. According to a simulation, the observation information processing device 1 can reduce the average processing time by 42.9%. In this simulation, one of 1, 3, or 5 observation points is selected depending on the size of the contour of the observed object. The observation points are extracted from the points that make up the contour and are within a range that can be observed from the observation body 100. Specifically, of the points on the contour observed from the observation body, the points that are closest to the observation body 100, the points at both ends of the contour, and the midpoints between them are selected. In other words, the observation points are selected evenly from positions on the contour observed from the observation body 100. Note that the larger the contour of the observed object, the greater the number of observation points selected.

[0067] The simulation is performed on a PC with a 4-core CPU with 4.2 GHz each and 32 GB of memory. The LiDAR frame rate is 10 Hz, and each frame is processed within 100 msec. There are approximately 100 objects to be tracked, and the simulation is performed using data measured by an observation unit 100 traveling through a real urban area (e.g., Odaiba, Tokyo).

[0068] Furthermore, when the observation information processing device 1 observes multiple target objects 200 and obtains multiple observation points, the third number of observation points used to determine which of the multiple target objects 200 each observation point corresponds to may be less than the fourth number of observation points used in the process of projecting each of the multiple third contour attention points on the third contour 300bc onto the second contour 300b as multiple second contour attention points.

[0069] For example, in the process of determining (association) to which of the multiple target objects 200 multiple observation points on multiple target objects 200 acquired by LiDAR correspond, the observation information processing device 1 may use, for example, only one of the multiple observation points. Note that the number of observation points used may be any number as long as it is less than the number of observation points used for updating, which will be described later. Specifically, when determining which of the multiple observed contours corresponds to the predicted contour, the observation information processing device 1 may associate one point on the observed contour with one point on the predicted contour, thereby associating one of the multiple observed contours with the predicted contour.

[0070] The observation information processing device 1 may use, for example, five of the multiple observation points in the process of updating (updating) the estimated value of the contour 300 at the current time from the associated predicted contour and observed contour. Note that the number of observation points used may be any number greater than or equal to the number of observation points used for the correspondence.

[0071] Specifically, when the observation information processing device 1 estimates the contour 300 at the current time from the observed contour and the predicted contour, five points on the observed contour may be associated with five points on the predicted contour.

[0072] This allows the observation information processing device 1 to reduce the processing load and shorten the processing time. According to a simulation, the observation information processing device 1 can reduce the average processing time by 29.7%. In this simulation, only one nearest point among the observation points observed from the observation object 100 is used as a point of interest, and correspondence with points on the predicted contour is performed. Note that two or more points of interest may be used for correspondence with points on the predicted contour.

[0073] The simulation is performed on a PC with a 4-core CPU with 4.2 GHz each and 32 GB of memory. The LiDAR frame rate is 10 Hz, and each frame is processed within 100 msec. There are approximately 100 objects to be tracked, and the simulation is performed using data measured by an observation unit 100 traveling through a real urban area (e.g., Odaiba, Tokyo).

[0074] Furthermore, in the process of associating and updating observation points, observation points that are not observed by the observing body 100 are not used. This makes it possible to exclude from the prediction process observation points that are likely to be subject to unstable observation and noise.

[0075] By reducing the number of observation points used as described above, the observation information processing device 1 enables real-time processing even while a car or the like is traveling on a public road, etc. For example, it can stably and accurately reproduce the outline of a moving object and track it several tens of times per second or more.

[0076] [Effects, etc.] An observation information processing method using extended object tracking, comprising: an acquisition step of acquiring first observation information acquired at a first time point, the first observation information including observation points obtained by observing a target object (200) to be tracked from an observing body (100) that is a subject observing the target object (200), and second observation information acquired at a second time point different from the first time point, the second observation information including the observation points; and generating a third contour (300bc) by changing the sizes of first contour (300a), which is a contour (300) of the target object (200) estimated from the observation points included in the first observation information, and second contour (300b), which is a contour estimated from the observation points included in the second observation information, so that the sizes of first contour (300a) and second contour (300b) are the same. and a projection step of projecting the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side, which is the side of the first contour 300a on which the first point of interest is located, is equal to the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, and projecting the second point of interest on the third contour 300bc as a third point of interest onto the second contour 300b so that the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located.

[0077] As a result, even if the size of the contour 300 of the target object 200 changes due to a change in the relative positional relationship between the observing subject and the target, the observation information processing method according to the embodiment can project the point of interest, which is useful for reproducing the contour, onto the changed contour 300 without changing the relative position of the point of interest with respect to the contour 300 before and after the change in contour 300, i.e., while maintaining the point of interest located at the same characteristic location. Thus, according to the observation information processing method according to the embodiment, even if the size of the target object 200 changes as seen by the observing subject, the contour of the target object 200 can be accurately reproduced, errors in tracking in EOT can be suppressed, and the target object can be stably tracked. Therefore, by applying this method to automobile driving control, accurate and robust automatic driving and following driving can be achieved.

[0078] Also, for example, in the observation information processing method according to the embodiment, the first point of interest is a point corresponding to an observation point included in the first observation information.

[0079] As a result, the observation information processing method according to the embodiment can use, from among the observation points of the target object 200, points of interest that are useful for reproducing the contour of the target object.

[0080] Furthermore, for example, in the observation information processing method according to the embodiment, the first observation information includes a plurality of observation points, each of which is a plurality of first contour attention points including a first attention point projected onto first contour 300a, and the relative position of the first attention point on a first side, which is the side of first contour 300a on which the first attention point is located, is equal to the relative position of the second attention point on a second side, which is the side of third contour 300bc on which the second attention point is located. The first point of interest on the first contour 300a is projected onto the third contour 300bc as a second point of interest so that the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located.

[0081] As a result, the observation information processing method according to the embodiment can use multiple points of interest to accurately track the target object 200. Therefore, the observation information processing method according to the embodiment can track the target object 200 with higher accuracy.

[0082] Also, for example, in the observation information processing method according to the embodiment, the multiple observation points include, among the observation points, a point located closest to the observation body 100, a point located at the rightmost point as viewed from the observation body 100, a point located at the leftmost point as viewed from the observation body 100, and a point located in the middle between the rightmost point and the leftmost point.

[0083] As a result, the observation information processing method according to the embodiment can use points of interest located at characteristic locations on the target object 200. Therefore, according to the observation information processing method according to the embodiment, an accurate contour can be reproduced, and the target object 200 can be tracked with higher accuracy.

[0084] Furthermore, for example, the observation information processing method according to the embodiment uses, of a first number of observation points obtained by observing target object 200, a second number of observation points that is less than the first number.

[0085] As a result, the observation information processing method according to the embodiment can speed up processing by using a smaller number of observation points than conventional methods for tracking the target object 200, thereby improving real-time performance.

[0086] Furthermore, for example, in the observation information processing method according to the embodiment, when multiple target objects 200 are observed and multiple observation points are obtained, the third number of observation points used in the process of determining which of the multiple target objects 200 each observation point corresponds to is less than the fourth number of observation points used in the process of projecting each of the multiple third contour attention points on third contour 300bc onto second contour 300b as multiple second contour attention points.

[0087] As a result, the observation information processing method according to the embodiment can speed up processing by using a smaller number of observation points than conventional methods for tracking the target object 200, thereby improving real-time performance.

[0088] Furthermore, for example, observation information processing device 1 according to the embodiment is an observation information processing device using extended object tracking, and includes acquisition unit 11 that acquires first observation information acquired at a first time point including observation points obtained by observing target object 200 to be tracked from observing body 100 that is the subject observing target object 200, and second observation information acquired at a second time point different from the first time point including the observation points, and a processing unit that changes the sizes of first contour 300a, which is the contour of target object 200 estimated from the observation points included in the first observation information, and second contour 300b, which is the contour estimated from the observation points included in the second observation information, so that the sizes of first contour 300a and second contour 300b are the same. The image forming apparatus includes an adjustment unit 12 that generates a third contour 300bc, and a projection unit 13 that projects the first focus point on the first contour 300a as a second focus point onto the third contour 300bc so that the relative position of the first focus point on the first side, which is the side of the first contour 300a on which the first focus point is located, is equal to the relative position of the second focus point on the second side, which is the side of the third contour 300bc on which the second focus point is located, and projects the second focus point on the third contour 300bc as a third focus point onto the second contour 300b so that the relative position of the second focus point on the second side, which is the side of the third contour 300bc on which the second focus point is located, is equal to the relative position of the third focus point on the third side, which is the side of the second contour 300b on which the third focus point is located.

[0089] As a result, the observation information processing device 1 according to the embodiment can achieve the same effects as the above observation information processing method.

[0090] (others) Although the embodiments have been described above, the present invention is not limited to the above embodiments. For example, in the above embodiments, the processing performed by a specific processing unit may be performed by another processing unit. Furthermore, the order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0091] In the above embodiments, each component may be realized by executing a software program suitable for that component using various hardware resources, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0092] Furthermore, each component may be realized by hardware that does not include a software program. For example, each component may be a circuit (or integrated circuit). These circuits may be configured as a single circuit as a whole, or may each be configured as a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0093] Furthermore, the general or specific aspects of the present invention may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, etc. Furthermore, the general or specific aspects of the present invention may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0094] For example, the present invention may be realized as a program for causing a computer to execute the observation information processing method of the above-described embodiment, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0095] In addition, the present invention also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the present invention. [Explanation of symbols]

[0096] 1. Observation information processing device 11a Acquisition part 11b Adjustment part 11c Projection section 11 Control section 12 Storage section 100 Observation Body 200 target objects 300 Contours 300a 1st contour 300b Second Contour 300ac, 300bc 3rd contour

Claims

1. 1. A method for processing observation information using extended object tracking, comprising: an acquisition step of acquiring first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is a subject observing the target object; an adjustment step of generating a third contour based on the first contour and the second contour so that a size of a first contour, which is a contour of the target object estimated from the first observation information, and a size of a second contour, which is a contour estimated from the second observation information, are the same; projecting the first attention point on the first contour as the second attention point onto the third contour so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located; a projection step of projecting the second attention point on the third contour as the third attention point onto the second contour so that a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located, is equal to a relative position of the third attention point on a third side, which is a side of the second contour on which the third attention point is located, Observation information processing methods.

2. the first point of interest is a point corresponding to an observation point included in the first observation information; The observation information processing method according to claim 1 .

3. the first observation information includes a plurality of observation points; the plurality of observation points are a plurality of first contour attention points including the first attention point, which are projected onto the first contour; projecting the first and second attention points onto the third contour so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located; projecting the second attention point onto the second contour as the third attention point so that a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located, is equal to a relative position of the third attention point on a third side, which is a side of the second contour on which the third attention point is located; 3. The observation information processing method according to claim 1 or 2.

4. The plurality of observation points include, among the observation points, a point closest to the observation body, a point at the right end as seen from the observation body, a point at the left end as seen from the observation body, and a point at the center between the right end point and the left end point. The observation information processing method according to claim 3.

5. using a second number of observation points, which is less than the first number, from the first number of observation points obtained by observing the target object; 3. The observation information processing method according to claim 1 or 2.

6. When a plurality of target objects are observed and a plurality of observation points are obtained, the third number of observation points used in the process of determining which of the plurality of target objects each observation point corresponds to is less than the fourth number of observation points used in the process of projecting, onto the second contour, each of the plurality of third contour attention points on the third contour as a plurality of second contour attention points. The observation information processing method according to claim 3.

7. 1. A method for processing observation information using extended object tracking, comprising: an acquisition step of acquiring first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is a subject observing the target object; an adjustment step of generating a third contour from the first contour, which is a contour of the target object estimated from the first observation information, so that a size of the first contour and a size of the second contour, which is a contour estimated from the second observation information, are the same; a projection step of projecting the first attention point on the first contour onto the third contour as the second attention point so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located, In the projecting step, the third contour onto which the second attention point is projected is changed to a size of the second contour while maintaining a relative position of the second attention point on the third contour. Observation information processing methods.

8. An observation information processing device using extended object tracking, a processor; Memory and an acquisition unit that acquires first observation information acquired at a first time point, the first observation information including an observation point obtained by observing a target object to be tracked from an observation body that is a subject observing the target object, and second observation information acquired at a second time point different from the first time point, the second observation information including the observation point; an adjustment unit that generates a third contour by changing the sizes of the first contour and the second contour so that a size of a first contour, which is a contour of the target object inferred from the observation points included in the first observation information, and a size of a second contour, which is a contour inferred from the observation points included in the second observation information, are the same; projecting the first attention point onto the third contour as the second attention point so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located; a projection unit that projects the second attention point onto the second contour as the third attention point so that a relative position of the second attention point on a second side that is a side of the third contour on which the second attention point is located is equal to a relative position of the third attention point on a third side that is a side of the second contour on which the third attention point is located, Observation information processing equipment.

9. A non-transitory computer-readable recording medium on which a program for causing a computer to execute the observation information processing method according to claim 1 is recorded.

Images