Mobile body identification system

Abstract

The present invention comprises: three-dimensional markers 1 disposed in a real space; a plurality of sensor devices 2 disposed in the real space; and a server device 3 connected to the sensor devices 2 in a communicable state. The three-dimensional markers 1 are labels each having a three-dimensional shape and individually disposed on the surface side of respective moving bodies M that move in the real space. The sensor devices 2 acquire image sensor data comprising a point group in the real space including the three-dimensional markers 1, and transmit the image sensor data to the server device 3. After aggregating the image sensor data comprising the point group, in the real space, transmitted from each of the sensor devices 2, the server device 3 detects the three-dimensional markers 1 in the real space and identifies the moving bodies M on which the three-dimensional markers 1 are respectively disposed.

Description

Mobile Identification System

[0001] The present invention relates to a moving object identification system that uses three-dimensional markers to identify moving objects that move in real space.

[0002] In recent years, as a global trend toward rapid urbanization, significant growth is predicted in the market for autonomous driving of various mobile objects such as automobiles, micromobility, and transport robots. Research is being conducted on tracking mobile objects using a sensor network equipped with multiple LiDAR sensors placed in the real world (see, for example, Non-Patent Documents 1 and 2).

[0003] However, previous research has not fully utilized a sensor network equipped with multiple LiDAR sensors deployed in real space to track moving objects, and various problems remain. For example, when multiple moving objects approach each other, the detection results for the moving objects may be interchanged. Furthermore, moving objects of the same type cannot be distinguished because they have the same shape in the LiDAR point cloud. Even if a moving object can be detected, at best, the instantaneous position of the moving object can be determined. Without the ability to identify each individual moving object, it is difficult to track and predict the moving object's trajectory. Furthermore, while cameras can identify individual vehicles of the same type if they are different colors, cameras cannot identify individual vehicles of the same type that are the same color.

[0004] K. Akiyama et al., “Edge computing system with multi-lidar sensor network for robustness of autonomous personal-mobility,” pp. 290-295, 2022. T. Kudo et al., “Edge system with multi-lidar sensor network for tracking micro-mobility vehicles,” pp. 983-984, 2023.

[0005] The present invention has been made in view of the above-mentioned problems, and has an object to provide a moving body identification system that can accurately identify moving bodies that move in real space.

[0006] In order to achieve the above-mentioned object, the present invention provides a system comprising: a three-dimensional marker arranged in a real space; a plurality of sensor devices arranged in the real space; and a server device connected to the sensor devices in a state capable of communicating with the sensor devices. The three-dimensional markers are signs having three-dimensional shapes that are individually arranged on the surface side of a moving object moving in the real space. The sensor devices comprise: a sensor unit that acquires image sensor data consisting of a point cloud in the real space including the three-dimensional marker; and a sensor-side transmitting unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the sensor unit to the server device. The server device comprises: a server-side receiving unit that receives the image sensor data consisting of the point cloud in the real space transmitted from each of the sensor devices; an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space received by the server-side receiving unit; a detection unit that detects the three-dimensional marker based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit; and an identification unit that identifies the moving object on which the three-dimensional marker is arranged based on the detection result of the three-dimensional marker by the detection unit.

[0007] The three-dimensional marker may have a marker portion whose position and / or orientation is detected by a three-dimensional shape in a predetermined direction.

[0008] The marker portion may have a three-dimensional shape with one or more protrusions formed along the circumferential direction.

[0009] The three-dimensional marker may also have a code portion to which predetermined identification information is given by a three-dimensional shape in a predetermined direction.

[0010] The cord portion may have a three-dimensional shape with a plurality of protrusions formed along the axial direction.

[0011] In addition, the sensor unit may be placed on a static object in real space, and when acquiring image sensor data consisting of a point cloud in real space, it may also acquire image sensor data consisting of a point cloud of the three-dimensional marker placed on a moving object moving in real space.

[0012] In addition, the sensor unit may be placed on one dynamic object moving in real space, and when acquiring image sensor data consisting of a point cloud in real space, it may also acquire image sensor data consisting of a point cloud of the three-dimensional markers placed on another moving object moving in real space.

[0013] The aggregation unit may also align the spatial coordinate axes of image sensor data consisting of point clouds in real space acquired by each sensor unit based on the detection results of the three-dimensional markers by the detection unit.

[0014] In addition, the detection unit may detect the three-dimensional marker placed in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit, by using a machine learning model that has learned the three-dimensional shape of the three-dimensional marker placed in real space by machine learning.

[0015] The server device may also include a server-side transmitting unit to which an application system is connected in a communicable state and which transmits information about the moving object identified by the identifying unit to the application system.

[0016] According to the present invention, by detecting three-dimensional markers based on image sensor data consisting of a cloud of points in real space acquired by a sensor device, it is possible to accurately identify moving objects moving in real space. As a result, it is possible to realize optimal traffic of moving objects, for example, by providing various traffic information to each moving object based on information about the identified moving object, or by automatically controlling the traffic of each moving object.

[0017] FIG. 1 is a diagram showing the overall configuration of a moving object identification system according to an embodiment of the present invention. FIG. 2 is a diagram showing the configuration of the sensor device of FIG. 1. FIG. 3 is a diagram showing the configuration of the server device of FIG. 1. FIG. 4 is a perspective view showing the marker portion of a three-dimensional marker. FIG. 5 is a perspective view showing the code portion of a three-dimensional marker. FIG. 6 is a diagram showing an example of image sensor data consisting of a point cloud in real space acquired by a sensor device. FIG. 7 is a diagram showing two spatial coordinate axes of image sensor data consisting of a point cloud in real space. FIG. 8 is a flowchart showing the operation of the moving object identification device of FIG. 1. FIG. 9 is a diagram showing an image of a moving object identification system according to Example 1. FIG. 10 is a diagram showing an image of a moving object identification system according to Example 2. FIG. 11 is a diagram showing (a) a three-dimensional marker and (b) a dummy used in an experiment of the moving object identification system according to Example 3. FIG. 12 is a diagram showing the arrangement of the sensor device and the positions of the three-dimensional markers in an experiment of the moving object identification system according to Example 3.

[0018] Next, an embodiment of a moving object identification system (hereinafter referred to as the present system) according to the present invention will be described with reference to FIGS.

[0019] As shown in Fig. 1, this system includes a three-dimensional marker 1 placed on a mobile object M, such as an automobile, micromobility, or transport robot, that moves in real space, a plurality of sensor devices 2 placed in the real space, and a server device 3 connected to the sensor devices 2 in a communicable manner, with an application system 4 connected to the server device 3 in a communicable manner. Note that the real space refers to a realistic three-dimensional space in which static objects, such as roads and walls, and the mobile object M exist. The three-dimensional marker 1, the sensor devices 2, and the server device 3 will be described in detail below.

[0020] [Configuration of Three-Dimensional Marker 1] The three-dimensional marker 1 is a sign having a three-dimensional shape used to detect a moving object M moving in real space, and is individually placed on a surface side such as the ceiling surface of the moving object M. Note that the three-dimensional marker 1 is not a normal part (for example, a simple door mirror or tail lamp) mounted on the moving object M, but is a sign dedicated to identification that is individually placed on the moving object M for the purpose of identifying the moving object M.

[0021] The three-dimensional marker 1 may also have a marker portion 11 whose position and / or direction can be detected based on its three-dimensional shape in a predetermined direction, as shown in Fig. 4. The marker portion 11 has a three-dimensional shape in which a plurality of protrusions 11b extending in the axial direction are arranged along the circumferential direction on the peripheral surface of a columnar main body 11a extending in the axial direction.

[0022] For example, the shape of the protrusion 11b of the marker unit 11 may be rectangular in plan view as shown in Fig. 4(a), approximately semicircular in plan view as shown in Fig. 4(b), or angular in plan view as shown in Fig. 4(c). This allows the detection unit 33 of the server device 3, which will be described later, to accurately detect not only the position of the three-dimensional marker 1 but also the orientation of the three-dimensional marker 1.

[0023] 5, the three-dimensional marker 1 may have a code portion 12 in which predetermined identification information can be detected by its three-dimensional shape in a predetermined direction. This code portion 12 has a three-dimensional shape in which a plurality of protrusions 12b extending in the circumferential direction are arranged along the axial direction on the peripheral surface of a columnar main body 12a extending in the axial direction. This gives the code portion 12 a three-dimensional shape with projections and recesses along the axial direction. Therefore, by corresponding the three-dimensional shape of the projections and recesses to a numerical value (e.g., by assigning 0 to a recess and 1 to a protrusion), identification information can be assigned to the three-dimensional marker.

[0024] For example, as shown in Fig. 5(a), when annular protrusions 12b are provided at the middle and lower ends of the circumferential surface of the main body 12a, the code portion 12 has a three-dimensional shape of a recess, a protrusion, a recess, and a protrusion, from top to bottom along the axial direction, and thus the identification information 0101 can be assigned to the three-dimensional marker 1. Also, as shown in Fig. 5(b), when annular protrusions 12b are provided at the upper and lower ends of the circumferential surface of the main body 12a, the code portion 12 has a three-dimensional shape of a protrusion, a recess, a recess, and a protrusion, from top to bottom along the axial direction, and thus the identification information 1001 can be assigned to the three-dimensional marker 1. Also, as shown in Fig. 5(c), when protrusions 12a are provided on the entire circumferential surface of the main body 12a, the code portion 12 has a three-dimensional shape of a protrusion, a protrusion, a protrusion, and a protrusion, from top to bottom along the axial direction, and thus the identification information 1111 can be assigned to the three-dimensional marker 1. Furthermore, as shown in Figure 5(d), if the code portion 12 does not have protrusions 12b on the entire peripheral surface 12a of the main body portion, it will have a three-dimensional shape of recess, recess, recess, recess from top to bottom along the axial direction, and therefore the identification information 0000 can be similarly assigned to the three-dimensional marker 1.

[0025] The three-dimensional marker 1 is usually placed so that the lower surface in the axial direction of Figures 4 and 5 is the installation surface and extends vertically from the surface of the moving body M, but other placement configurations are also possible.

[0026] [Configuration of sensor device 2] The sensor device 2 is installed in real space on a static object such as a moving body M or a support pillar, and as shown in FIG. 2 , includes a sensor unit 21 that acquires image sensor data consisting of a point cloud in real space including a three-dimensional marker 1, and a terminal device 22 that performs predetermined processing on the image sensor data consisting of the point cloud in real space acquired by the sensor unit 21 and transmits it to the server device 3.

[0027] The sensor unit 21 is a so-called LiDAR (light detection and ranging) sensor. LiDAR is a type of sensor that uses laser light, and by scanning and irradiating an object with short-wavelength laser light, which has a higher radiant flux density than radio waves, the sensor acquires image sensor data consisting of a point cloud in real space and accurately detects not only the distance to the object but also the position and shape of the object.

[0028] Examples of this LiDAR include a method of acquiring image sensor data in all directions of 360 degrees by rotating light emitters that emit multiple laser beams, and a method of acquiring image sensor data by directly irradiating laser beams within a predetermined light irradiation angle range. Furthermore, the more light emitters that emit laser beams, the higher the accuracy of the image sensor data, but the higher the cost, so an inexpensive LiDAR with a small number of light emitters may be used.

[0029] Furthermore, when image sensor data is acquired using one sensor device 2, there is a risk that a visual area will be generated on the opposite side of the wall or moving body M as viewed from the sensor device 2, so it is preferable to provide multiple sensor devices 2 so as to acquire image sensor data consisting of point clouds from different directions relative to the same real space.

[0030] 2, the terminal device 22 is composed of a capture unit 221, a storage unit 222, a filter unit 223, and a sensor-side transmission unit 224. Of these, the capture unit 221 captures image sensor data consisting of a point cloud received from the sensor unit 21 and temporarily stores the data in the storage unit 222 at predetermined scan intervals. The filter unit 223 receives image sensor data consisting of a point cloud in real space from the storage unit 222 and extracts image sensor data consisting of a point cloud in a target area in real space where an object exists. The sensor-side transmission unit 224 fragments the image sensor data consisting of the point cloud in the target area in real space extracted by the filter unit 223 into packets and transmits the fragmented data to the server device 3.

[0031] In this embodiment, LiDAR is used as the sensor unit 11, but other sensors may be used as long as they can acquire image sensor data consisting of a point cloud.

[0032] [Configuration of Server Device 3] As shown in FIG. 3, the server device 3 is configured with a server-side receiving unit 31, an aggregating unit 32, a detecting unit 33, an identifying unit 34, and a server-side transmitting unit 35.

[0033] The server-side receiving unit 31 receives image sensor data consisting of a group of points in real space transmitted from each sensor device 2 .

[0034] The aggregation unit 32 aggregates the image sensor data consisting of point clouds in real space received by the server-side receiving unit 31. Specifically, the aggregation unit 32 aggregates the image sensor data consisting of point clouds in real space transmitted from each sensor device 2 by synthesizing them in chronological order based on timestamps and aligning them in three-dimensional space. The image sensor data consisting of point clouds before and after aggregation may be stored in a storage device (not shown). Furthermore, when aggregating the image sensor data consisting of point clouds in real space, the aggregation unit 32 may perform other processes such as smoothing of the point clouds of the image sensor data.

[0035] The detection unit 33 detects the three-dimensional marker 1 placed on the moving body M moving in real space, based on the image sensor data consisting of a point cloud in real space aggregated by the aggregation unit 32. Specifically, the detection unit 33 detects the position and orientation of the three-dimensional marker 1 from a three-dimensional shape in a predetermined direction (for example, the three-dimensional shape in the circumferential direction of the marker unit 11 shown in FIG. 4), or detects a three-dimensional shape in another predetermined direction (for example, the three-dimensional shape in the axial direction of the code unit 12 shown in FIG. 5).

[0036] For example, Figure 6 shows an example of image sensor data consisting of a cloud of points across a certain real space acquired by the sensor device 2. Since a three-dimensional marker 1 is present in the center of the image sensor data (the area surrounded by a circle), the detection unit 33 detects the three-dimensional marker 1 from the image sensor data consisting of a cloud of points across the real space.

[0037] The identification unit 34 identifies the moving body M on which the three-dimensional marker 1 is placed, based on the detection result of the three-dimensional marker 1 by the detection unit 33. Specifically, as shown in Fig. 6, when the detection unit 33 detects the position of the three-dimensional marker 1 in real space, the identification unit 34 can identify the position of the moving body M on which the three-dimensional marker 1 is placed in real space (the center of real space).

[0038] Furthermore, when the detection unit 33 detects the orientation of the three-dimensional shape of the marker unit 11 of the three-dimensional marker 1, the identification unit 34 can identify the direction (e.g., the direction of travel) of the moving body M on which the three-dimensional marker 1 is placed in real space. For example, as shown in Fig. 4, if the marker unit 11 of the three-dimensional marker 1 has a three-dimensional shape in which a columnar main body 11a extending in the axial direction has a plurality of protrusions 11b extending in the axial direction arranged side by side on the circumferential surface, the detection unit 33 detects the orientation of the protrusions 11b of the marker unit 11 of the three-dimensional marker 1, and the identification unit 34 can identify the direction of the moving body M.

[0039] Furthermore, when the detection unit 33 detects the three-dimensional shape of the code portion 12 of the three-dimensional marker 1, the identification unit 34 can identify the identification information of the moving body M on which the three-dimensional marker 1 is placed in real space. For example, as shown in Fig. 5, when the code portion 12 of the three-dimensional marker 1 has a three-dimensional shape in which a plurality of protrusions 12b extending circumferentially are arranged along the axial direction on the peripheral surface of a columnar main body 12a extending axially, the detection unit 33 detects the arrangement of the projections and recesses caused by the protrusions 12b of the code portion 12 of the three-dimensional marker 1, and the identification unit 34 can identify the identification information of the moving body M.

[0040] The server-side transmitting unit 35 transmits information about the moving object M (the position, direction or identification information of the moving object M) to the application system 4 .

[0041] In this way, it is possible to accurately identify moving bodies M moving in real space by detecting the three-dimensional marker 1 based on image sensor data consisting of a point cloud in real space acquired by the sensor device 2. Therefore, for example, the application system 4 can provide various traffic information to each moving body M based on information about the identified moving body M, or automatically control the traffic of each moving body M, thereby realizing optimal traffic for the moving bodies M.

[0042] The aggregation unit 32 may align the spatial coordinate axes of the image sensor data, which are composed of point clouds in real space acquired by each sensor unit 21, based on the detection results of the three-dimensional marker 1 by the detection unit 33. Specifically, because the spatial coordinate axes of each sensor unit 21 arranged throughout real space are different, it is necessary to align these spatial coordinate axes (this is generally called calibration or registration; see FIG. 7 ). To align these spatial coordinate axes, the real-space point clouds acquired by each sensor unit 21 must overlap. However, since the reason for installing multiple sensor units 21 is to expand the coverage area of ​​the axis space, it is not possible to overlap the real-space point clouds to a large extent. Therefore, when the aggregation unit 32 aligns the spatial coordinate axes of the image sensor data, which are composed of point clouds in real space acquired by each sensor unit 21, based on the detection results of the three-dimensional marker 1 by the detection unit 33, it is possible to accurately align the positions of the spatial coordinate axes without significant overlap of the point clouds of the sensor units 21.

[0043] In addition, the detection unit 33 may detect the three-dimensional marker 1 placed in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit 32, by using a machine learning model that has learned the three-dimensional shape of the three-dimensional marker 1 by machine learning.

[0044] [Operation of the Present System] Next, the flow of the object identification method by the present system will be described with reference to the flowchart shown in Fig. 8. In the following description, "step" will be abbreviated as "S".

[0045] First, in the sensor device 2, the sensor unit 21 acquires image sensor data consisting of a group of points in the real space including the three-dimensional marker 1 (S1).

[0046] Then, the terminal device 22 captures the image sensor data consisting of the point cloud received from the sensor unit 21, temporarily stores it in the storage unit 222 at each specified scanning period, receives the image sensor data consisting of the point cloud in real space from the storage unit 222, extracts the image sensor data consisting of the point cloud in the target area where the object in real space exists, and then fragments the image sensor data consisting of the point cloud in the target area in real space extracted by the filter unit 223 into packets and transmits them to the server device 3 (S2).

[0047] Next, in the server device 3, the server-side receiving unit 31 receives the image sensor data consisting of a group of points in real space transmitted from each sensor device 2 (S3).

[0048] Then, the aggregation unit 32 aggregates the image sensor data consisting of a group of points in real space received by the server-side receiving unit 31 (S4).

[0049] Then, the detection unit 33 detects the three-dimensional marker 1 placed on the moving body M moving in the real space based on the image sensor data consisting of a cloud of points in the real space aggregated by the aggregation unit 32 (S5).

[0050] Then, the identification unit 34 identifies the moving object M on which the three-dimensional marker 1 is placed, based on the detection result of the three-dimensional marker 1 by the detection unit 33 (S6).

[0051] Then, the server-side transmitting unit 35 transmits information about the moving object M (position, direction or identification information of the moving object M) to the application system 4 (S7).

[0052] Next, the application system 4 can provide various traffic information to each mobile body M based on the information about the identified mobile body M, and automatically control the traffic of each mobile body M, thereby realizing optimal traffic for the mobile body M (S8).

[0053] <Example 1> In Example 1, as shown in FIG. 9 , the sensor device 2 is placed at various locations on a static object in real space, and when acquiring image sensor data consisting of a point cloud in real space, the sensor device 2 acquires image sensor data consisting of a point cloud of a three-dimensional marker 1 placed on a moving body M (e.g., an autonomous vehicle) moving in real space.

[0054] In addition, the server device 3 aggregates image sensor data consisting of point clouds in real space transmitted from each sensor device 2, and then detects a three-dimensional marker 1 based on the aggregated image sensor data consisting of point clouds in real space, and identifies the moving body M on which the three-dimensional marker 1 is placed based on the detection result of the three-dimensional marker 1 by the detection unit 33.

[0055] The application system 4 provides various traffic information to each mobile body M based on the information about the mobile body M transmitted from the server device 3, and automatically controls the traffic of each mobile body M.

[0056] <Example 2> In Example 2, as shown in FIG. 10 , the sensor device 2 is placed on one moving body M (e.g., an electric wheelchair) moving in real space, and when acquiring image sensor data consisting of a point cloud in real space, it acquires image sensor data consisting of a point cloud of a three-dimensional marker 1 placed on another moving body M (e.g., another electric wheelchair) moving in real space.

[0057] In addition, the server device 3 aggregates image sensor data consisting of point clouds in real space transmitted from each sensor device 2, and then detects a three-dimensional marker 1 based on the aggregated image sensor data consisting of point clouds in real space, and identifies the moving body M on which the three-dimensional marker 1 is placed based on the detection result of the three-dimensional marker 1 by the detection unit 33.

[0058] The application system 4 provides various traffic information to each mobile unit M based on the information about the mobile unit M transmitted from the server device 3, and automatically controls the traffic of each movement.

[0059] Example 3 In this example 3, an experiment on the detection of the three-dimensional marker 1 will be described.

[0060] [Experimental Setup] In this experiment, a prototype three-dimensional marker 1 shown in Figure 11(a) was produced. The height of the three-dimensional marker 1 was 300 mm. The three-dimensional marker 1 was produced using extruded polystyrene foam made of styrene material. The three-dimensional marker 1 exhibits rotational symmetry and returns to the same shape after a 60-degree rotation. The three-dimensional marker 1 is composed of six convex portions and six concave portions, with the convex portions having a radius of 100 mm and the concave portions having a radius of 65 mm. The concave portion of the three-dimensional marker 1 has an arc of 44.44 degrees, and the convex portion has an arc of 15.56 degrees.

[0061] Figure 12 shows the placement of the sensor device 2 and the position of the three-dimensional marker 1 in the experiment. The sensor device 2 was installed at the position indicated by the dot in Figure 12. The sensor device 2 was installed approximately 130 cm above the ground. In the experiment, the three-dimensional marker 1 was placed at the position indicated by the x in Figure 12. The three-dimensional marker 1 was placed 1.35 m above the ground. The position of the three-dimensional marker 1 was changed as shown in Figure 12 at points 3.0 m, 4.5 m, 6.0 m, 7.5 m, and 9.0 m from the sensor device 2. At each installation point, the three-dimensional marker 1 was rotated 10 degrees to capture data in six different directions. A Velodyne VLP-16 LIDAR unit was used to acquire the point cloud data. An NVIDIA Jetson Nano was used as the terminal device 22. An NVIDIA Jetson NX was used as the server device 3, and the acquired data was stored on a connected HDD.

[0062] [Experimental Data Acquisition] The data acquisition process followed the steps outlined below. As shown in Figure 12, a three-dimensional marker 1 (K = 1) was placed at five distances (n = 5) from the sensor device 2. The three-dimensional marker 1 was rotated up to 60 degrees (m = 6) in 10-degree increments, and data was acquired at each position. Data was acquired at a rate of one frame every 110 milliseconds, for a total of 2,500 frames. 250 frames were captured for each pattern, resulting in a total of 75,000 frames of point cloud data. OpenPCDet was also employed as the machine learning tool. Three deep learning models, PointPillars, SECOND, and PointRCNN, were used for object detection in this experiment. 80% and 20% of the data obtained in the experiment were used for training and testing, respectively.

[0063] [Experimental Results] We use average precision (AP) and (AP|R40) as evaluation metrics. Table 1 shows AP and AP|R40 as the detection accuracy of the three deep learning models. This table suggests that PointRCNN and SECOND performed well with an accuracy of approximately 99%.

[0064]

[0065] Next, we evaluated the system from the perspective of false positives. We used the dummy object shown in Figure 11(b). The experimental setup for the dummy object was essentially the same as for 3D marker 1, except that the number of frames for each acquired pattern was set to 300 for the dummy object. We counted the number of false positives detected during the detection process. The number of false positives was 11 for PointPillars, 1,062 for PointRCNN, and 23 for SECOND. This suggests that PointPillars performed well in terms of false positives, despite its low accuracy. SECOND's overall performance was high in the experiment.

[0066] <Example 4> [Setting the conditions for the experiment and analysis (equipment, location, parameters used)] ・Equipment used: Livox avia * Jetson orin nano 3D marker ・Experiment location: Shibaura Institute of Technology Toyosu Campus Exchange Foyer ・Library used: OpenPCDet ・Method used: Centerpoint Parameter: epoch: 1000 Number of frames used: 26289 frames

[0067] [Explanation of the results in Table 2] From the prepared point cloud data, 80% was extracted as training data, and the remaining 20% ​​was used as test data. These are the AP and AP|R40 (%) at IoU thresholds of 0.5 and 0.7. Here, AP is the average precision, and AP|R40 is the method of dividing the recall axis of the precision-recall curve into 40 parts when the IoU threshold is 0.7 or higher, calculating the precision at each point, and averaging them.

[0068] [Practical benefits to society from the results] By building a generalized detection model using multiple LiDARs, the detection accuracy of 3D markers in practical environments will be further improved.

[0069] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the illustrated embodiments. Various modifications and variations can be made to the illustrated embodiments within the same scope as the present invention or within an equivalent scope.

[0070] REFERENCE SIGNS LIST 1... Three-dimensional marker 11... Marker unit 12... Code unit 2... Sensor device 21... Sensor unit 22... Terminal device 221... Capture unit 222... Storage unit 223... Filter unit 224... Sensor-side transmission unit 3... Server device 31... Server-side reception unit 32... Aggregation unit 33... Detection unit 34... Identification unit 35... Server-side transmission unit 4... Application system M... Mobile body

Claims

1. A moving object identification system comprising: a three-dimensional marker arranged in real space; a plurality of sensor devices also arranged in the real space; and a server device connected to the sensor devices in a state capable of communicating with the sensor devices, wherein the three-dimensional markers are signs having three-dimensional shapes individually arranged on the surface side of a moving object moving in the real space, the sensor devices comprising: a sensor unit that acquires image sensor data consisting of a point cloud in real space including the three-dimensional marker; and a sensor-side transmitting unit that transmits the image sensor data consisting of the point cloud in real space acquired by the sensor unit to the server device, and the server device comprising: a server-side receiving unit that receives the image sensor data consisting of the point cloud in real space transmitted from each of the sensor devices; an aggregating unit that aggregates the image sensor data consisting of the point cloud in real space received by the server-side receiving unit; a detection unit that detects the three-dimensional marker based on the image sensor data consisting of the point cloud in real space aggregated by the aggregating unit; and an identification unit that identifies the moving object on which the three-dimensional marker is arranged based on the detection result of the three-dimensional marker by the detection unit.

2. A moving body identification system according to claim 1, wherein the three-dimensional marker has a marker portion whose position and / or orientation is detected by a three-dimensional shape in a predetermined direction.

3. A moving body identification system according to claim 2, wherein the marker portion has a three-dimensional shape with one or more protrusions formed along the circumferential direction.

4. A moving body identification system according to claim 1, wherein the three-dimensional marker has a code portion to which predetermined identification information is given by a three-dimensional shape in a predetermined direction.

5. A mobile unit identification system according to claim 4, wherein the code portion has a three-dimensional shape with a plurality of protrusions formed along the axial direction.

6. A moving body identification system according to any one of claims 1 to 5, wherein the sensor unit is arranged on a static object in real space, and when acquiring image sensor data consisting of a point cloud in real space, the sensor unit acquires image sensor data consisting of a point cloud of the three-dimensional marker arranged on a moving body moving in real space.

7. A moving body identification system according to any one of claims 1 to 5, wherein the sensor unit is arranged on a dynamic object moving in real space, and when acquiring image sensor data consisting of a point cloud in real space, it acquires image sensor data consisting of a point cloud of the three-dimensional markers arranged on another moving body moving in real space.

8. A moving object identification system according to any one of claims 1 to 5, wherein the aggregation unit aligns the spatial coordinate axes of image sensor data consisting of point clouds in real space acquired by each sensor unit based on the detection results of the three-dimensional markers by the detection unit.

9. A moving object identification system as described in any one of claims 1 to 5, wherein the detection unit detects the three-dimensional marker placed in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit, by using a machine learning model that has learned the three-dimensional shape of the three-dimensional marker placed in real space by machine learning.

10. A mobile object identification system as described in any one of claims 1 to 5, wherein the server device is connected to an application system in a communicable state and is provided with a server-side transmitting unit that transmits information about the mobile object identified by the identifying unit to the application system.

Images