Mobile body identification system

Abstract

To provide a mobile body identification system that can accurately identify mobile bodies moving in a real space.SOLUTION: A mobile body identification system comprises: three-dimensional markers 1 disposed in a real space; a plurality of sensor devices 2 disposed in the real space; and a server device 3 connected to the sensor devices 2 in a communicable state. The three-dimensional markers 1 are labels each having a three-dimensional shape and individually disposed on the surface side of respective moving bodies M that move in the real space. The sensor devices 2 acquire image sensor data comprising a point group in the real space including the three-dimensional markers 1, and transmit the image sensor data to the server device 3. After aggregating the image sensor data comprising the point group, in the real space, transmitted from each of the sensor devices 2, the server device 3 detects the three-dimensional markers 1 in the real space and identifies the moving bodies M on which the three-dimensional markers 1 are respectively disposed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a moving object identification system that uses three-dimensional markers to identify moving objects that move in real space. [Background technology]

[0002] In recent years, as a global trend toward rapid urbanization, significant growth is predicted in the market for autonomous driving of various mobile vehicles, such as automobiles, micromobility, and transport robots. Research is being conducted into tracking mobile vehicles using sensor networks equipped with multiple LiDAR sensors placed in real space (see, for example, Non-Patent Documents 1 and 2).

[0003] However, previous research has not fully utilized sensor networks equipped with multiple LiDAR sensors deployed in real space to track moving objects, and various problems remain. For example, when multiple moving objects approach each other, the detection results for the moving objects may be interchanged. Furthermore, moving objects of the same type cannot be distinguished because they have the same shape in the LiDAR point cloud. Even if a moving object can be detected, at best, the object's instantaneous position can be determined. Without the ability to identify each individual moving object, tracking and predicting the moving object's trajectory is difficult. Furthermore, while cameras can identify individual vehicles of the same type if they are different colors, cameras cannot identify individual vehicles of the same type that are the same color. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] K. Akiyama et al., “Edge computing system with multi-lidar sensor network for robustness of autonomous personal-mobility,” pp. 290-295, 2022. [Non-patent document 2] T. Kudo et al., “Edge system with multi-lidar sensor network for tracking micro-mobility vehicles,” pp. 983-984, 2023. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above-mentioned problems, and has an object to provide a moving body identification system that can accurately identify moving bodies that move in real space. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the present invention provides a system comprising: a three-dimensional marker arranged in a real space; a plurality of sensor devices arranged in the real space; and a server device connected to the sensor devices in a state capable of communicating with the sensor devices. The three-dimensional markers are signs having three-dimensional shapes that are individually arranged on the surface side of a moving object moving in the real space. The sensor devices comprise: a sensor unit that acquires image sensor data consisting of a point cloud in the real space including the three-dimensional marker; and a sensor-side transmitting unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the sensor unit to the server device. The server device comprises: a server-side receiving unit that receives the image sensor data consisting of the point cloud in the real space transmitted from each of the sensor devices; an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space received by the server-side receiving unit; a detection unit that detects the three-dimensional marker based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit; and an identification unit that identifies the moving object on which the three-dimensional marker is arranged based on the detection result of the three-dimensional marker by the detection unit.

[0007] The three-dimensional marker may have a marker portion whose position and / or orientation is detected by a three-dimensional shape in a predetermined direction.

[0008] The marker portion may have a three-dimensional shape with one or more protrusions formed along the circumferential direction.

[0009] The three-dimensional marker may also have a code portion to which predetermined identification information is given by a three-dimensional shape in a predetermined direction.

[0010] The cord portion may have a three-dimensional shape with a plurality of protrusions formed along the axial direction.

[0011] In addition, the sensor unit may be placed on a static object in real space, and when acquiring image sensor data consisting of a point cloud in real space, it may also acquire image sensor data consisting of a point cloud of the three-dimensional marker placed on a moving object moving in real space.

[0012] In addition, the sensor unit may be placed on one dynamic object moving in real space, and when acquiring image sensor data consisting of a point cloud in real space, it may also acquire image sensor data consisting of a point cloud of the three-dimensional markers placed on another moving object moving in real space.

[0013] The aggregation unit may also align the spatial coordinate axes of image sensor data consisting of point clouds in real space acquired by each sensor unit based on the detection results of the three-dimensional markers by the detection unit.

[0014] In addition, the detection unit may detect the three-dimensional marker placed in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit, by using a machine learning model that has learned the three-dimensional shape of the three-dimensional marker placed in real space by machine learning.

[0015] The server device may also include a server-side transmitting unit to which an application system is connected in a communicable state and which transmits information about the moving object identified by the identifying unit to the application system. [Effects of the Invention]

[0016] According to the present invention, by detecting three-dimensional markers based on image sensor data consisting of a cloud of points in real space acquired by a sensor device, it is possible to accurately identify moving objects moving in real space. As a result, it is possible to realize optimal traffic of moving objects, for example, by providing various traffic information to each moving object based on information about the identified moving object, or by automatically controlling the traffic of each moving object. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a diagram showing the overall configuration of a moving object identification system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a configuration of the sensor device of FIG. [Figure 3] FIG. 2 is a diagram illustrating a configuration of a server device in FIG. [Figure 4] FIG. 2 is a perspective view showing a marker portion of a three-dimensional marker. [Figure 5] FIG. 2 is a perspective view showing a code portion of the three-dimensional marker. [Figure 6] FIG. 1 is a diagram illustrating an example of image sensor data consisting of a point cloud in real space acquired by a sensor device. [Figure 7] FIG. 1 is a diagram showing two spatial coordinate axes of image sensor data consisting of a point cloud in real space. [Figure 8] 2 is a flowchart showing the operation of the moving object identification device of FIG. [Figure 9] FIG. 1 is a diagram illustrating an image of a moving body identification system according to a first embodiment. [Figure 10] FIG. 10 is a diagram showing an image of a moving body identification system according to a second embodiment. [Figure 11]10A and 10B are diagrams showing a three-dimensional marker and a dummy used in an experiment of a moving body identification system according to a third embodiment. [Figure 12] 10 is a diagram showing the arrangement of sensor devices and the positions of three-dimensional markers in an experiment of a moving body identification system according to a third embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] Next, an embodiment of a moving body identification system (hereinafter referred to as the present system) according to the present invention will be described with reference to FIGS.

[0019] As shown in Fig. 1, this system includes a three-dimensional marker 1 placed on a moving body M such as an automobile, micromobility, or transport robot that moves in real space, a plurality of sensor devices 2 placed in the real space, and a server device 3 connected to the sensor devices 2 in a communicable state, with an application system 4 connected to the server device 3 in a communicable state. Note that the real space refers to a real three-dimensional space where static objects such as roads and walls and the moving body M exist. The three-dimensional marker 1, the sensor devices 2, and the server device 3 will be described in detail below.

[0020] [Configuration of 3D marker 1] The three-dimensional markers 1 are signs having a three-dimensional shape that are used to detect a moving object M moving in real space, and are individually placed on the surface side, such as the ceiling surface, of the moving object M. Note that these three-dimensional markers 1 are not ordinary parts (such as simple door mirrors or tail lamps) that are mounted on the moving object M, but are signs specifically for identification that are individually placed on the moving object M for the purpose of identifying the moving object M.

[0021] Furthermore, the three-dimensional marker 1 may have a marker portion 11 whose position and / or direction can be detected by its three-dimensional shape in a predetermined direction, as shown in Fig. 4. This marker portion 11 has a three-dimensional shape in which a plurality of protrusions 11b extending in the axial direction are arranged along the circumferential direction on the peripheral surface of a columnar main body portion 11a extending in the axial direction.

[0022] For example, the shape of the protrusion 11b of the marker unit 11 may be rectangular in plan view as shown in Fig. 4(a), approximately semicircular in plan view as shown in Fig. 4(b), or angular in plan view as shown in Fig. 4(c). This allows the detection unit 33 of the server device 3, which will be described later, to accurately detect not only the position of the three-dimensional marker 1 but also the orientation of the three-dimensional marker 1.

[0023] 5, the three-dimensional marker 1 may have a code portion 12 in which predetermined identification information can be detected by its three-dimensional shape in a predetermined direction. This code portion 12 has a three-dimensional shape in which a plurality of protrusions 12b extending in the circumferential direction are arranged along the axial direction on the peripheral surface of a columnar main body 12a extending in the axial direction. This gives the code portion 12 a three-dimensional shape with concave and convex shapes along the axial direction, and by associating the three-dimensional shape of the concave and convex shapes with numerical values ​​(for example, by assigning 0 to concave and 1 to convex), identification information can be assigned to the three-dimensional marker.

[0024] For example, as shown in FIG. 5( a), when annular protrusions 12b are provided at the middle and lower ends of the circumferential surface of the main body 12a, the code portion 12 has a three-dimensional shape of a recess, a protrusion, a recess, and a protrusion, from top to bottom along the axial direction, making it possible to assign identification information 0101 to the three-dimensional marker 1. Also, as shown in FIG. 5( b), when annular protrusions 12b are provided at the upper and lower ends of the circumferential surface of the main body 12a, the code portion 12 has a three-dimensional shape of a protrusion, a recess, a recess, and a protrusion, from top to bottom along the axial direction, making it possible to similarly assign identification information 1001 to the three-dimensional marker 1. Also, as shown in FIG. 5( c), when protrusions 12a are provided over the entire circumferential surface of the main body 12a, the code portion 12 has a three-dimensional shape of a protrusion, a protrusion, a protrusion, and a protrusion, from top to bottom along the axial direction, making it possible to similarly assign identification information 1111 to the three-dimensional marker 1. Furthermore, as shown in Figure 5(d), if the code portion 12 does not have protrusions 12b on the entire peripheral surface 12a of the main body portion, it will have a three-dimensional shape of recess, recess, recess, recess from top to bottom along the axial direction, and therefore the identification information 0000 can be similarly assigned to the three-dimensional marker 1.

[0025] The three-dimensional marker 1 is usually placed so that the lower surface in the axial direction of FIGS. 4 and 5 is the installation surface and extends vertically from the surface of the moving body M, but other placement configurations are also possible.

[0026] [Configuration of sensor device 2] The sensor device 2 is installed in real space on a static object such as a moving body M or a support pillar, and as shown in Figure 2, it comprises a sensor unit 21 that acquires image sensor data consisting of a point cloud in real space including a three-dimensional marker 1, and a terminal device 22 that performs predetermined processing on the image sensor data consisting of the point cloud in real space acquired by the sensor unit 21 and transmits it to the server device 3.

[0027] The sensor unit 21 is a so-called LiDAR (light detection and ranging) sensor. This LiDAR is a type of sensor that uses laser light, and by scanning and irradiating an object with laser light having a higher radiant flux density than radio waves and a short wavelength, it acquires image sensor data consisting of a point cloud in real space and accurately detects not only the distance to the object but also the position and shape of the object.

[0028] Examples of this LiDAR include a method of acquiring image sensor data in all directions of 360 degrees by rotating a light emitter that emits multiple laser beams, and a method of acquiring image sensor data by directly irradiating laser beams within a predetermined light irradiation angle range. Also, the more light emitters that emit laser beams, the higher the accuracy of the image sensor data, but the higher the cost, so an inexpensive LiDAR with a small number of light emitters may be used.

[0029] Furthermore, when image sensor data is acquired using one sensor device 2, there is a risk that a visual area will be generated on the opposite side of the wall or moving body M as viewed from the sensor device 2, so it is preferable to provide multiple sensor devices 2 so as to acquire image sensor data consisting of point clouds from different directions relative to the same real space.

[0030] 2, the terminal device 22 is composed of a capture unit 221, a storage unit 222, a filter unit 223, and a sensor-side transmission unit 224. Of these, the capture unit 221 captures image sensor data consisting of a point cloud received from the sensor unit 21 and temporarily stores it in the storage unit 222 at predetermined scan intervals. The filter unit 223 also receives image sensor data consisting of a point cloud in real space from the storage unit 222 and extracts image sensor data consisting of a point cloud in a target area where a real-space object exists. The sensor-side transmission unit 224 also fragments the image sensor data consisting of a point cloud in the target area in real space extracted by the filter unit 223 into packets and transmits them to the server device 3.

[0031] In this embodiment, a LiDAR is used as the sensor unit 11, but other sensors may be used as long as they can acquire image sensor data consisting of a point cloud.

[0032] [Configuration of server device 3] As shown in FIG. 3, the server device 3 comprises a server-side receiving unit 31, an aggregating unit 32, a detecting unit 33, an identifying unit , and a server-side transmitting unit .

[0033] The server-side receiving unit 31 receives image sensor data consisting of a group of points in real space transmitted from each sensor device 2.

[0034] The aggregation unit 32 aggregates the image sensor data consisting of point clouds in real space received by the server-side receiving unit 31. Specifically, the aggregation unit 32 aggregates the image sensor data consisting of point clouds in real space transmitted from each sensor device 2 by synthesizing them in chronological order based on timestamps and aligning them in three-dimensional space. The image sensor data consisting of point clouds before and after aggregation may be stored in a storage device (not shown). Furthermore, when aggregating the image sensor data consisting of point clouds in real space, the aggregation unit 32 may perform other processes such as smoothing of the point clouds of the image sensor data.

[0035] The detection unit 33 detects the three-dimensional marker 1 placed on the moving object M moving in real space based on image sensor data consisting of a point cloud in real space aggregated by the aggregation unit 32. Specifically, the detection unit 33 detects the position and orientation of the three-dimensional marker 1 from the three-dimensional shape in a predetermined direction (for example, the three-dimensional shape in the circumferential direction of the marker unit 11 shown in FIG. 4), or detects the three-dimensional shape in another predetermined direction (for example, the three-dimensional shape in the axial direction of the code unit 12 shown in FIG. 5).

[0036] For example, Figure 6 shows an example of image sensor data consisting of a cloud of points across a certain real space acquired by the sensor device 2. Since a three-dimensional marker 1 is present in the center of the image sensor data (the area surrounded by a circle), the detection unit 33 detects the three-dimensional marker 1 from the image sensor data consisting of a cloud of points across the real space.

[0037] The identification unit 34 identifies the moving body M on which the three-dimensional marker 1 is placed, based on the detection result of the three-dimensional marker 1 by the detection unit 33. Specifically, as shown in Fig. 6, when the detection unit 33 detects the position of the three-dimensional marker 1 in real space, the identification unit 34 can identify the position of the moving body M on which the three-dimensional marker 1 is placed in real space (the center of real space).

[0038] Furthermore, when the detection unit 33 detects the orientation of the three-dimensional shape of the marker unit 11 of the three-dimensional marker 1, the identification unit 34 can identify the direction (e.g., the direction of travel) of the moving object M on which the three-dimensional marker 1 is placed in real space. For example, as shown in Fig. 4, when the marker unit 11 of the three-dimensional marker 1 has a three-dimensional shape in which a columnar main body 11a extending in the axial direction has a plurality of protrusions 11b extending in the axial direction on the circumferential surface thereof lined up along the circumferential direction, the detection unit 33 detects the orientation of the protrusions 11b of the marker unit 11 of the three-dimensional marker 1, and the identification unit 34 can identify the direction of the moving object M.

[0039] Furthermore, when the detection unit 33 detects the three-dimensional shape of the code portion 12 of the three-dimensional marker 1, the identification unit 34 can identify the identification information of the moving body M on which the three-dimensional marker 1 is placed in real space. For example, as shown in Fig. 5, when the code portion 12 of the three-dimensional marker 1 has a three-dimensional shape in which a plurality of protrusions 12b extending circumferentially are arranged along the axial direction on the peripheral surface of a columnar main body 12a extending axially, the detection unit 33 detects the arrangement of the protrusions and recesses caused by the protrusions 12b of the code portion 12 of the three-dimensional marker 1, and the identification unit 34 can identify the identification information of the moving body M.

[0040] The server-side transmitting unit 35 transmits information about the moving object M (the position, direction or identification information of the moving object M) to the application system 4.

[0041] In this way, it is possible to accurately identify the moving body M moving in the real space by detecting the three-dimensional marker 1 based on image sensor data consisting of a point cloud in the real space acquired by the sensor device 2. Therefore, for example, the application system 4 can provide various traffic information to each moving body M based on the information about the identified moving body M, or automatically control the traffic of each moving body M, thereby realizing optimal traffic for the moving body M.

[0042] The aggregation unit 32 may align the spatial coordinate axes of the image sensor data, which is made up of point clouds in real space acquired by each sensor unit 21, based on the detection results of the three-dimensional marker 1 by the detection unit 33. Specifically, because the spatial coordinate axes of each sensor unit 21 arranged at various locations in real space are different, it is necessary to align these spatial coordinate axes (this is generally called calibration or registration; see FIG. 7 ). To align these spatial coordinate axes, the real-space point clouds acquired by each sensor unit 21 must overlap. However, since the reason for installing multiple sensor units 21 is to expand the coverage area of ​​the axis space, it is not possible to overlap the real-space point clouds to a large extent. Therefore, when the aggregation unit 32 aligns the spatial coordinate axes of the image sensor data, which is made up of point clouds in real space acquired by each sensor unit 21, based on the detection results of the three-dimensional marker 1 by the detection unit 33, it is possible to accurately align the positions of the spatial coordinate axes without significant overlap of the point clouds of the sensor units 21.

[0043] In addition, the detection unit 33 may detect the three-dimensional marker 1 placed in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit 32, by using a machine learning model that has learned the three-dimensional shape of the three-dimensional marker 1 by machine learning.

[0044] [System Operation] Next, the flow of the object identification method using this system will be described with reference to the flowchart shown in Figure 8. In the following description, "step" will be abbreviated as "S".

[0045] First, in the sensor device 2, the sensor unit 21 acquires image sensor data consisting of a group of points in the real space including the three-dimensional marker 1 (S1).

[0046] Then, the terminal device 22 captures the image sensor data consisting of the point cloud received from the sensor unit 21, temporarily stores it in the storage unit 222 at each predetermined scan period, receives the image sensor data consisting of the point cloud in real space from the storage unit 222, extracts the image sensor data consisting of the point cloud in the target area where the object in real space exists, and then fragments the image sensor data consisting of the point cloud in the target area in real space extracted by the filter unit 223 into packets and transmits them to the server device 3 (S2).

[0047] Next, in the server device 3, the server-side receiving unit 31 receives the image sensor data made up of a group of points in real space transmitted from each sensor device 2 (S3).

[0048] Then, the aggregation unit 32 aggregates the image sensor data consisting of the point cloud in the real space received by the server-side receiving unit 31 (S4).

[0049] Then, the detection unit 33 detects the three-dimensional marker 1 placed on the moving object M moving in the real space based on the image sensor data consisting of a cloud of points in the real space aggregated by the aggregation unit 32 (S5).

[0050] Then, the identification unit 34 identifies the moving object M on which the three-dimensional marker 1 is placed, based on the detection result of the three-dimensional marker 1 by the detection unit 33 (S6).

[0051] Then, the server-side transmitting unit 35 transmits information about the moving object M (the position, direction or identification information of the moving object M) to the application system 4 (S7).

[0052] The application system 4 then provides various traffic information to each mobile body M based on the information about the identified mobile body M, and automatically controls the traffic of each mobile body M, thereby making it possible to realize optimal traffic for the mobile body M (S8). [Example]

[0053] Example 1 In this embodiment 1, as shown in Figure 9, the sensor device 2 is placed at various locations on a static object in real space, and when acquiring image sensor data consisting of a point cloud in real space, it acquires image sensor data consisting of a point cloud of a three-dimensional marker 1 placed on a moving body M (e.g., an autonomous vehicle) moving in real space.

[0054] In addition, the server device 3 aggregates image sensor data consisting of point clouds in real space transmitted from each sensor device 2, and then detects a three-dimensional marker 1 based on the aggregated image sensor data consisting of point clouds in real space, and identifies the moving body M on which the three-dimensional marker 1 is placed based on the detection result of the three-dimensional marker 1 by the detection unit 33.

[0055] The application system 4 provides various traffic information to each mobile object M based on the information about the mobile object M transmitted from the server device 3, and automatically controls the traffic of each mobile object M.

[0056] <Example 2> In this second embodiment, as shown in Figure 10, the sensor device 2 is placed on one moving body M (e.g., an electric wheelchair) moving in real space, and when acquiring image sensor data consisting of a point cloud in real space, it acquires image sensor data consisting of a point cloud of a three-dimensional marker 1 placed on another moving body M (e.g., another electric wheelchair) moving in real space.

[0057] In addition, the server device 3 aggregates image sensor data consisting of point clouds in real space transmitted from each sensor device 2, and then detects a three-dimensional marker 1 based on the aggregated image sensor data consisting of point clouds in real space, and identifies the moving body M on which the three-dimensional marker 1 is placed based on the detection result of the three-dimensional marker 1 by the detection unit 33.

[0058] The application system 4 provides various traffic information to each mobile object M based on the information about the mobile object M transmitted from the server device 3, and automatically controls the traffic of each movement.

[0059] Example 3 In this third embodiment, an experiment on the detection of the three-dimensional marker 1 will be described.

[0060] [Experimental Setup] In this experiment, a prototype three-dimensional marker 1 shown in Figure 11(a) was fabricated. The height of the three-dimensional marker 1 is 300 mm. The three-dimensional marker 1 was fabricated using extruded polystyrene foam made from styrene material. The three-dimensional marker 1 exhibits rotational symmetry, returning to the same shape after a 60-degree rotation. The three-dimensional marker 1 is composed of six convex portions and six concave portions, with the radius of the convex portions being 100 mm and the radius of the concave portions being 65 mm. The arc of the concave portion of the three-dimensional marker 1 is 44.44 degrees, and the arc of the convex portions is 15.56 degrees.

[0061] Figure 12 shows the placement of the sensor device 2 and the position of the three-dimensional marker 1 in the experiment. The sensor device 2 was installed at the location indicated by the dot in Figure 12. The sensor device 2 was installed approximately 130 cm above the ground. In the experiment, the three-dimensional marker 1 was placed at the location indicated by the x in Figure 12. The three-dimensional marker 1 was placed 1.35 m above the ground. The position of the three-dimensional marker 1 was changed as shown in Figure 12 at points 3.0 m, 4.5 m, 6.0 m, 7.5 m, and 9.0 m from the sensor device 2. At each installation point, the three-dimensional marker 1 was rotated 10 degrees and data was captured in six different directions. A Velodyne VLP-16 LIDAR unit was used to acquire the point cloud data. An NVIDIA Jetson Nano was used as the terminal device 22. An NVIDIA Jetson NX was used as the server device 3, and the acquired data was stored on a connected HDD.

[0062] [Acquisition of experimental data] The data acquisition process followed the steps outlined below. As shown in Figure 12, a three-dimensional marker 1 (K = 1) was placed at five distances (n = 5) from the sensor device 2. The three-dimensional marker 1 was rotated up to 60 degrees (m = 6) in 10-degree increments, and data was acquired at each position. Data was acquired at a rate of one frame every 110 milliseconds, for a total of 2,500 frames. 250 frames were captured for each pattern, resulting in a total of 75,000 frames of point cloud data. OpenPCDet was also employed as the machine learning tool. In this experiment, three deep learning models were used for object detection: PointPillars, SECOND, and PointRCNN. 80% and 20% of the data obtained in the experiment were used for training and testing, respectively.

[0063] [Experimental Results] We use average precision (AP) and (AP|R40) as evaluation metrics. Table 1 shows AP and AP|R40 as the detection accuracy of the three deep learning models. This table suggests that PointRCNN and SECOND performed well with an accuracy of about 99%.

[0064] [Table 1]

[0065] Next, we evaluate the system from the perspective of false positives. We used the dummy object shown in Figure 11(b). The experimental setup for the dummy object was essentially the same as for 3D Marker 1, but the number of frames for each acquired pattern was set to 300 for the dummy object. We counted the number of false positives detected during the detection process. The number of false positives was 11 for PointPillars, 1062 for PointRCNN, and 23 for SECOND. This suggests that although the accuracy was not very high, PointPillars performed well in terms of false positives. SECOND's overall performance was high in the experiment.

[0066] Example 4 [Experiment and analysis conditions (equipment, location, parameters)] ·Equipment used: Livox avia * Jetson Orin Nano 3D marker Test location: Shibaura Institute of Technology Toyosu Campus Communication Foyer Libraries used: OpenPCDet ·How to use: Centerpoint Parameters: epoch: 1000 Number of frames used: 26289 frames

[0067] [Explanation of the results in Table 2] From the prepared point cloud data, 80% was extracted as training data, and the remaining 20% ​​was used as test data. These are the AP and AP|R40(%) at IoU thresholds of 0.5 and 0.7. Here, AP is the average precision, and AP|R40 is the method of dividing the recall axis of the precision-recall curve into 40 parts when the IoU threshold is 0.7 or higher, calculating the precision at each point, and averaging them.

[0068] [Table 2] [Practical benefits to society from the results obtained] By constructing a generalized detection model using multiple LiDARs, the detection accuracy of 3D markers in practical environments can be further improved.

[0069] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the illustrated embodiments. Various modifications and variations can be made to the illustrated embodiments within the same scope as the present invention or within an equivalent scope. [Explanation of symbols]

[0070] 1...3D marker 11...Marker section 12...Code section 2...Sensor device 21...Sensor section 22...Terminal device 221...Capturing part 222…Storage section 223...Filter section 224...Sensor side transmitter 3. Server equipment 31...Server side receiving unit 32...Consolidation section 33...Detection unit 34...Identification section 35...Server-side transmission unit 4. Application system M...Moving object

Claims

1. The system includes a three-dimensional marker arranged in a real space, a plurality of sensor devices arranged in the real space, and a server device connected to the sensor devices in a state capable of communicating with the sensor devices, The three-dimensional markers are signs having a three-dimensional shape that are individually arranged on the surface side of a moving object moving in real space, The sensor device includes: a sensor unit that acquires image sensor data consisting of a point cloud in a real space including the three-dimensional marker; a sensor-side transmitting unit that transmits image sensor data consisting of a point cloud in real space acquired by the sensor unit to the server device, The server device a server-side receiving unit that receives image sensor data consisting of point clouds in real space transmitted from each of the sensor devices; an aggregation unit that aggregates image sensor data consisting of a point cloud in real space received by the server-side receiving unit; a detection unit that detects the three-dimensional marker based on image sensor data consisting of a point cloud in real space aggregated by the aggregation unit; a recognition unit that recognizes a moving object on which the three-dimensional marker is placed based on a result of detection of the three-dimensional marker by the detection unit.

2. The moving body identification system according to claim 1 , wherein the three-dimensional marker has a marker portion whose position and / or orientation is detected by a three-dimensional shape in a predetermined direction.

3. 3. The moving body identification system according to claim 2, wherein the marker portion has a three-dimensional shape with one or more protrusions formed along a circumferential direction.

4. 2. The moving body identification system according to claim 1, wherein the three-dimensional marker has a code portion to which predetermined identification information is given by a three-dimensional shape in a predetermined direction.

5. 5. The moving body identification system according to claim 4, wherein the code portion has a three-dimensional shape with a plurality of protrusions formed along the axial direction.

6. A moving body identification system as described in any one of claims 1 to 5, wherein the sensor unit is arranged on a static object in real space, and when acquiring image sensor data consisting of a point cloud in real space, the sensor unit acquires image sensor data consisting of a point cloud of the three-dimensional marker arranged on a moving body moving in real space.

7. A moving body identification system as described in any one of claims 1 to 5, wherein the sensor unit is arranged on one dynamic object moving in real space, and when acquiring image sensor data consisting of a point cloud in real space, it acquires image sensor data consisting of a point cloud of the three-dimensional marker arranged on another moving body moving in real space.

8. A moving object identification system as described in any one of claims 1 to 5, wherein the aggregation unit aligns the spatial coordinate axes of image sensor data consisting of point clouds in real space acquired by each sensor unit based on the detection results of the three-dimensional markers by the detection unit.

9. A moving object identification system as described in any one of claims 1 to 5, wherein the detection unit detects the three-dimensional marker placed in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit, by using a machine learning model that has machine-learned the three-dimensional shape of the three-dimensional marker placed in real space.

10. the server device is connected to an application system in a communicable state; 6. The moving body identification system according to claim 1, further comprising a server-side transmitting unit that transmits information about the moving body identified by the identifying unit to the application system.

Images