Mobile object traffic control system

Abstract

To provide a mobile object traffic control system that controls traffic of a mobile object in real space with high accuracy.SOLUTION: A sensor device acquires image sensor data composed of point clouds in the real space. A server device generates a digital twin based on the image sensor data composed of the point clouds in the real space, and generates a map on which a virtual obstacle for controlling the traffic of the mobile object is placed on the digital twin. The mobile object generates a travel route of the mobile object in the real space based on information related to the map including the virtual obstacle, and controls each drive mechanism for travel.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a mobile traffic control system that controls the traffic of mobile objects in real space. [Background technology]

[0002] In recent years, global demand for autonomous mobile robots (AMRs) has been increasing, prompting many cities to integrate AMRs into society, especially in labor-intensive tasks such as deliveries. Particularly in urban areas, it is important to ensure the safety of all actors in the environment as multiple delivery robots are integrated and society operates. It is desirable to achieve optimal transportation by providing safety not only for pedestrians and cars, but also for robots and micromobility.

[0003] Previous research has proposed digital twin systems that support robots and micromobility by outsourcing sensing processing (see, for example, Non-Patent Document 1) and navigation processing (see, for example, Non-Patent Document 2) from in-vehicle devices to digital twins (technology that collects real-world data and generates digital twins). However, these technologies did not take into account the limitations of relying solely on raw sensing information to avoid situations with a high risk of collision. Furthermore, this problem could arise not only for the autonomous mobile robots (AMRs) mentioned above, but also for all moving objects that move autonomously in real space, such as vehicles, electric wheelchairs, and micromobility. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] K. Akiyama et al., "Edge computing system with multi-lidar sensor network for robustness of autonomous personal-mobility," in 2022 IEEE 42nd ICDCSW, 2022, pp. 290-295. [Non-patent document 2] M. Wago et al., "Prototype of edge sensing and computing system with multi-lidar network for autonomous micro-mobility," in 2023 IEEE 20th CCNC, 2023, pp. 927-928. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above-mentioned technical background, and has an object to provide a mobile object traffic control system that can accurately control the traffic of mobile objects in real space. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the present invention provides a mobile object traffic control system comprising one or more sensor devices arranged on the environment side of a real space, a server device connected to the sensor devices in a state capable of communicating with the server device, and mobile objects connected to the server device in a state capable of communicating with the system, and controlling traffic of the mobile objects in the real space, wherein the sensor devices comprise an environment-side sensor unit that acquires image sensor data consisting of a point cloud in the real space, and a sensor-side transmission unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the environment-side sensor unit to the server device, and the server device comprises an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space transmitted from the sensor devices, a digital twin generation unit that generates a digital twin corresponding to the real space based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit, and a digital twin generation unit that generates a digital twin corresponding to the real space based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit. a traffic control unit that places virtual obstacles on the digital twin to regulate traffic of the moving body based on the analysis of the situation in the real space by the situation analysis unit; a map generation unit that generates a map including the virtual obstacles based on information about the digital twin in which the virtual obstacles are placed by the traffic control unit; and a server-side transmission unit that transmits to the moving body information about the map including the virtual obstacles generated by the map generation unit, wherein the moving body is equipped with a route generation unit that generates a travel route for the moving body in the real space based on the information about the map including the virtual obstacles transmitted from the server device, and a control unit that controls the travel of the moving body in the real space based on the travel route of the moving body generated by the route generation unit.

[0007] In addition, the situation analysis unit may detect areas where the risk of collision of the moving body is estimated to be high based on the situation in real space, and the traffic control unit may place virtual obstacles to regulate traffic of the moving body in areas in the digital twin detected by the situation analysis where the risk of collision of the moving body is estimated to be high.

[0008] In addition, the situation analysis unit may detect blind spots in the real space in the digital twin, and the traffic control unit may place virtual obstacles to regulate traffic of the moving body in the blind spots in the real space in the digital twin detected by the situation analysis.

[0009] The situation analysis unit may detect blind spot areas of the moving body traveling in real space in the digital twin, and the traffic control unit may place virtual obstacles to regulate traffic of the moving body in the blind spot areas of the moving body in the digital twin detected by the situation analysis.

[0010] In addition, the situation analysis unit may geometrically determine whether or not a real obstacle exists between the moving body and a specified space, and if it determines that a real obstacle exists between the moving body and the specified space, may detect the opposite side of the real obstacle relative to the moving body as a blind spot area of ​​the moving body.

[0011] The situation analysis unit may also include a voxel partitioning unit that partitions the digital twin generated by the digital twin generation unit into a plurality of voxels, a voxel extraction unit that extracts voxels in blind spot areas of the moving body in the digital twin partitioned into a plurality of voxels by the voxel partitioning unit, and a blind spot area detection unit that detects blind spot areas of the moving body in the digital twin based on the voxels in the blind spot areas extracted by the voxel extraction unit.

[0012] The voxel extraction unit may also connect the center of the voxel of the viewpoint of the moving body to the center of each blank voxel where no real obstacles exist with a first line segment, and if the first line segment passes through the center of a voxel of a real obstacle, extract the blank voxel connected by the first line segment as a voxel in a blind spot area.

[0013] In addition, the voxel extraction unit may connect the centers of the voxels of adjacent real obstacles with a second line segment, and when the first line segment intersects with the second line segment, extract the blank voxels connected by the first line segment as voxels in the blind spot area.

[0014] In addition, the situation analysis unit may extract voxels in the blind spot area of ​​the moving body from the voxels of the moving body and real obstacles in the digital twin using a learning model that has been machine-learned using test data regarding voxels of the moving body and real obstacles in the digital twin and correct answer data regarding voxels in the blind spot area of ​​the moving body in the digital twin for the test data.

[0015] The moving body may also include a moving body side sensor unit that acquires image sensor data consisting of a cloud of points in real space around the moving body, and a moving body position estimation unit that estimates the moving body's own position based on information regarding a predetermined map in real space and the image sensor data consisting of the cloud of points in real space around the moving body acquired by the moving body side sensor unit, and the path generation unit may generate a driving path for the moving body in real space based on information regarding the moving body's own position estimated by the moving body position estimation unit and information regarding a map including virtual obstacles generated by the map generation unit.

[0016] Further, in the server device, the map generation unit generates a map for obstacle avoidance that includes a virtual obstacle and a map for self-position estimation that does not include a virtual obstacle, the server-side transmission unit transmits information regarding the map for obstacle avoidance that includes a virtual obstacle and the map for self-position estimation that does not include a virtual obstacle to the mobile body, and in the mobile body, the mobile body position estimation unit estimates the self-position of the mobile body in real space based on the information regarding the map for self-position estimation that does not include a virtual obstacle and image sensor data consisting of a point cloud in real space around the mobile body acquired by the mobile body-side sensor unit, and the path generation unit generates a driving path for the mobile body in real space based on the information regarding the self-position of the mobile body estimated by the mobile body position estimation unit and the information regarding the map for obstacle avoidance that includes a virtual obstacle.

[0017] Further, in the server device, the traffic regulation unit places a first virtual obstacle on the digital twin to regulate traffic of a first moving body, while locating a second virtual obstacle to regulate traffic of a second moving body, in a location on the digital twin different from the first virtual obstacle, based on the analysis result of the situation in real space by the situation analysis unit; the map generation unit generates a first map including the first virtual obstacle based on information about the digital twin in which the first virtual obstacle is placed by the traffic regulation unit, while generating a second map including the second virtual obstacle based on information about the digital twin in which the second virtual obstacle is placed by the traffic regulation unit; and the server-side transmission unit transmits information about the first map including the first virtual obstacle generated by the map generation unit to the first moving body, while and transmitting information regarding a second map including the selected second virtual obstacle to the second moving body, wherein in the first moving body, the path generation unit generates a travel path for the first moving body to avoid the first virtual obstacle in real space based on the information regarding the first map including the first virtual obstacle transmitted from the server device, and the control unit controls the travel of the first moving body based on the travel path for the first moving body generated by the path generation unit; and in the second moving body, the path generation unit generates a travel path for the second moving body to avoid the second virtual obstacle in real space based on the information regarding the second map including the second virtual obstacle transmitted from the server device, and the control unit controls the travel of the second moving body based on the travel path for the second moving body generated by the path generation unit.

[0018] The present invention also provides a mobile object traffic control system that includes one or more sensor devices arranged on the environment side of a real space, a server device connected to the sensor devices in a state where they can communicate with each other, and mobile objects connected to the server device in a state where they can communicate with each other, and that controls traffic of the mobile objects in the real space, wherein the sensor devices include an environment-side sensor unit that acquires image sensor data consisting of a point cloud in the real space, and a sensor-side transmission unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the environment-side sensor unit to the server device, and the server device includes an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space transmitted from the sensor devices, a digital twin generation unit that generates a digital twin corresponding to the real space based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit, and a digital twin generation unit that generates a digital twin corresponding to the real space based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit. a traffic control unit that places virtual obstacles on the digital twin to regulate traffic of the moving body based on the analysis of the situation in the real space by the situation analysis unit; a map generation unit that generates a map including the virtual obstacles based on information about the digital twin in which the virtual obstacles are placed by the traffic control unit; a route generation unit that generates a travel route for the moving body in the real space based on information about the map including the virtual obstacles generated by the map generation unit; and a server-side transmission unit that transmits the travel route for the moving body generated by the route generation unit to the moving body, and the moving body is equipped with a control unit that controls the travel of the moving body in the real section based on the travel route for the moving body transmitted from the server device. [Effects of the Invention]

[0019] According to the present invention, a sensor device acquires image sensor data consisting of a point cloud in real space. A server device then generates a digital twin based on the image sensor data consisting of the point cloud in real space, and generates a map on the digital twin in which virtual obstacles are arranged for controlling traffic of mobile objects. The mobile object then generates a travel path for the mobile object in real space based on information about the map including the virtual obstacles. Therefore, by having the mobile object travel a travel path that avoids the virtual obstacles, traffic of mobile objects in real space can be controlled with high accuracy. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a diagram showing the overall configuration of a mobile traffic control system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram illustrating a configuration of the sensor device of FIG. [Figure 3] FIG. 2 is a diagram illustrating a configuration of a server device in FIG. [Figure 4] FIG. 2 is a diagram illustrating a configuration of the moving body of FIG. [Figure 5] FIG. 1 is a diagram illustrating an example of image sensor data consisting of a point cloud in real space. [Figure 6] FIG. 1 is a diagram showing an example of 3D data in real space. [Figure 7] FIG. 1 is a diagram illustrating an example of digital twin data in a real space. [Figure 8] FIG. 10 is a diagram showing simulation results of a moving body according to the present invention and a conventional moving body; [Figure 9] FIG. 1 is a diagram illustrating a method for geometrically detecting blind spots of a moving object. [Figure 10] FIG. 10 is a diagram illustrating a method for detecting blind spot areas using voxels. [Figure 11] FIG. 10 is a diagram showing the configuration of a situation analysis unit of a mobile traffic control system according to a fourth embodiment. [Figure 12] 1A and 1B are diagrams showing (a) a collision of a robot, (b) a restricted area, (c) point cloud data, and (d) a 3D model according to the first embodiment. [Figure 13] 3A to 3C are diagrams showing travel routes of a robot in each method according to the first embodiment. [Figure 14] FIG. 10 is a diagram showing an experimental site according to Example 2. [Figure 15] 10 is a diagram showing blind spot information as viewed from viewpoints A and E according to the second embodiment. FIG. [Figure 16] 10A and 10B are diagrams illustrating results when a blind spot area is detected according to the third embodiment. [Figure 17] 11 is a table showing errors and accuracy of the detection results of the blind spot area according to the third embodiment. [Figure 18] FIG. 10 is a diagram showing an estimation of the learning process (left diagram: loss function, right diagram: performance) of the machine learning model according to Example 3. [Figure 19] FIG. 10 is a diagram showing the travel paths of V1 and V2 in the third trial of each method according to Example 4. [Figure 20] FIG. 10 is a diagram showing the results of an experiment according to Example 5. DETAILED DESCRIPTION OF THE INVENTION

[0021] First Embodiment Next, a first embodiment of a mobile traffic control system according to the present invention (hereinafter referred to as the present system) will be described with reference to FIGS.

[0022] 1, this system includes a plurality of sensor devices 1 arranged in the environment of a real space such as an intersection or a road, a server device 2 connected to the sensor devices 1 in a state where they can communicate with each other, and a plurality of mobile objects 3 connected to the server device 2 in a state where they can communicate with each other, and controls the traffic of the mobile objects 3 in the real space. The configurations of the sensor devices 1, the server device 2, and the mobile objects 3 will be specifically described below.

[0023] [Configuration of sensor device 1] As shown in Figure 2, the sensor device 1 is installed on a support or the like on the environment side in real space, and includes an environment-side sensor unit 11 that acquires image sensor data consisting of a point cloud in real space at predetermined time intervals, and a sensor-side transmission unit 12 that sequentially transmits the image sensor data consisting of the point cloud in real space acquired by the environment-side sensor unit 11 to the server device 2.

[0024] The environment-side sensor unit 11 is a so-called LIDAR (light detection and ranging) sensor. LIDAR is a type of sensor that uses laser light, which has a higher radiant flux density than radio waves and irradiates a target with short-wavelength laser light while scanning it, thereby acquiring image sensor data consisting of a point cloud in real space and accurately detecting not only the distance to the target but also the position and shape of the target.

[0025] Examples of this LIDAR include a method of acquiring image sensor data in all directions of 360 degrees by rotating a light emitter that emits multiple laser beams, and a method of acquiring image sensor data by directly irradiating laser beams within a predetermined light irradiation angle range. Also, the more light emitters that emit laser beams, the higher the accuracy of the image sensor data, but the higher the cost, so an inexpensive LIDAR with a small number of light emitters may be used.

[0026] When image sensor data is acquired by one sensor device 1, there is a risk that a visual area may appear on the opposite side of the wall or moving object 3 as seen from the sensor device 1. Therefore, in this embodiment, by aggregating image sensor data consisting of point clouds acquired by multiple sensor devices 11 as described below, it is possible to reduce blind spots, and therefore one sensor network is configured by providing multiple sensor devices 1 (LIDAR) so as to acquire image sensor data consisting of point clouds from different directions in the same real space.

[0027] [Configuration of server device 2] The server device 2 is installed in a state where it can communicate with the sensor devices 1 via a wired network, and as shown in Figure 3, it is equipped with an aggregation unit 21 that aggregates image sensor data consisting of point clouds in real space transmitted from each sensor device 1, a digital twin generation unit 23 that generates a digital twin corresponding to the real space based on the image sensor data consisting of point clouds in real space, a situation analysis unit 24 that analyzes the situation in the real space based on information related to the digital twin, a traffic control unit 25 that places a virtual obstacle V on the digital twin to regulate the traffic of the mobile body 3 based on the analysis results of the situation in the real space, a map generation unit 26 that generates information related to the map, and a server-side transmission unit 28 that transmits information related to the map to the mobile body 3.

[0028] The aggregating unit 21 aggregates image sensor data consisting of point clouds in real space transmitted from each sensor device 1. Specifically, the aggregating unit 21 chronologically synthesizes the image sensor data consisting of point clouds in real space transmitted from each sensor device 1 based on timestamps, aggregates the data by aligning them in three-dimensional space, and then stores the aggregated image sensor data consisting of point clouds as environmental information in the environmental information storage unit 22. Note that when aggregating image sensor data consisting of point clouds in real space, the aggregating unit 21 may perform other processing such as smoothing of the point clouds of the image sensor data.

[0029] The digital twin generation unit 23 generates a digital twin corresponding to the real space based on the image sensor data consisting of the aggregated point cloud stored by the environmental information storage unit 22. This digital twin refers to a digital reproduction of the real-space environment in cyberspace, obtained by acquiring realistic real-space information using sensors such as LIDAR. As information related to the digital twin, for example, Figure 5 shows an example of image sensor data consisting of a point cloud in the real space of a certain intersection, and Figure 6 shows an example of 3D data for the same intersection. Both of these data are digital twins themselves, or corresponding digital twins, that show the intersection in a bird's-eye view from directly above, based on the image sensor data consisting of the aggregated point cloud.

[0030] The situation analysis unit 24 analyzes the situation in the real space based on information about the digital twin generated by the digital twin generation unit 23. Specifically, the situation analysis unit 24 analyzes the situation in the real space by detecting the positions of static objects in the real space, such as walls, roads, plants, and buildings, and by detecting the positions, movement directions, and movement speeds of dynamic objects in the real space, such as moving bodies 3, such as vehicles and mobile robots, and pedestrians, based on the information about the digital twin. When analyzing the situation in the real space by the situation analysis unit 24, it may detect areas estimated to have a high risk of collision with the moving bodies from the situation in the real space, or may detect blind spots in the real space as described in the third embodiment.

[0031] The traffic regulation unit 25 places a virtual obstacle V on the digital twin to regulate the travel of the moving body 3, based on the analysis result of the situation in the real space by the situation analysis unit 24. Specifically, the traffic regulation unit 25 places a virtual obstacle V that does not exist in the actual real space on the digital twin in an area at an intersection in the real space where it is desired to regulate the traffic of the moving body 3 (for example, an area where it is estimated that there is a high risk of collision with the moving body 3 because other moving bodies 3 or pedestrians are entering, or an area where it is estimated that there is a high risk of collision with the moving body 3 because it is a blind spot for the moving body 3). For example, FIG. 7 shows an example in which a virtual obstacle V is placed on a digital twin corresponding to the real space of a certain intersection.

[0032] The map generation unit 26 generates a map for the moving body 3 to travel in real space based on information about the digital twin in which the virtual obstacle V has been placed by the traffic regulation unit 25. In this embodiment, an obstacle avoidance map that includes the virtual obstacle V for the moving body 3 to avoid the virtual obstacle V, and a self-position estimation map that does not include the virtual obstacle V for the moving body 3 to estimate its own position are generated and stored in the map information storage unit 27. Note that the information about this map may be information about the digital twin as shown in FIG. 7, or information about a map in another format may be generated so that it is easier for the moving body 3 to recognize.

[0033] The server-side transmitting unit 28 transmits the information about the map stored in the map information storage unit 27 to the mobile object 3 traveling in the real space as needed or periodically.

[0034] [Configuration of mobile unit 3] The mobile body 3 is an autonomous mobile robot that is installed in a state where it can communicate with the sensor device 1 via a wireless network and moves through real space such as an intersection, and is equipped with an onboard sensor device 31 and an onboard computer device 32.

[0035] The sensor device 31 is composed of a moving body sensor unit 311 , an IMU unit 312 and a wheel sensor unit 313 .

[0036] The moving body sensor unit 311 is a so-called LIDAR (light detection and ranging) sensor, which is provided on the surface (particularly the front) of the vehicle and acquires image sensor data consisting of a point cloud around the moving body 3 in real space.

[0037] The IMU unit 312 is equipped with an angular velocity (gyro) sensor, an acceleration sensor, and a temperature sensor, and is a unit that detects three-dimensional inertial motion (translational motion and rotational motion in three orthogonal axis directions). It detects translational motion using an acceleration sensor and rotational motion using a gyro sensor, measures acceleration and angular velocity with high precision, and uses the measurement results to understand, estimate, and control the behavior (attitude and trajectory) of the moving body.

[0038] The wheel sensor unit 313 is a sensor for reading the rotation speed of a wheel of a vehicle, and is usually composed of a toothed ring and a pickup.

[0039] The on-board computer device 32 also includes a self-position estimation unit 321 , a route generation unit 322 , and a control unit 323 .

[0040] After receiving information about the map transmitted from the server device 2, the self-position estimation unit 321 estimates its own position using various sensor information from the sensor device 1 mounted on the moving body 3 based on the information about the map for self-position estimation that does not include the virtual obstacle V.

[0041] The path generation unit 322 generates a driving path for the moving body 3 to travel in real space based on information about the map, including information about the collision avoidance map that includes the virtual obstacle V. Specifically, based on information about the self-position of the moving body 3 estimated by the self-position estimation unit 321 and information about the collision avoidance map that includes the virtual obstacle V, the path generation unit 322 generates a driving path that avoids the virtual obstacle V included in the map as well as static objects such as walls that have already been recognized and dynamic objects that may become obstacles, such as other vehicles and pedestrians detected by the in-vehicle sensor device 1.

[0042] The control unit 323 calculates the speed required to reach the destination, and controls the drive mechanism 33 such as wheels to travel along the route generated by the route generation unit 322 at the calculated speed.

[0043] In this embodiment, the self-position estimation unit 321 estimates the self-position of the moving body 3 based on information about a map for self-position estimation that does not include the virtual obstacle V, but the self-position estimation unit 321 may estimate the self-position of the moving body 3 based on information about a map for collision avoidance that includes the virtual obstacle V. However, if the map includes the virtual obstacle V, there is a risk that the self-position estimation unit 321 will confuse the virtual obstacle V with a real obstacle, so it is preferable to estimate the self-position of the moving body 3 based on information about a map for self-position estimation that does not include the virtual obstacle V.

[0044] Furthermore, the self-position estimation unit 321 that estimates the self-position of the moving body 3 and the route generation unit 322 that generates the travel route of the moving body 3 are provided in the moving body 3, but they may also be provided in the server device 2 (particularly between the map information storage unit 27 and the server-side transmission unit 28).

[0045] <Second embodiment> Next, a second embodiment of the present system will be described with reference to Figures 3, 7, and 8. Note that only the configurations that are different from the above embodiment will be described below, and the same configurations will be denoted by the same reference numerals and will not be described again.

[0046] In this embodiment, the configurations of the sensor device 1, server device 2, and moving body 3 are the same as those of the first embodiment, and a different first virtual obstacle Va and second virtual obstacle Vb are placed on the digital twin for the first moving body 3a and the second moving body 3b in order to avoid a collision between the first moving body 3a and the second moving body 3b.

[0047] Specifically, in the server device 2, the traffic regulation unit 25 places a first virtual obstacle Va on the digital twin to regulate the traffic of the first moving body 3a, while placing a second virtual obstacle Vb on the digital twin to regulate the traffic of the second moving body 3b, based on the analysis results of the situation in real space by the situation analysis unit 24. For example, as information about the digital twin of a certain intersection, FIG. 7(a) shows an example of digital twin data in which a first virtual obstacle Va is placed for the first moving body 3a, and FIG. 7(b) shows an example of digital twin data in which a second virtual obstacle Vb is placed for the second moving body 3b, with the positions of the first virtual obstacle Va and the second virtual obstacle Vb being shifted so that they are adjacent to each other.

[0048] The map generation unit 26 generates a first map that includes a first virtual obstacle Va based on information about the digital twin in which the first virtual obstacle Va has been placed by the traffic regulation unit 25, while generating a second map that includes a second virtual obstacle Vb based on information about the digital twin in which the second virtual obstacle Vb has been placed by the traffic regulation unit 25.

[0049] The server-side transmitting unit 28 then transmits information regarding the first map including the first virtual obstacle Va generated by the map generating unit 26 to the first moving body 3a, while transmitting information regarding the second map including the second virtual obstacle Vb generated by the map generating unit 26 to the second moving body 3b.

[0050] Furthermore, in the first moving body 3a, the route generation unit 322 generates a travel route for the first moving body 3a to avoid the first virtual obstacle Va in the real space based on information about the first map including the first virtual obstacle Va transmitted from the server device 2.

[0051] The control unit 323 controls the travel of the first moving body 3a based on the travel route of the first moving body 3a generated by the route generation unit 322.

[0052] Meanwhile, in the second moving body 3b, the path generation unit 322 generates a driving path for the second moving body 3b to avoid the second virtual obstacle Vb in real space based on information about the second map including the second virtual obstacle Vb transmitted from the server device 2.

[0053] The control unit 323 controls the travel of the second moving body 3b based on the travel route of the second moving body 3b generated by the route generation unit 322.

[0054] Therefore, if a virtual obstacle V is not placed on the digital twin as in the conventional case, the first moving body 3a and the second moving body 3b will attempt to move to the destination via the shortest route, but since the performance of the sensor device 1 of the moving body 3 is generally poor and the body of the moving body 3 is small, there is a risk of them colliding with each other if they are unable to recognize each other (see Figure 8(a)).

[0055] In contrast, in the case where the position of the first virtual obstacle Va relative to the first moving body 3a and the position of the second virtual obstacle Vb relative to the second moving body 3b are offset from each other on the digital twin as in the present system, the first virtual obstacle Va poses a traffic obstacle to the first moving body 3a but the second virtual obstacle Vb does not, while the second virtual obstacle Vb poses a traffic obstacle to the second moving body 3b but the first virtual obstacle Va does not. Therefore, when the first moving body 3a and the second moving body 3b travel at a certain intersection, they travel in a way that avoids the different first and second obstacles, and therefore it is possible to reliably prevent a collision between the first moving body 3a and the second moving body 3b (see FIG. 8(b)).

[0056] The first map including the first virtual obstacle Va and the second map including the second virtual obstacle Vb may be generated as physically different maps, but they may also be generated as a single map as long as the first moving body 3a and the second moving body 3b can recognize the first virtual obstacle Va and the second virtual obstacle Vb that they must avoid, respectively.

[0057] <Third embodiment> Next, a third embodiment of the present system will be described with reference to Fig. 9. Note that only the configurations different from the above embodiments will be described below, and the same configurations will be denoted by the same reference numerals and will not be described again.

[0058] In this embodiment, when the situation analysis unit 24 analyzes the situation in the real space based on information about the digital twin, it detects blind spots in the real space in the digital twin and places a virtual obstacle V in the blind spots.

[0059] For example, when the position of a moving body 3 (host vehicle) that is the control target in the real space in the digital twin is set as the viewpoint of the moving body 3, the situation analysis unit 24 detects a blind spot area relative to the viewpoint of the moving body 3. Then, the traffic regulation unit 25 places a virtual obstacle V for regulating traffic of the moving body 3 in the blind spot area of ​​the digital twin analyzed by the situation analysis.

[0060] The method for detecting the blind spot area of ​​the moving body 3 by the situation analyzing unit 24 is not particularly limited, but may include, for example, as shown in FIG. 9, geometrically determining whether or not a real obstacle O exists between the moving body 3 and a predetermined space, and if it is determined that an obstacle exists between the moving body 3 and the predetermined space, detecting the opposite side of the obstacle O from the moving body 3 as the blind spot area of ​​the moving body 3. For example, in the configuration shown in FIG. 9(a), since no real obstacle O exists between the moving body 3 (Viewpoint) and the predetermined space (Visible space), the predetermined space (Visible space) is determined to be a visible area. On the other hand, in the configuration shown in FIG. 9(b), since a real obstacle O exists between the moving body 3 (Viewpoint) and the predetermined space (Blind spot), the predetermined space (Blind spot) located on the opposite side of the real obstacle O from the moving body 3 is determined to be the blind spot area of ​​the moving body 3, and a virtual obstacle V is placed in the blind spot area.

[0061] In this embodiment, when the position of the moving body 3 (own vehicle) that is the object of control in the real space in the digital twin is taken as the viewpoint of the moving body 3, blind spots relative to the viewpoint of the moving body 3 are detected. However, blind spots from other moving bodies or pedestrians other than the moving body 3 (own vehicle) that is the object of control may also be detected, as well as areas that are likely to become blind spots from various viewpoints in the real space, regardless of whether the moving body 3 is currently moving or not.

[0062] <Fourth embodiment> Next, a fourth embodiment of the present system will be described with reference to Figures 10 and 11. Note that only the configurations that are different from the above embodiments will be described below, and the same configurations will be denoted by the same reference numerals and will not be described again.

[0063] In this embodiment, when the situation analysis unit 24 detects blind spots of the moving body 3 in the digital twin, the digital twin is divided into a plurality of voxels.

[0064] Specifically, as shown in FIG. 10, the situation analysis unit 24 includes a voxel partitioning unit 241 that partitions the digital twin into multiple voxels based on information about the digital twin generated by the digital twin generation unit 23, a voxel extraction unit 242 that extracts voxels of blind spot areas in the digital twin partitioned into multiple voxels by the voxel partitioning unit 241, and a blind spot area detection unit 243 that detects blind spot areas of the digital twin based on the voxels of the blind spot areas extracted by the voxel extraction unit.

[0065] As a method for extracting voxels in the blind spot area by the voxel extracting unit 242, the following extraction methods (1) and (2) are available, as shown in FIG. 11, for example.

[0066] (1) The voxel extraction unit 242 connects the center of the voxel of the viewpoint of the moving object 3 with the center of each blank voxel where no obstacle exists by a first line segment A, and if the first line segment A passes through the center of a voxel of an obstacle, extracts the blank voxel connected by the first line segment A as a voxel in a blind spot area (see Figures 11(a), (c), and (d)).

[0067] (2) The voxel extraction unit connects the centers of the voxels of adjacent obstacles with a second line segment B, and when the first line segment A intersects with the second line segment B, the blank voxels connected by the first line segment A are extracted as voxels in the blind spot area (see Figures 11(a) and 11(b)).

[0068] Note that a machine-learned learning model may be used when the situation analysis unit 24 detects blind spot areas of the moving body 3 in the digital twin. Specifically, the situation analysis unit 24 extracts voxels of the blind spot areas of the moving body 3 from the voxels of the moving body 3 and obstacles in the digital twin using a learning model that has been machine-learned using test data related to the voxels of the moving body 3 and obstacles in the digital twin as shown in the left diagram of Fig. 11 and correct answer data related to the voxels of the blind spot areas of the moving body 3 in the digital twin for the test data as shown in the right diagram of Fig. 11. [Example]

[0069] [Example 1] Examples of the first and second embodiments will be described with reference to FIGS.

[0070] In this Example 1, feasibility was verified through simulation. Two robots (hereinafter, the first robot R1 and the second robot R2) serving as mobile objects 3 were assumed to move toward their respective targets. The first robot R1 and the second robot R2 were initially positioned at (-11, 8) and (-11, -8), respectively, as shown in FIG. 13 . Each robot R1 and R2 was assumed to move within a 10×20 m square area, with the origin of the square at (-11, 0). Each robot R1 and R2 was expected to reach their target within 90 seconds. The experiment was conducted under a scenario in which each robot R1 and R2 crossed a road while avoiding collisions. Due to the height of each robot R1 and R2, the LIDAR unit serving as the mobile object sensor unit 311 could not detect the body of the other robot, which posed a risk of the robots colliding with each other, as shown in FIG. 12(a). In this example, to simplify the experiment, a restricted area where virtual obstacles V1 and V2 are placed at the intersection is defined in advance as shown in Fig. 12(b). The virtual obstacles V1 and V2 in the restricted area represent the restricted areas of the robots R1 and R2, respectively (the virtual obstacle V1 in the restricted area for the robot R1, and the virtual obstacle V2 in the restricted area for the robot R2).

[0071] We implemented a prototype of the system using the Robot Operating System (ROS). We used the ROS 2 Humble Hawksbill version. Experiments were conducted through simulation using Gazebo. A 3D model of the TurtleBot3 Waffle was used as the robot model. This robot was equipped with two sensors: a wheel odometry sensor and a 2D LIDAR unit. The maximum range of the 2D LIDAR unit was set to 20 meters. The 3D model of the field, shown in Figure 12(d), was modeled based on an actual intersection located southwest of the Shibaura Institute of Technology Toyosu Campus in Koto Ward, Tokyo. As shown in Figure 12(c), the experimental field was modeled using image sensor data consisting of a point cloud collected using a LIDAR sensor. The PLATEAU model was then placed and edited to match the details obtained from the image sensor data consisting of a point cloud. The center of the map is visualized as a box, as shown in Figure 12(d). The Navigation2 package was used for the autonomous navigation of each robot, R1 and R2. Robots R1 and R2 navigated autonomously toward the designated targets (-11, -4) and (-11, 4) in that order.

[0072] Each robot, R1 and R2, uses the adaptive Monte Carlo localization (AMCL) method to estimate its own position using data from its own LIDAR unit. Each robot, R1 and R2, predicted and corrected its posture using a package called "robot localization." We recorded exceptions, such as collisions between robots, the routes each robot, R1 and R2, took in Gazebo, and "OUT OF BOUNDS" when each robot, R1 and R2, left the navigation area, and "TIMEOUT" when the robot took too long to reach the goal. We also employed benchmarks in which our system was not used, a common method in which the localization map and obstacle avoidance map were shared, and a separation method in which the localization map and obstacle avoidance map were separated.

[0073] A total of 30 tests were conducted, with 10 tests for each method (benchmark, common method, and separated method). Figure 13 shows the paths traveled by robots R1 and R2 in the first trial of each method. In the benchmark, the robots traveled dangerously close to each other, resulting in collisions in four of the ten trials. In the common method, robots R1 and R2 were "out of range" in nine of the ten trials. In the separated method, all ten trials were successful, with robots R1 and R2 completing a full lap around the restricted area. This confirmed that the robots could be safely guided without sacrificing their travel efficiency, especially by adopting a hierarchical restricted area.

[0074] [Example 2] An example of the third embodiment will be described with reference to FIGS.

[0075] The goal of this example is to detect blind spots within a 12m x 12m area in a specified room (Shibaura Institute of Technology, Toyosu Campus Research Building 14Q32) from multiple first-person viewpoints. As shown in Figure 14, LIDAR sensors were installed at viewpoints A to E in the specified room. From viewpoint E, a chair S on the left side is hidden in the blind spot by a central partition P. Blind spots were detected based on the principle of converting image sensor data consisting of point clouds acquired by the LIDAR into rectangular voxels, and determining that a blind spot exists if an object voxel exists on the line between each pair of voxels. Figure 15 shows the results. Figure 15(a) shows visual information of viewpoint E as seen from viewpoint E, and the blind spot of viewpoint E cannot be seen from the same viewpoint E. On the other hand, Figure 15(b) shows visual information of viewpoint E as seen from viewpoint A, where the blind spot for viewpoint E is surrounded by a frame from viewpoint A. In order to improve safety during autonomous driving of multiple moving bodies 3, it is advisable to integrate the blind spot areas of these multiple viewpoints and reflect this in the control.

[0076] [Example 3] An example of the fourth embodiment will be described with reference to FIGS.

[0077] <Methodology> This system uses a LIDAR sensor network to extract blind spots from any viewpoint on the field. Because LIDAR uses lasers to acquire 3D data, blind spots occur behind real obstacles for the moving object 3. By using multiple sensor devices 1, the coverage area expands as the number of sensor devices 1 increases, ultimately reducing or eliminating blind spots. Since this system can acquire 3D data for the entire field, it can detect blind spots from any viewpoint on the field.

[0078] This system detects blind spots in first-person perspectives by using a machine learning model that performs semantic segmentation to classify each voxel (space separated by a fixed interval). This semantic segmentation model is a supervised learning model that is created by learning as input data labeled for each voxel as one of three types: blank, object (real-world obstacle), or viewpoint, and as output data labeled for each voxel as one of four types: blank, object (obstacle), viewpoint, or blind spot.

[0079] <Procedures and Algorithms> Image sensor data consisting of a point cloud obtained from the sensor network is divided into multiple voxels and returned. Based on the viewpoint coordinates received from the moving object 3 and the converted voxel data, one of three labels is assigned to each voxel: blank, object (real-world obstacle), or viewpoint. The labeled voxel data is used as input and semantic segmentation is performed to classify each voxel, outputting voxel data labeled with blind spot information.

[0080] <Experimental system for evaluation> Experimental scenario The experiment was conducted based on a scenario in which a two-dimensional planar grid consisting of nine 3x3 cells was used, with at least one cell representing an object (a real obstacle), as shown in Figure 16, and blind spots in the viewpoint from any one cell were to be detected.

[0081] Experimental specifications We verified the feasibility of the proposed system, which detects spatial blind spots using machine learning for semantic segmentation. We applied the semantic segmentation method, which classifies images pixel by pixel, to blind spot detection. This semantic segmentation model was implemented based on U-Net, which is used for semantic segmentation in 2D images. U-Net is a model consisting of an encoder and decoder, and by using the feature map extracted by the encoder for upsampling, it is possible to capture object position information more accurately.

[0082] The dataset used a 2D planar grid consisting of nine 3x3 cells, as described above. For each cell, the input data was labeled with one of three types: blank, object, or viewpoint, while the ground truth data was labeled with one of four types: blank, object, viewpoint, or blind spot. In this experiment, 2,304 pieces of data were divided into training data and test data in a 7:3 ratio for training.

[0083] The dataset was created by labeling using the blind spot detection method described in the third embodiment. Figure 11 shows how the ground truth data was created. Labeling is performed by connecting the center of the viewpoint and the center of the blank space with a first line segment A and determining whether the first line segment A overlaps with an obstacle. First, if the obstacles are adjacent, the centers of the adjacent obstacles are connected with a second line segment B. If the obstacles are not adjacent, the center of the obstacle is used for determination. Intersection determination is performed between the first line segment A and the second line segment B, and between the first line segment A and the center of the obstacle. If an intersection occurs, the blank space is considered a blind spot. The program was run on a laptop computer with Windows 11. The GPU used was a GeForce RTX 3060 Laptop GPU, manufactured by NVIDIA.

[0084] Evaluation indicators The evaluation was performed from the perspective of blind spot detection accuracy. The evaluation index used was mIoU (mean Intersection over Union). mIoU is the average value of IoU calculated for each class, so it is less likely to result in an unfair evaluation even if there is a bias in the class.

[0085] <Evaluation results> In this experiment, we describe the results of detecting blind spots using semantic segmentation on a 3x3 two-dimensional grid. Figure 16 shows an example of blind spot detection results, demonstrating that the model can detect blind spots cell by cell. Figure 17 shows the loss and mIoU (loss over unit of error over unit of error) as the error and accuracy of the detection results, respectively. The mIoU of the trained model was high at 0.816, demonstrating its ability to capture features. Figure 18 shows the training process of the machine learning model. Since the model's performance on the training data and test data was similar, it is highly likely that the model has high generalization capabilities and can provide reliable predictions even for unknown data. These findings demonstrate the feasibility of detecting blind spots using a machine learning model that performs semantic segmentation.

[0086] [Example 4] <Setting the conditions for the experiment and analysis (equipment, location, parameters)>

[0087] (Experimental scenario) ·vehicle Number of vehicles: 2 (hereafter referred to as vehicle 1 = V1, vehicle 2 = V2) ·Initial position V1: (-14, 8) V2: (-10, 12) Driving route -Rotate around a 4x4 square Center coordinates of the square: (-12, 10) Travel direction V1: Clockwise V2: Counterclockwise ·Destination V1: (-14, 7) V2: (-9, 12) Driving area: Square (-8 > x > -16, 14 > y > 6 ) Of these, the range (-13 > x > -11 and 11 > y > 9) is excluded from the allowable area. Allowable arrival time: within 90 seconds Driving condition Due to height restrictions on the onboard sensors, the vehicle cannot sense both. No entry area Set as per Note 1 V1: Orange ·V2: Blue - Set at the point where a collision is expected in the scenario

[0088] (Experimental environment) ·simulation Software used: Gazebo, ROS2 Humble Vehicle: Turtlebot3 Waffle 3D model of driving environment: Location: Shibaura Institute of Technology Toyosu Campus Usage Data: Point cloud data acquired from a 3D LIDAR sensor in the real world ·Open Data PLATEAU by the Ministry of Land, Infrastructure, Transport and Tourism [1] Driving map Generation method: Generated using point cloud data obtained from a virtual LIDAR installed in the simulation. Virtual LIDAR: VLP-16

[0089] <Explanation of the results in Figure 19> In the experiment, 10 trials were conducted for each method of marking no-entry areas on the driving map (Benchmark, Common, Separation). Figure 19 shows the driving paths of V1 and V2 in the third trial for each method. In the Benchmark method, collisions between vehicles occurred in all 10 trials. Of these, a collision occurred at (-10, 8) in the third trial of the Benchmark method. In the Common method, collisions between vehicles occurred in five out of ten trials, and the vehicles deviated from the permitted driving area in two trials. Of these, V1 deviated from the permitted driving area in the third trial of the Common method. In the Separation method, the vehicle completed the driving safely in all ten trials without collisions or deviations from the permitted driving area.

[0090] <Practical benefits to society as a result of the results> By providing autonomous vehicles with no-entry areas created based on information from infrastructure sensors, it becomes possible for the vehicle to avoid danger while driving in areas that could be blind spots.

[0091] [Example 5] <Setting the conditions for the experiment and analysis (equipment, location, parameters)>

[0092] (Data acquisition experiment environment) Location Shibaura Institute of Technology Toyosu Campus (Koto-ku, Tokyo) Equipment used LiDAR Velodyne VLP-16 ·Livox Avia. Sensor device NVIDIA Jetson Nano Edge computer NVIDIA Jetson Orin Nano (Experimental environment for obtaining results) ·Equipment used raytrek GEFORCE RTX

[0093] <Explanation of the results in Figure 20> The table in Fig. 20(a) shows the detection accuracy of the blind spot detection system using machine learning for point cloud data acquired by LiDAR. As an evaluation metric, we use mean intersection over union (mIoU), which is commonly used in semantic segmentation.

[0094] The evaluation is carried out in the following cases: The evaluation was performed with a grid size of 8x8. The grid size is evaluated as 16x16. The grid size is evaluated as 32x32. The evaluation uses eight data patterns obtained in the experiment. In an 8x8 grid, the correlation coefficient was 0.99 or higher for all data patterns. In a 16x16 grid, the correlation coefficient was 0.97 or higher for all data patterns. In a 32x32 grid, the coefficient was 0.81 or higher for all data patterns.

[0095] Figure 20(b) is a visualization of the point cloud data obtained in the experiment.

[0096] Figure 20(c) is an example of visualization of the detection results by the proposed system for the point cloud data shown on the left. The columns of the figure show the input, the detection results by the proposed system, and the correct data from the right. The rows of the figure show the grid size, which is 8x8, 16x16, and 32x32 from top to bottom.

[0097] The graph in Figure 20(d) shows the cumulative distribution function (DCF) of the detection time for the proposed system and the comparative method (geometric calculation method). The vertical axis of the graph shows the cumulative probability. The vertical axis of the graph shows the detection time [s]. The points on the left show the detection time for the proposed system. The points on the right show the detection time for the comparative method (geometric calculation method). The three graphs show the detection time for 8x8, 16x16, and 32x32 grids from the left.

[0098] On an 8x8 grid, the mean detection time for the proposed method was 0.0076 seconds, the median was 0.0057 seconds, and the standard deviation was 0.0434 seconds. The mean detection time for the comparison method was 0.0195 seconds, the median was 0.0194 seconds, and the standard deviation was 0.0017 seconds.

[0099] On a 16x16 grid, the mean detection time for the proposed method was 0.0087 seconds, the median was 0.0066 seconds, and the standard deviation was 0.0412 seconds. The mean detection time for the comparison method was 0.301 seconds, the median was 0.300 seconds, and the standard deviation was 0.0131 seconds.

[0100] On a 32x32 grid, the mean detection time for the proposed method was 0.012 seconds, the median was 0.01 seconds, and the standard deviation was 0.0478 seconds. The mean detection time for the comparison method was 3.27 seconds, the median was 3.27 seconds, and the standard deviation was 0.128 seconds.

[0101] <Practical benefits to society as a result of the results> By sensing the entire space without any blind spots and then detecting blind spots from each viewpoint, it becomes possible for each party to recognize in real time what they cannot see.

[0102] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the illustrated embodiments. Various modifications and variations can be made to the illustrated embodiments within the same scope as the present invention or within an equivalent scope. [Explanation of symbols]

[0103] 1...Sensor device 11...Environment side sensor part 12...Sensor side transmitter 2. Server device 21...Consolidation section 22...Environmental Information Preservation Department 23...Digital Twin Generation Department 24...Situation Analysis Department 25...Traffic Control Department 26...Map generation section 27...Map information storage section 28...Server-side transmission unit 3...Mobile 31...Sensor device 311...Mobile body side sensor unit 312…IMU section 313...Wheel sensor part 32...In-vehicle computer device 321…Self-position estimation unit 322...Path generation unit 333...Control unit 33...Drive mechanism

Claims

1. A mobile object traffic control system comprising one or more sensor devices arranged in an environment of a real space, a server device connected to the sensor devices in a state capable of communicating with the sensor devices, and mobile objects connected to the server device in a state capable of communicating with the sensor devices, and controlling traffic of the mobile objects in the real space, The sensor device includes: an environment-side sensor unit that acquires image sensor data consisting of a point cloud in real space; a sensor-side transmitting unit that transmits image sensor data consisting of a point cloud in real space acquired by the environment-side sensor unit to the server device; The server device an aggregation unit that aggregates image sensor data consisting of point clouds in real space transmitted from the sensor devices; a digital twin generation unit that generates a digital twin corresponding to the real space based on image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit; a situation analysis unit that analyzes the situation in the real space based on information about the digital twin generated by the digital twin generation unit; a traffic control unit that places virtual obstacles on the digital twin to regulate traffic of the moving object based on the analysis result of the situation in the real space by the situation analysis unit; and a map generation unit that generates a map including the virtual obstacle based on information about the digital twin in which the virtual obstacle is placed by the traffic regulation unit; and a server-side transmitting unit that transmits to the mobile unit information about a map including the virtual obstacle generated by the map generating unit; the moving body includes a route generation unit that generates a travel route for the moving body in real space based on information about a map including a virtual obstacle transmitted from a server device; A mobile traffic control system comprising: a control unit that controls the travel of the mobile object in real space based on the travel route of the mobile object generated by the route generation unit.

2. the situation analysis unit detects an area where a risk of collision between the moving object and the vehicle is estimated to be high based on a situation in real space; The mobile traffic control system of claim 1, wherein the traffic regulation unit places virtual obstacles to regulate traffic of the mobile body in areas in the digital twin detected by the situation analysis that are estimated to have a high risk of collision with the mobile body.

3. The situation analysis unit detects blind spots in the real space in the digital twin, The mobile traffic control system according to claim 1, wherein the traffic regulation unit places a virtual obstacle for regulating traffic of the mobile body in a blind spot area of ​​the real space in the digital twin detected by the situation analysis.

4. The situation analysis unit detects blind spots of the moving object traveling in a real space in the digital twin, The mobile traffic control system according to claim 3, wherein the traffic regulation unit places a virtual obstacle for regulating traffic of the mobile body in a blind spot area of ​​the mobile body in the digital twin detected by the situation analysis.

5. 5. The mobile traffic control system according to claim 4, wherein the situation analysis unit geometrically determines whether a real obstacle exists between the mobile body and a predetermined space, and if it determines that a real obstacle exists between the mobile body and the predetermined space, detects the side of the real obstacle opposite the mobile body as a blind spot area of ​​the mobile body.

6. The situation analysis unit includes a voxel partitioning unit that partitions the digital twin generated by the digital twin generation unit into a plurality of voxels; a voxel extraction unit that extracts voxels in blind spot areas of the moving body from the digital twin that has been partitioned into a plurality of voxels by the voxel partition unit; The mobile traffic control system according to claim 4, further comprising a blind spot area detection unit that detects blind spot areas of the moving body in the digital twin based on the voxels of the blind spot areas extracted by the voxel extraction unit.

7. 7. The mobile traffic control system of claim 6, wherein the voxel extraction unit connects the center of a voxel of the viewpoint of the mobile body with the center of each blank voxel where no real obstacle exists, with a first line segment, and if the first line segment passes through the center of a voxel of a real obstacle, extracts the blank voxel connected by the first line segment as a voxel in a blind spot area.

8. 8. The mobile traffic control system according to claim 7, wherein the voxel extraction unit connects the centers of adjacent voxels of real obstacles with second line segments, and when the first line segment intersects with the second line segment, extracts blank voxels connected by the first line segment as voxels in a blind spot area.

9. The mobile traffic control system of claim 6, wherein the situation analysis unit extracts voxels in the blind spot area of ​​the moving body from the voxels of the moving body and real obstacles in the digital twin using a learning model that has been machine-learned using test data regarding voxels of the moving body and real obstacles in the digital twin and correct answer data regarding voxels in the blind spot area of ​​the moving body in the digital twin for the test data.

10. The moving body is a moving body side sensor unit that acquires image sensor data consisting of a point cloud in a real space around the moving body; a mobile object position estimation unit that estimates a self-position of the mobile object based on information relating to a predetermined map in real space and image sensor data consisting of a point cloud in real space around the mobile object acquired by the mobile object side sensor unit, 2. The mobile traffic control system according to claim 1, wherein the route generation unit generates a travel route for the mobile body in real space based on information relating to the self-position of the mobile body estimated by the mobile body position estimation unit and information relating to a map including a virtual obstacle generated by the map generation unit.

11. In the server device, the map generation unit generates a map for obstacle avoidance including a virtual obstacle and a map for self-location estimation not including the virtual obstacle; the server-side transmitting unit transmits to the moving body information relating to an obstacle avoidance map including a virtual obstacle and information relating to a self-position estimation map not including a virtual obstacle; In the moving body, the moving body position estimation unit estimates the self-position of the moving body in real space based on information about a map for self-position estimation that does not include virtual obstacles and image sensor data consisting of a point cloud in real space around the moving body acquired by the moving body sensor unit; The mobile traffic control system according to claim 10, wherein the route generation unit generates a travel route for the mobile body in real space based on information relating to the self-position of the mobile body estimated by the mobile body position estimation unit and information relating to an obstacle avoidance map including a virtual obstacle.

12. In the server device, the traffic regulation unit places a first virtual obstacle on the digital twin to regulate traffic of a first moving body, based on the analysis result of the situation in the real space by the situation analysis unit, and places a second virtual obstacle on the digital twin to regulate traffic of a second moving body at a location different from the first virtual obstacle; the map generation unit generates a first map including a first virtual obstacle based on information about a digital twin in which a first virtual obstacle has been placed by the traffic regulation unit, and generates a second map including a second virtual obstacle based on information about a digital twin in which a second virtual obstacle has been placed by the traffic regulation unit; the server-side transmitting unit transmits information about a first map including a first virtual obstacle generated by the map generating unit to the first moving body, and transmits information about a second map including a second virtual obstacle generated by the map generating unit to the second moving body; In the first moving body, the path generation unit generates a travel path for the first moving object to avoid the first virtual obstacle in real space, based on information about a first map including the first virtual obstacle transmitted from the server device; and the control unit controls travel of the first moving object based on the travel route of the first moving object generated by the route generation unit; In the second moving body, the path generation unit generates a travel path for the second moving object to avoid the second virtual obstacle in real space, based on information about a second map including the second virtual obstacle transmitted from the server device; and The mobile traffic control system according to claim 1 , wherein the control unit controls the travel of the second mobile object based on the travel route of the second mobile object generated by the route generation unit.

13. A mobile object traffic control system comprising one or more sensor devices arranged in an environment of a real space, a server device connected to the sensor devices in a state capable of communicating with the sensor devices, and mobile objects connected to the server device in a state capable of communicating with the sensor devices, and controlling traffic of the mobile objects in the real space, The sensor device includes: an environment-side sensor unit that acquires image sensor data consisting of a point cloud in real space; a sensor-side transmitting unit that transmits image sensor data consisting of a point cloud in real space acquired by the environment-side sensor unit to the server device; The server device an aggregation unit that aggregates image sensor data consisting of point clouds in real space transmitted from the sensor devices; a digital twin generation unit that generates a digital twin corresponding to the real space based on image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit; a situation analysis unit that analyzes the situation in the real space based on information about the digital twin generated by the digital twin generation unit; a traffic control unit that places virtual obstacles on the digital twin to regulate traffic of the moving object based on the analysis result of the situation in the real space by the situation analysis unit; and a map generation unit that generates a map including the virtual obstacle based on information about the digital twin in which the virtual obstacle is placed by the traffic regulation unit; and a route generation unit that generates a travel route of the moving object in real space based on information about a map including a virtual obstacle generated by the map generation unit; a server-side transmitting unit that transmits the travel route of the moving object generated by the route generating unit to the moving object; The mobile object traffic control system is characterized in that the mobile object is equipped with a control unit that controls the travel of the mobile object in an actual section based on the travel route of the mobile object transmitted from the server device.

Images