Mobile body autonomous driving assistance system

Abstract

To provide a mobile body autonomous driving assistance system that can assist a safe autonomous driving by a mobile body in real space.SOLUTION: A mobile body autonomous driving assistance system comprises: a sensor device 10 having a first sensor unit 10a provided on the environment side in real space; an edge computer 11 as a server device communicably connected to the sensor device 10; and a mobile body 2 having a second sensor unit 20, which is communicably connected to the edge computer 11. The mobile body 2 generates a driving route of itself by identifying a self position in real space and / or by recognizing surrounding obstacles thereof, on the basis of first image sensor point-cloud data acquired by the first sensor unit 10a of the sensor device 10 in real space and second image sensor point-cloud data acquired by the second sensor unit 20 of the mobile body 2 in real space.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a mobile object autonomous driving support system that supports autonomous driving of a mobile object in real space. [Background technology]

[0002] In recent years, personal mobility vehicles (PMVs) have emerged as a promising solution to transportation needs, including flexible last-mile travel, reducing traffic and carbon emissions, providing exercise and recreation, delivering services, mobility services for residents and tourists, and enabling door-to-door travel.

[0003] Meanwhile, digital twins, which are digital replicas of real-world entities, are gaining attention. Digital twin technology allows physical systems to update their real-time behavior according to feedback generated by the digital twin. The concept of a digital twin framework needs to be discussed not only for cars and trucks, but also for PMVs.

[0004] One of the most important elements of a PMV digital twin is a real-time monitoring system that replicates physical entities in the digital world, and the use of LIDAR (Light-Detection-and-Ranging) technology as a sensor for this is a promising means of collecting spatial information. Data collected by LIDAR can identify the positions of not only PMVs, but also other moving objects such as cars and trucks, and pedestrians (see Non-Patent Document 1). In particular, the use of multiple LIDAR sensors has been studied because it can reduce blind spots in the monitoring area and increase the amount of information in overlapping areas (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] G. Bhatti, H. Mohan, and RR Singh, “Towards the future of smart electric vehicles: Digital twin technology,” Renewable and Sustainable Energy Reviews, vol. 141, p. 110801, 2021. [Non-patent document 2] C. Li, R. Shinkuma, T. Sato, and E. Oki, ``Real-time Data Selection and Merging for 3D-image Sensing Network with Multiple Sensors,'' IEEE Sensors Journal, Vol.21(19), pp.22058-22076, Aug. 2021. Summary of the Invention [Problem to be solved by the invention]

[0006] However, previous research has not addressed safety issues when onboard sensors are disabled. In particular, PMV onboard sensors can easily be disabled due to external hardware problems such as dirt or snow, internal hardware problems, or software problems such as malware infection. Such problems could lead to localization failures, which are fatal errors in autonomous driving. Therefore, PMVs must ensure autonomous driving, enabling them to reach a safe location even if their onboard sensors are disabled. Such issues can occur not only in PMVs, but also in other moving vehicles, such as cars, trucks, and electric wheelchairs, that operate autonomously in the real world.

[0007] The present invention has been made in consideration of the above-mentioned technical background, and aims to provide a mobile object autonomous driving support system that can support safe autonomous driving of a mobile object in real space. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the present invention provides a mobile body autonomous driving assistance system comprising a sensor device arranged in an environment of a real space, a server device connected to the sensor device in a state capable of communicating with the sensor device, and the mobile body connected to the server device in a state capable of communicating with the sensor device, and which assists the autonomous driving of the mobile body in a real space, wherein the sensor device comprises a first sensor unit that acquires first image sensor data consisting of a point cloud in the real space, the mobile body comprises a second sensor unit that acquires second image sensor data consisting of a point cloud in the real space, and the server device or the mobile body comprises a mobile body position identification unit that identifies the position of the mobile body in the real space based on the first image sensor data consisting of the point cloud in the real space acquired by the first sensor unit of the sensor device and the second image sensor data consisting of the point cloud in the real space acquired by the second sensor unit of the mobile body.

[0009] The moving body position determination unit may also include an estimation unit that estimates the self-position of the moving body based on second image sensor data consisting of a point cloud in real space acquired by a second sensor unit of the moving body, and a correction unit that corrects the self-position of the moving body estimated by the estimation unit based on first image sensor data consisting of a point cloud in real space acquired by a first sensor unit of the sensor device.

[0010] The server device or the mobile body may also include an obstacle recognition unit that recognizes obstacles in real space based on first image sensor data consisting of a point cloud in real space acquired by a first sensor unit of the sensor device and second image sensor data consisting of a point cloud in real space acquired by a second sensor unit of the mobile body.

[0011] The obstacle recognition unit may also include a detection unit that detects obstacles around the moving body, a tracking unit that captures the trajectory of the obstacle based on the detection result of the obstacle by the detection unit, and a prediction unit that predicts the position of the obstacle after a predetermined time has passed based on the trajectory of the obstacle captured by the tracking unit.

[0012] The server device or the moving body may also include a path generation unit that generates a travel path for the moving body in real space based on the position of the moving body in real space identified by the moving body position identification unit and the position of an obstacle in real space identified by the obstacle recognition unit.

[0013] Furthermore, the server device or the mobile body may include a selection unit that selects image sensor data consisting of a point cloud in real space, either first image sensor data consisting of a point cloud in real space acquired by a first sensor unit of the sensor device or second image sensor data consisting of a point cloud in real space acquired by a second sensor unit of the mobile body, and the mobile body position identification unit may identify the self-position of the mobile body in real space based on the first image sensor data or the second image sensor data consisting of a point cloud in real space selected by the selection unit.

[0014] The selection unit may also preferentially select second image sensor data consisting of a point cloud in real space acquired by a second sensor unit of the moving body, and if it determines that the second image sensor data consisting of a point cloud in real space is invalid, select first image sensor data consisting of a point cloud in real space acquired by a first sensor unit of the sensor device.

[0015] The moving body may also include a control unit that controls the traveling of the moving body based on the traveling path of the moving body in real space generated by the path generation unit.

[0016] The server device may also include an aggregation unit that aggregates first image sensor data made up of a point cloud in real space acquired by the first sensor unit.

[0017] The server device may also include a map generation unit that generates a map of the real space based on first image sensor data consisting of a cloud of points in the real space aggregated by the aggregation unit, and the moving body position identification unit may use information regarding the map of the real space generated by the map generation unit when identifying the position of the moving body.

[0018] The server device may also include a digital twin generation unit that generates a digital twin corresponding to the real space based on first image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit, and the moving body position identification unit may use information regarding the digital twin corresponding to the real space generated by the digital twin generation unit when identifying the position of the moving body.

[0019] In addition, the digital twin generation unit may detect the moving body in real space using a learning model that has been machine-learned in advance to detect the moving body based on first image sensor data consisting of a point cloud in real space. [Effects of the Invention]

[0020] According to the present invention, the position of a moving object in real space is determined based on first image sensor data consisting of a point cloud in real space acquired by a first sensor unit of an environment-side sensor device and second image sensor data consisting of a point cloud in real space acquired by a second sensor unit of the moving object, thereby enabling the position of the moving object in real space to be determined with high accuracy. Moreover, even if the second sensor unit arranged on the moving object is disabled due to an external hardware problem such as dirt or snow, an internal hardware problem, or a software problem such as malware infection, the first image sensor data consisting of a point cloud in real space acquired by the first sensor unit arranged on the environment side of the real space can be used to determine the moving object's own position and recognize obstacles, thereby enabling the moving object's travel path to be continuously generated and enabling safe autonomous travel of the moving object. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram showing the overall configuration of a mobile object autonomous driving assistance system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram illustrating a configuration of the edge computing system of FIG. 1. [Figure 3] FIG. 2 is a diagram illustrating a configuration of the moving body of FIG. [Figure 4] FIG. 2 is a diagram showing first image sensor data consisting of a point cloud in real space. [Figure 5] FIG. 1 is a diagram showing a digital twin (including the detection results of moving objects) corresponding to the real space. [Figure 6] FIG. 10 is a diagram showing second image sensor data consisting of a point cloud in real space. [Figure 7] FIG. 10 is a diagram showing simulation results of estimated positions and corrected positions of a moving object. [Figure 8] FIG. 10 is a diagram showing a state in which obstacles around a moving object are recognized. [Figure 9] FIG. 10 is a diagram showing a state in which a travel path of a moving object is generated. [Figure 10]FIG. 10 is a diagram showing a configuration of a moving body of a moving body autonomous driving assistance system according to a second embodiment of the present invention. [Figure 11] FIG. 10 is a diagram showing the configuration of a moving body in a moving body autonomous driving assistance system according to a third embodiment of the present invention. [Figure 12] 10A and 10B are diagrams showing cumulative distributions of processing times when processing required for self-localization in the present invention is performed by (a) one edge computer and (b) four edge computers. [Figure 13] 10A and 10B are diagrams showing the paths traveled by the robot in benchmark results for a normal scenario and an abnormal scenario according to Example 1. FIG. [Figure 14] 1A and 1B are diagrams showing the path of the robot in the results of the present invention in a normal scenario and an abnormal scenario according to Example 1. FIG. [Figure 15] FIG. 10 is a diagram showing the DTW distance between the reference point and the passing point of the robot in each trial according to the first embodiment. [Figure 16] FIG. 10 is a diagram showing the results when one micromobility vehicle is autonomously driven on a simulator according to the second embodiment. [Figure 17] FIG. 11 is a diagram showing the results when ten micromobilities are made to travel autonomously on a simulator according to the third embodiment. [Figure 18] FIG. 10 is a diagram showing the results when ten micromobilities are made to travel autonomously on a simulator according to Example 4. [Figure 19] FIG. 11 is a diagram showing the results of autonomous driving of one micromobility vehicle in a real environment and ten micromobilities on a simulator according to Example 5. [Figure 20] FIG. 10 is a diagram showing the driving trajectories of each micromobility when a dynamic load balancing method using multiple system statistics is used. DETAILED DESCRIPTION OF THE INVENTION

[0022] First Embodiment Next, a first embodiment of a mobile object autonomous driving assistance system according to the present invention will be described with reference to FIGS.

[0023] As shown in Figure 1, this system is composed of an edge computing system 1 and an autonomous mobile object 2, which are connected to each other via a wireless network so that they can communicate with each other. The specific configurations of the edge computing system 1 and the mobile object 2 are described below. The mobile object 2 refers to a vehicle that can travel autonomously, such as a PMV (Personal Mobility Vehicle), automobile, truck, electric wheelchair, or delivery robot.

[0024] [Edge Computing System 1 Configuration] As shown in FIG. 2, the edge computing system 1 includes a plurality of sensor devices 10 arranged in the environment of the real space, and an edge computer 11 as a server device connected to each sensor device 10 in a communicable state via a local network.

[0025] The sensor device 10 has a first sensor unit 10a placed on a static object such as a pillar in the real-space environment, and after acquiring first image sensor data consisting of a point cloud in the entire real space including the moving object and its surroundings, performs predetermined processing on the first image sensor data and transmits it to the edge computer 11.

[0026] The first sensor unit 10a is, for example, a sensor called LiDAR (light detection and ranging). This LiDAR is a type of sensor that uses laser light, and by scanning and irradiating an object with laser light having a higher radiant flux density than radio waves and a short wavelength, it acquires image sensor data consisting of a point cloud in real space and accurately detects not only the distance to the object but also the position and shape of the object.

[0027] Examples of this LiDAR include a method of acquiring image sensor data in all directions of 360 degrees by rotating a light emitter that emits multiple laser beams, and a method of acquiring image sensor data by directly irradiating laser beams within a predetermined light irradiation angle range. Also, the more light emitters that emit laser beams, the higher the accuracy of the image sensor data, but the higher the cost, so an inexpensive LiDAR with a small number of light emitters may be used.

[0028] In addition, when the sensor device 10 transmits the first image sensor data consisting of a point cloud in real space acquired by the first sensor unit 10a to the edge computer 11, the sensor device 10 may perform filtering on the first image sensor data or fragment the data into multiple packets.

[0029] 2, the edge computer 11 includes an aggregation unit 111 that aggregates first image sensor data consisting of a point cloud in real space transmitted from the first sensor unit 10a of the sensor device 10, a map generation unit 112 that generates a map of the real space based on the first image sensor data consisting of the point cloud in real space, a digital twin generation unit 113 that generates a digital twin corresponding to the real space based on the first image sensor data consisting of the point cloud in real space, an information storage unit 114 that stores information about the map (map data) and information about the digital twin (digital twin data), and a transmission unit 115 that transmits information about the map (map data) and information about the digital twin (digital twin data) to the mobile object 2. Note that a digital twin refers to a digital reproduction of a real-space environment in cyberspace, obtained by acquiring information about the real space using a sensor such as LIDAR.

[0030] The aggregation unit 111 aggregates first image sensor data consisting of a point cloud in real space acquired by the first sensor unit 10a of each sensor device 10. Specifically, the aggregation unit 111 aggregates each frame of the first image sensor data consisting of a point cloud in real space by synthesizing them in chronological order based on timestamps and aligning them in three-dimensional space. Note that when aggregating the image sensor data consisting of a point cloud in real space, the aggregation unit 111 may also perform other processing such as smoothing of the point cloud of the first image sensor data.

[0031] The map generation unit 112 generates a two-dimensional or three-dimensional map of the real space based on the first image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit 111, and stores information about the map (map data) in the information storage unit 114. For example, Fig. 4 shows an example of map data obtained by removing the ceiling and floor from the first image sensor data consisting of a point cloud in the real space (a predetermined indoor space for the experiment), converting the xyz coordinates into xy coordinates, and then converting the processed point cloud into a format that can be used for autonomous driving in ROS (Robot Operating System).

[0032] The digital twin generation unit 113 generates a three-dimensional digital twin corresponding to the real space based on first image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit 111, and stores information about the digital twin (digital twin data) in the information storage unit 114. When generating a three-dimensional digital twin corresponding to the real space, the digital twin generation unit 113 detects a moving object 2 in the real space, and includes information about the position, orientation (travel direction), speed, etc. of the moving object 2 in the information about the digital twin (digital twin data). For example, FIG. 5 shows information (digital twin data) about a three-dimensional digital twin corresponding to the real space (a specified indoor space for experimentation) generated based on first image sensor data consisting of a point cloud in the real space, in which each moving object 2 detected by the digital twin generation unit 113 is surrounded by a box.

[0033] The digital twin generation unit 113 may detect the moving body 2 in real space (or a digital twin corresponding to real space) using a learning model (AI: Artificial Intelligence) that has been machine-learned to detect in advance the position, orientation, speed, etc. of the moving body 2 based on first image sensor data consisting of a point cloud in real space. In this case, the digital twin generation unit 113 detects and tracks the three-dimensional posture (position, orientation, speed) of the moving body 2 from information related to the digital twin (digital twin data).

[0034] The information storage unit 114 stores information (map data) about the map of the real space generated by the map generation unit 112, and information (digital twin data) about the digital twin corresponding to the real space generated by the digital twin generation unit 113.

[0035] The transmitting unit 115 transmits information (map data) regarding the map of the real space stored in the information storage unit 114 and information (digital twin data) regarding the digital twin corresponding to the real space to the corresponding mobile body 2 via a wireless network.

[0036] [Configuration of Mobile Unit 2] As shown in Figure 3, the mobile body 2 is equipped with one or more second sensor units 20 that acquire image sensor data consisting of a point cloud in real space, a first receiving unit 21 that receives information (map data) related to the map of real space and information (digital twin data) related to the digital twin transmitted from the edge computer 11, a second receiving unit 22 that receives second image sensor data consisting of the point cloud in real space acquired by the second sensor unit 20, an on-board computer 23 installed inside the mobile body 2, and a drive mechanism 24 that drives the mobile body 2.

[0037] The second sensor unit 20 is disposed on the surface of the moving object 2, acquires second image sensor data consisting of a point cloud in real space around the moving object 2, performs predetermined processing on the second image sensor data, and transmits the data to the on-board computer 23. Similar to the first sensor unit 10a disposed on the environment side of real space, a so-called LiDAR (light detection and ranging) sensor is preferably used as the second sensor unit 20. For example, Fig. 6 shows an example of second image sensor data consisting of a point cloud in real space acquired by the second sensor unit 20 disposed on the moving object 2.

[0038] The on-board computer 23 includes a mobile body position determination unit 231 that determines the self-position of the mobile body 2, an obstacle recognition unit 232 that recognizes obstacles around the mobile body 2 in real space, a path generation unit 233 that generates a travel path for the mobile body 2, and a control unit 234 that controls the motor and various devices of the mobile body 2.

[0039] The moving object position specifying unit 231 includes an estimation unit 231a that estimates the self-position of the moving object 2, and a correction unit 231b that corrects the self-position of the moving object 2 whose position has been estimated by the estimation unit 231a.

[0040] The estimation unit 231a estimates the position of the moving object 2 on the map of real space based on information (map data) relating to the map of real space received by the first receiving unit 21 and second image sensor data consisting of a point cloud received by the second receiving unit 22. Note that the information (map data) relating to the map of real space received by the first receiving unit 21 is generated in the edge computer 11 based on the first image sensor data consisting of a point cloud in real space, as described above. Note that in this embodiment, map data generated in the edge computer 11 based on the first image sensor data is used, but existing map data generated by another device may also be used.

[0041] The correction unit 231b corrects the position of the moving body 2 on the map of real space by comparing the detected position of the moving body 2 included in the information (digital twin data) about the digital twin corresponding to the real space received by the first receiving unit 21 with the position of the moving body 2 estimated by the estimation unit 231a. Note that the correction unit 231b also processes measurement values ​​received with a delay from the edge computer 11 in order to make corrections taking into account communication delays of data from the edge computer 11 to the moving body 2. For example, FIG. 7 shows the estimated position of the moving body 2 by the estimation unit 231a and the corrected position of the moving body 2 by the correction unit 231b.

[0042] Thus, the information (digital twin data) relating to the digital twin corresponding to the real space received by the first receiving unit 21 is generated using first image sensor data consisting of a point cloud in the real space acquired by the first sensor unit 10a of the sensor device 10 arranged on the environment side in the real space, while the position of the moving body 2 estimated by the estimation unit 231a is generated using second image sensor data consisting of a point cloud in the real space acquired by the second sensor unit 20 arranged on the moving body 2 in the real space. Therefore, the moving body position identification unit 231 can accurately identify the position of the moving body 2 in the real space based on two different image sensor data: the first image sensor data consisting of a point cloud in the real space acquired by the first sensor unit 10a of each sensor device 10 arranged on the environment side in the real space, and the second image sensor data arranged on the moving body 2 and consisting of a point cloud in the real space.

[0043] The mobile object position identification unit 231 may, for example, predict and correct the three-dimensional attitude (position, orientation, and velocity) of the mobile object 2 using an extended Kalman filter (EKH). This scheme saves a history of the estimated values ​​and measurements of the extended Kalman filter, and if the timestamp embedded in the data from the edge computer 11 is older than the current time, it returns to the last state before the timestamp. In this scheme, all measurements up to the current time are processed, and the estimated values ​​are recalculated.

[0044] The obstacle recognition unit 232 recognizes obstacles such as pillars, walls, other moving bodies 2, pedestrians, etc. in the real space based on information (map data) relating to the map of the real space received by the first receiving unit 21 and second image sensor data consisting of a point cloud in the real space received by the second receiving unit 22, and is composed of a detection unit 232a, a tracking unit 232b, and a prediction unit 232c.

[0045] The detection unit 232a detects the position and size of obstacles around the moving object 2 for each frame of second image sensor data consisting of a point cloud in real space.

[0046] The tracking unit 232b captures the trajectory of the moving obstacle 2 (e.g., another moving object 2 or a pedestrian) by stitching together the obstacle detection results by the detection unit 232a across multiple frames of the second image sensor data consisting of a point cloud in real space.

[0047] The prediction unit 232c predicts the trajectory (position) of the obstacle after a predetermined time (for example, several seconds) from the current time based on the trajectory of the obstacle movement captured by the tracking unit 232b.

[0048] For example, Fig. 8 shows an image diagram in which obstacles are recognized by the obstacle recognition unit 232. Obstacles such as other moving objects 2 and pedestrians recognized by the obstacle recognition unit 232 are enclosed in boxes.

[0049] The path generation unit 233 generates a travel path for the moving body 2 on a map of real space based on information about the position of the moving body 2 in real space identified by the moving body position identification unit 231 and information about obstacles in real space recognized by the obstacle recognition unit 232. Specifically, when a static obstacle (current position) or a dynamic obstacle (predicted position after a predetermined time has elapsed) is recognized for the moving body 2 in real space, the path generation unit 233 generates a travel path for the moving body 2 so as to avoid the static and dynamic obstacles. For example, FIG. 9 is a diagram showing the travel path for the moving body 2 generated by the path generation unit 233, and shows the travel path for the moving body 2 on a map of real space.

[0050] The control unit 234 controls the driving mechanism 24 such as a motor based on information about the travel route of the moving object 2 generated by the route generation unit 233.

[0051] <Second embodiment> Next, a second embodiment of a mobile object autonomous driving assistance system according to the present invention will be described with reference to Fig. 10. Note that only configurations different from the above embodiment will be described below, and the same configurations will be denoted by the same reference numerals and explanations will be omitted.

[0052] In this embodiment, a selection unit 235 is provided on the output side of the first receiving unit 21 and the second receiving unit 22 in the vehicle-mounted computer 23 of the moving body 2, and on the input side of the moving body position identification unit 231 and the obstacle recognition unit 232.

[0053] The selection unit 235 selects either the information (digital twin data) related to the digital twin corresponding to the real space received by the first receiving unit 21, or the second image sensor data consisting of a point cloud in the real space received by the second receiving unit 22. In other words, the information (digital twin data) related to the digital twin corresponding to the real space received by the first receiving unit 21 is generated based on the first image sensor acquired by the first sensor unit 10a arranged on the environment side of the real space, and therefore the selection unit 235 selects the image sensor data consisting of a point cloud in the real space from either the first image sensor data consisting of a point cloud in the real space acquired by the first sensor unit 10a, or the second image sensor data consisting of a point cloud in the real space acquired by the second sensor unit 20 of the moving object 2.

[0054] In addition, the selection unit 235 preferentially selects the second image sensor data when the second receiving unit 22 is able to effectively receive the second image sensor data consisting of a point cloud in real space, but when it determines that the second image sensor data consisting of a point cloud in real space is invalid, it selects the information about the digital twin received by the first receiving unit 21 (digital twin data), that is, the first image sensor data consisting of a point cloud in real space acquired by the first sensor unit 10a of each sensor device 10.

[0055] The timing of disabling the second sensor unit 20 is determined by, for example, measuring the time interval between data arrivals. If the data acquired by the selector 235 is from the second receiver 22 (i.e., the second sensor unit 20), the selector 235 records a timestamp indicating the time at which the data was acquired. On the other hand, if the data acquired by the selector 235 is from the first receiver 21 (i.e., the edge computer 11), the selector 235 refers to the timestamp (indicating the time at which the last packet was transmitted by any of the first sensor units 10a) recorded in the data by the edge computer 11. Then, if the difference between the recorded timestamp and the current time is within a predetermined threshold, the selection unit 235 continues to drop the data from the first receiving unit 21 (data from the edge computer 11) and waits for data from the second receiving unit 22 (data from the second sensor unit 20), while if the difference between the recorded timestamp and the current time exceeds a predetermined threshold, the selection unit 235 determines that the second sensor unit 20 has been disabled, and selects and starts transferring the data from the first receiving unit 21 (data from the edge computer 11). Note that if the selection unit 235 selects the first image sensor data consisting of a point cloud in real space, the correction unit 231b does not need to correct the position of the moving object 2 based on information about the digital twin (digital twin data).

[0056] According to this, even if the second sensor unit 20 arranged on the moving body 2 is disabled due to an external hardware problem such as dirt or snow, an internal hardware problem, or a software problem such as malware infection, the first image sensor data consisting of a point cloud in real space acquired by the first sensor unit 10a arranged on the real space environment side can be used to identify the moving body's own position and recognize obstacles, so that the moving body 2's driving path can be continuously generated, enabling the moving body 2 to drive safely autonomously.

[0057] <Third embodiment> Next, a third embodiment of a mobile object autonomous driving assistance system according to the present invention will be described with reference to Fig. 11. Note that only configurations different from the above-described embodiments will be described below, and the same configurations will be denoted by the same reference numerals and explanations will be omitted.

[0058] In this embodiment, as shown in FIG. 11, a mobile body position identification unit 231, an obstacle recognition unit 232, and a path generation unit 233 are provided in the edge computer 11, and second image sensor data consisting of a point cloud in real space acquired by the second sensor unit 20 of the mobile body 2 is transmitted from the mobile body 2 to the edge computer 11 at any time.

[0059] The moving body position identifying unit 231 identifies the position of the moving body 2 based on information about a map of real space (map data) and information about a digital twin corresponding to the real space (digital twin data) stored in the information storage unit 114, and second image sensor data consisting of a point cloud in real space transmitted from the moving body 2. The method of identifying the self-position of the moving body 2 by the moving body position identifying unit 231 is the same as in the first embodiment.

[0060] The obstacle recognition unit 232 recognizes obstacles such as pillars, walls, other moving bodies 2, pedestrians, etc. in the real space based on information (map data) related to the map of the real space stored in the information storage unit 114 and second image sensor data consisting of a point cloud in the real space transmitted from the moving body 2. The method of recognizing obstacles by the obstacle recognition unit 232 is the same as in the first embodiment.

[0061] The route generation unit 233 generates a travel route for the moving body 2 based on information regarding the position of the moving body 2 in the real space identified by the moving body position identification unit 231 and information regarding obstacles in the real space recognized by the obstacle recognition unit 232.

[0062] The transmission unit 115 transmits information about the travel route of the moving object 2 generated by the route generation unit 233 to the moving object 2 via a wireless network.

[0063] Thereafter, in the moving body 2, the control unit 234 controls the drive mechanism 24 based on the information about the travel route of the moving body 2 transmitted from the edge computer 11.

[0064] In the third embodiment, as in the second embodiment, a selection unit 235 may be provided on the data input side of the moving object position identification unit 231 and the obstacle recognition unit 232. In this case, the selection unit 235 selects, as necessary, information about the digital twin (digital twin data) stored in the information storage unit 114 and second image sensor data consisting of a point cloud in real space transmitted from the moving object 2. As a result, even if the second sensor unit 20 is disabled, the moving object position identification unit 231 can identify its own position and the obstacle recognition unit 232 can recognize obstacles based on information about the digital twin (digital twin data), that is, the first image sensor data consisting of a point cloud in real space acquired by the first sensor unit 10a arranged on the environment side of the real space. This makes it possible to continuously generate a route for the moving object 2 and enable safe autonomous traveling of the moving object 2.

[0065] In the above embodiment, all of the functions of the mobile body position identification unit 231, the obstacle recognition unit 232, and the path generation unit 233 are provided in the mobile body 2 or the edge computer 11, but some of these functions may be provided in the edge computer 11 and other functions may be provided in the mobile body 2, so that the functions of the mobile body position identification unit 231, the obstacle recognition unit 232, and the path generation unit 233 are separately located in the mobile body 2 and the edge computer 11.

[0066] Furthermore, although the edge computer 11 is configured as a single computer, it may also be configured as a plurality of computers. In particular, when the edge computer 11 is configured as a distributed system across a plurality of computers arranged in a cloud-like manner on the Internet, the calculations of each function of the edge computer 11 can be distributed and processed at high speed. For example, FIG. 12 shows the cumulative distribution of processing time when the processing required for self-localization in this system is performed by (a) one edge computer 11 and (b) four edge computers 11. As shown in FIG. 12(a), when processing is performed by one edge computer 11, the processing time is approximately several seconds. However, as shown in FIG. 12(b), when processing is distributed across four edge computers 11, it can be confirmed that the processing time is reduced to less than one second. [Example]

[0067] Next, examples of embodiments of the present invention will be described with reference to FIGS.

[0068] Example 1 (scenario) The feasibility of this system was verified through simulation. A 3D model called TurtleBot3 World was used as the test field, with only one central column left out of nine columns. One robot followed designated waypoints while moving. The robot (mobile body 2) was initially positioned at (1,1). Waypoints were set at the four corners of a 2.0 meter square. The experiment consisted of two scenarios. One assumed normal operation, in which the robot's onboard sensor (second sensor unit 20) worked normally without any problems. The other assumed an abnormality, in which the onboard sensor became disabled during the trial for some reason.

[0069] (Experimental Setup) A prototype of the system was implemented using the robot operating system (ROS 2 Humble Hawksbill). Experiments were conducted using simulations using Gazebo. The robot model used was a 3D model of TurtleBot3 Waffle. This robot is equipped with two sensors: a wheel odometry unit and a 2D LIDAR unit (second sensor unit 20). A package called Navigation2 was used to enable the robot to navigate autonomously.

[0070] A marker was attached to the top of the robot to indicate its posture (position, orientation, and speed). This marker is a square marker that encodes a unique ID. A sensor unit (first sensor unit 10a) was installed on a pillar in the testing area, and the real space where the experiment took place was scanned. The AI ​​periodically detects the posture of the robot by detecting the posture of the marker from the image.

[0071] The robot navigated autonomously according to four given waypoints. The waypoints were set in the following order: (1,1), (-1,1), (-1,-1), (1,-1). The robot did not replan its route while traveling through each waypoint. The robot used adaptive Monte Carlo localization (AMCL) to estimate its pose from data from the onboard 2D LIDAR unit. The robot predicted and corrected its pose using the Extended Kalman Filter (EKF) package. In the normal scenario, the EKF received wheel odometry, pose estimated by AMCL, and pose estimated by the AI. In the abnormal scenario, the EKF only received wheel odometry and pose estimated by the AI.

[0072] The communication delay between the robot and the edge computer was reproduced by delaying the data transmission. The delay was set from 0.0 to 1.0 seconds in 0.1 second increments.

[0073] (Benchmarks and metrics) In this example, we compared our system with a benchmark in which the AI-estimated pose was dropped if the embedded timestamp was older than the current time. To evaluate the dissimilarity of the robot's path, we used the DTW (Dynamic Time Warping) method.

[0074] For each trial, the marker detection result was recorded as the pass score. The sequence of passing points in a normal scenario without communication delays was considered as the reference point for each scheme and benchmark. For each trial, the DTW distance between the recorded passing point and the reference point was calculated.

[0075] (result) Figure 13 shows the robot's path in the benchmark results for (a) the normal scenario and (b) the abnormal scenario. The robot performed autonomous driving according to two waypoints in every trial. In both scenarios, the route taken became more curved as the communication delay increased. Figure 14 shows the robot's path in the results of this system for (a) the normal scenario and (b) the abnormal scenario. The robot performed autonomous driving according to the same four waypoints in every trial. In both scenarios, the route taken did not become more curved, even as the communication delay increased. This is thought to be because the posture estimated by the AI ​​was used in every trial, taking communication delay into account.

[0076] The DTW method was used to evaluate the dissimilarity of the robot's route. Figure 15 shows the DTW distance between the robot's reference point and the waypoint for each trial, representing the results for the benchmark and the present system in normal and abnormal scenarios. In the benchmark results, the DTW distance increased as the communication delay increased in both scenarios. The maximum distance in the normal scenario for the present system was approximately 4.46 when the communication delay was set to 1.0 s. The maximum distance in the abnormal scenario for the present system was approximately 3.25 when the communication delay was set to 0.9 s. In this trial, the DTW distance in the abnormal scenario was almost the same as that in the normal scenario. The average absolute difference between the DTW distances in the normal and abnormal scenarios was approximately 2.69. In the results of this example, the DTW distances in all trials were less than 5.0 in both scenarios. The average absolute difference between the DTW distances in the normal and abnormal scenarios was approximately 0.71. These results demonstrate that autonomous driving was assisted regardless of communication delays, even when the robot's onboard sensors were disabled.

[0077] <Example 2> [Experiment and analysis conditions (equipment, location, parameters)] An evaluation experiment was conducted to verify the effectiveness of the proposed system. The experiment involves one micro-mobility vehicle driving autonomously in a simulator environment. · Micro-mobility autonomously drives on a pre-determined course. ·Micro-mobility autonomous driving processing is handled by worker nodes. The experimental system consisted of one master node, one worker node, and one hub. One desktop PC was used as the master node. The desktop PC was configured with an RTX 3060Ti GPU and an Intel Core i7 12700K CPU. In the experimental system, a simulator environment for running multiple micro-mobility vehicles was constructed on a desktop PC. Four Jetson Xavier NX were used as worker nodes. One LSW6-GT-5EPL / NBK was used as the hub.

[0078] [Explanation of the results in Figure 16] This shows the results when one micro-mobility vehicle was driven autonomously using one worker node on a simulator. Figure 16 shows the driving trajectory when one micro-mobility vehicle was driven autonomously using one worker node on a simulator. Figure 17 shows each micro-mobility route so that they can be distinguished. R1 refers to the micro-mobility vehicle numbered 1. The micro-mobility vehicle drove autonomously from a predetermined start position to a goal position. The time it took to drive autonomously from the predetermined start position to the goal position was 158.37 seconds. The average execution time for self-localization was 0.20 seconds, with a standard deviation of 0.00.

[0079] Example 3 [Experiment and analysis conditions (equipment, location, parameters)] An evaluation experiment was conducted to verify the effectiveness of the proposed system. The experiment involves 10 micro-mobility vehicles driving autonomously in a simulator environment. · Micro-mobility autonomously drives on a pre-determined course. ·Micro-mobility autonomous driving processing is handled by worker nodes. The experimental system consisted of one master node, two worker nodes, and one hub. One desktop PC was used as the master node. The desktop PC was configured with an RTX 3060Ti GPU and an Intel Core i7 12700K CPU. In the experimental system, a simulator environment for running multiple micro-mobility vehicles was constructed on a desktop PC. Four Jetson Xavier NX were used as worker nodes. One LSW6-GT-5EPL / NBK was used as the hub.

[0080] [Explanation of the results in Figure 17] This shows the results when 10 micro-mobilities were driven autonomously using one worker node on a simulator. Figure 17 shows the driving trajectories when 10 micro-mobilities were driven autonomously using one worker node on a simulator. In Figure 17, symbols are used to distinguish between micro-mobility routes. R1-10 refers to micro-mobilities numbered 1 to 10. None of the micro-mobilities were able to drive autonomously from the predetermined start position to the goal position. The average execution time for localization was 0.62 seconds, with a standard deviation of 0.16. It is believed that processing 10 micro-mobilities autonomously using one worker node increased the processing load, which in turn increased the execution time for localization.

[0081] [Practical benefits to society from the results obtained] It became clear that if a single edge computer were to process the autonomous driving of many vehicles, the load on that single edge computer would be high, causing processing delays and making it impossible to drive the vehicles autonomously safely.

[0082] Example 4 [Experiment and analysis conditions (equipment, location, parameters)] An evaluation experiment was conducted to verify the effectiveness of the proposed system. The experiment involves 10 micro-mobility vehicles driving autonomously in a simulator environment. · Micro-mobility autonomously drives on a pre-determined course. ·Micro-mobility autonomous driving processing is handled by worker nodes. The experimental system consisted of one master node, two worker nodes, and one hub. One desktop PC was used as the master node. The desktop PC was configured with an RTX 3060Ti GPU and an Intel Core i7 12700K CPU. In the experimental system, a simulator environment for running multiple micro-mobility vehicles was constructed on a desktop PC. Four Jetson Xavier NX were used as worker nodes. One LSW6-GT-5EPL / NBK was used as the hub.

[0083] [Explanation of the results in Figure 18] The results are shown below for 10 micro-mobility vehicles running autonomously on a simulator using two worker nodes. The following three load balancing methods were used. The first is a static load balancing method that uses one system statistic. Here, CPU usage is used as the system statistic. The second is a dynamic load balancing method that uses CPU usage. Here, CPU usage is used as the system statistic. The third is a dynamic load balancing method that uses multiple system statistics. Here, CPU usage, CPU queue length, memory usage, and network traffic are used as system statistics.

[0084] Figure 18(a) shows the execution time of localization using three load balancing methods in a box plot. In the figure, static_cpu_x% is the result when a static load balancing method using CPU utilization is used. x indicates the threshold value of CPU utilization that is the standard for load balancing. x was set in increments of 5 between 70 and 95. In the figure, dynamic_cpu is the result when a dynamic load balancing method using one system statistic value is used. In the figure, dynamic_multiple is the result when a load balancing method using multiple system statistic values ​​is used.

[0085] The following can be read from Figure 18(a). The execution time of localization when using the static load balancing method using CPU utilization was as follows: When the CPU utilization threshold was set to 70, 75...95%, the average (standard deviation) execution time of localization was 0.27s (0.0), 0.27s (0.0), 0.27s (0.0), 0.28s (0.0), 0.29s (0.0), and 0.28s (0.0), respectively. The execution time of localization using a dynamic load balancing method using one system statistic was as follows: The average execution time of localization was 0.27 seconds with a standard deviation of 0.01 seconds. The execution time for localization when using a load balancing method that uses multiple system statistics was as follows: The average execution time for localization was 0.29 seconds, with a standard deviation of 0.00 seconds. This shows the comparison results of 10 micromobility vehicles running autonomously on a simulator when using one worker node without background load and when using two worker nodes with three load balancing methods. The results of a comparison with a static load balancing method using CPU utilization are shown below. At all CPU utilization threshold settings, the execution time for self-localization was reduced compared to when 10 micro-mobilities were running on a single worker node, and all micro-mobilities were able to travel autonomously from the predetermined starting position to the goal position. The results of a comparison with a dynamic load balancing method using a single system statistic are shown. The execution time for self-localization was reduced compared to when 10 micro-mobilities were run on a single worker node, and all micro-mobilities were able to travel autonomously from a predetermined starting position to a goal position. We present the results of a comparison with a dynamic load balancing method using multiple system statistics. The execution time for self-localization was reduced compared to when 10 micro-mobilities were run on a single worker node, and all micro-mobilities were able to travel autonomously from a predetermined starting position to a goal position. The comparison results are discussed below. This is thought to be because the load balancing method was used to process autonomous driving on two worker nodes, reducing the load on one worker node.

[0086] All load balancing methods enabled all micro-mobilities to autonomously drive along the determined route. Figure 18(b) shows the driving trajectory when using the load balancing method that uses multiple system statistics.

[0087] The following can be read from Figure 18(b). Using a dynamic load balancing method that uses multiple system statistics, all micro-mobilities were able to autonomously navigate from a predetermined starting position to a goal position. In the dynamic load balancing method using multiple system statistics, each micro-mobility was assigned 6 nodes to worker node1 and 4 nodes to worker node2. Using dynamic load balancing techniques that use multiple system statistics, robot1, robot2, robot3, robot4, robot5, and robot6 were assigned to worker node1, and robot7, robot8, robot9, and robot10 were assigned to worker node2. Using the dynamic load balancing method with multiple system statistics, the time it took for robots 1, 2...10 to travel autonomously from a predetermined start position to a goal position was 227.43s, 229.28s, 234.86s, 228.31s, 228.46s, 224.58s, 227.34s, 225.28s, 231.69s, and 225.55s, respectively.

[0088] [Practical benefits to society from the results obtained] When outsourcing autonomous driving processing to the edge, the load of the autonomous driving processing can be distributed using multiple edge computers, making it possible to operate many autonomous vehicles simultaneously.

[0089] <Example 5> [Experiment and analysis conditions (equipment, location, parameters)] An evaluation experiment was conducted to verify the effectiveness of the proposed system. The experiment involves 10 micro-mobility vehicles driving autonomously in real and simulated environments. Nine micro-mobilities will be driven autonomously in a simulator environment, and one actual micro-mobility will be driven autonomously in a real environment. · Micro-mobility autonomously drives on a pre-determined course. ·Micro-mobility autonomous driving processing is performed on worker nodes. The experimental system consisted of one master node, two worker nodes, one router, and one actual micro-mobility node. One laptop was used as the master node. The laptop PC was configured with an RTX 3050 Laptop GPU as the GPU and an Intel Core i7-1280P as the CPU. In the experimental system, a simulator environment was created on a laptop PC to run multiple micro-mobility vehicles. Two Jetson Xavier NX were used as worker nodes. One tp-link Archer AX73 was used as a router. One Turtlebot 3 burger was used as the actual micro-mobility.

[0090] [Explanation of the results in Figure 19] Figure 19 shows the results when one micro-mobility vehicle was driven autonomously in a real environment and nine micro-mobility vehicles were driven autonomously in a simulator environment simultaneously using one worker node with a background load of 50% (CPU and memory). Figure 19 shows the driving trajectory of each micro-mobility vehicle. Figure 19(b) illustrates each driving trajectory so that the micro-mobility routes can be distinguished. Figure 19(a) shows the driving trajectory of one micro-mobility vehicle driven autonomously in a real environment. Figure 19(b) shows the driving trajectory of nine micro-mobility vehicles driven autonomously in a simulator environment.

[0091] The following can be read from Figure 19. When 10 micro-mobilities were driven autonomously by one worker node, none of the micro-mobilities were able to drive autonomously from the predetermined starting position to the goal position. The average execution time for localization was 1.24 seconds with a standard deviation of 0.61.

[0092] Example 6 [Experiment and analysis conditions (equipment, location, parameters)] An evaluation experiment was conducted to verify the effectiveness of the proposed system. The experiment involves 10 micro-mobility vehicles driving autonomously in real and simulated environments. - Nine micro-mobilities will be driven autonomously in a simulator environment, and one actual micro-mobility will be driven autonomously in a real environment. · Micro-mobility autonomously drives on a pre-determined course. ·Micro-mobility autonomous driving processing is performed on worker nodes. The experimental system consisted of one master node, two worker nodes, one router, and one actual micro-mobility node. One laptop was used as the master node. The laptop PC was configured with an RTX 3050 Laptop GPU and an Intel Core i7-1280P CPU. In the experimental system, a simulator environment was created on a laptop PC to run multiple micro-mobility vehicles. Two Jetson Xavier NX were used as worker nodes. One tp-link Archer AX73 was used as a router. One Turtlebot 3 burger was used as the actual micro-mobility.

[0093] [Explanation of the results in Figure 20] Figure 20 shows the driving trajectories of each micro-mobility vehicle when using a dynamic load balancing method using multiple system statistics. Figure 20(b) shows each driving trajectory so that the micro-mobility routes can be distinguished. Figure 20(a) shows the driving trajectory of one micro-mobility vehicle driving autonomously in a real environment. Figure 20(b) shows the driving trajectories of nine micro-mobility vehicles driving autonomously in a simulator environment.

[0094] The following can be read from Figure 20. All miciro-mobility vehicles were able to autonomously navigate from a predetermined starting position to a goal position. The average execution time for localization was 0.78 seconds with a standard deviation of 0.22.

[0095] The following can be read from Figure 20(a). Using a dynamic load balancing method that uses multiple system statistics, all micro-mobilities were able to autonomously navigate from a predetermined starting position to a goal position. In the dynamic load balancing method using multiple system statistics, micro-mobility was assigned to worker node1. Using the dynamic load balancing method that uses multiple system statistics, the time it took for the robot to travel autonomously from a predetermined starting position to a goal position was 145.33 seconds.

[0096] The following can be read from Figure 20(b). Using a dynamic load balancing method that uses multiple system statistics, all micro-mobilities were able to autonomously navigate from a predetermined starting position to a goal position. In the dynamic load balancing method using multiple system statistics, each micro-mobility was assigned 4 and 5 nodes to worker node 1 and 2, respectively. In the dynamic load balancing method using multiple system statistics, robot1, robot2, robot3, and robot4 were assigned to worker node1, and robot5, robot6, robot7, robot8, and robot9 were assigned to worker node2. Using the dynamic load balancing method with multiple system statistics, the time it took for robots 1, 2...9 to travel autonomously from a predetermined start position to a goal position was 340.88s, 345.19s, 346.90s, 345.11s, 358.11s, 336.52s, 358.65s, 352.30s, and 329.16s, respectively.

[0097] [Practical benefits to society from the results obtained] We were able to verify in a real-world environment that when outsourcing autonomous driving processing to the edge, it is possible to operate many autonomous vehicles simultaneously by distributing the load of the autonomous driving processing using multiple edge computers.

[0098] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the illustrated embodiments. Various modifications and variations can be made to the illustrated embodiments within the same scope as the present invention or within an equivalent scope. [Explanation of symbols]

[0099] 1. Edge computing system 10...Sensor device 10a...first sensor unit 11...Edge computer (server device) 111...Concentration section 112...Map generation unit 113...Digital Twin Generation Department 114...Information storage unit 115...Transmitter 2...Mobile 20...Second sensor unit 21...First receiving unit 22...Second receiving unit 23...In-vehicle computer 231…Mobile object position identification unit 231a...Estimation part 231b...Correction section 232... Obstacle recognition unit 232a...Detection unit 232b...Tracking section 232c…Prediction Section 233...Path generation unit 234...Control unit 235...Selection section 24...Drive mechanism

Claims

1. A mobile object autonomous driving assistance system that includes a sensor device disposed in an environment of a real space, a server device connected to the sensor device in a state capable of communicating with the sensor device, and a mobile object connected to the server device in a state capable of communicating with the sensor device, and that assists the mobile object in autonomous driving in the real space, the sensor device includes a first sensor unit that acquires first image sensor data consisting of a point cloud in real space; the moving object includes a second sensor unit that acquires second image sensor data consisting of a point cloud in real space; The server device or the mobile body A mobile object autonomous driving assistance system characterized by comprising a mobile object position identification unit that identifies the position of the mobile object in real space based on first image sensor data consisting of a point cloud in real space acquired by a first sensor unit of the sensor device and second image sensor data consisting of a point cloud in real space acquired by a second sensor unit of the mobile object.

2. 2. The mobile body autonomous driving assistance system according to claim 1, wherein the mobile body position identification unit includes an estimation unit that estimates the position of the mobile body based on second image sensor data consisting of a point cloud in real space acquired by the second sensor unit of the mobile body, and a correction unit that corrects the position of the mobile body estimated by the estimation unit based on first image sensor data consisting of a point cloud in real space acquired by the first sensor unit of the sensor device.

3. 2. The mobile body autonomous driving assistance system according to claim 1, wherein the server device or the mobile body comprises an obstacle recognition unit that recognizes obstacles in real space based on first image sensor data consisting of a point cloud in real space acquired by the first sensor unit of the sensor device and second image sensor data consisting of a point cloud in real space acquired by the second sensor unit of the mobile body.

4. 4. The mobile body autonomous driving assistance system according to claim 3, wherein the obstacle recognition unit comprises: a detection unit that detects obstacles around the mobile body; a tracking unit that captures a trajectory of the obstacle based on the detection result of the obstacle by the detection unit; and a prediction unit that predicts the position of the obstacle after a predetermined time has elapsed based on the trajectory of the obstacle captured by the tracking unit.

5. 4. The mobile body autonomous driving assistance system according to claim 3, wherein the server device or the mobile body includes a route generation unit that generates a travel route for the mobile body in real space based on the position of the mobile body in real space identified by the mobile body position identification unit and the position of an obstacle in real space identified by the obstacle recognition unit.

6. the server device or the mobile body includes a selection unit that selects image sensor data consisting of a point cloud in real space from either first image sensor data consisting of a point cloud in real space acquired by the first sensor unit of the sensor device or second image sensor data consisting of a point cloud in real space acquired by the second sensor unit of the mobile body, 2. The mobile body autonomous driving assistance system according to claim 1, wherein the moving body position identification unit identifies the position of the moving body in real space based on first image sensor data or second image sensor data consisting of a point cloud in real space selected by the selection unit.

7. 7. The mobile body autonomous driving assistance system of claim 6, wherein the selection unit preferentially selects second image sensor data consisting of a point cloud in real space acquired by the second sensor unit of the mobile body, and if it determines that the second image sensor data consisting of a point cloud in real space is invalid, selects first image sensor data consisting of a point cloud in real space acquired by the first sensor unit of the sensor device.

8. The mobile body autonomous driving assistance system according to claim 5 , wherein the mobile body includes a control unit that controls the traveling of the mobile body based on the traveling route of the mobile body in the real space generated by the route generation unit.

9. The mobile object autonomous driving assistance system according to claim 1 , wherein the server device includes an aggregation unit that aggregates first image sensor data consisting of a point cloud in real space acquired by the first sensor unit.

10. the server device includes a map generation unit that generates a map of the real space based on first image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit; The mobile object autonomous driving assistance system according to claim 9 , wherein the mobile object position specifying unit uses information related to the map of the real space generated by the map generating unit when specifying the position of the mobile object.

11. the server device includes a digital twin generation unit that generates a digital twin corresponding to the real space based on first image sensor data consisting of a point cloud in the real space aggregated by the aggregation unit; The mobile body autonomous driving assistance system according to claim 9, wherein the mobile body position identification unit uses information about the digital twin corresponding to the real space generated by the digital twin generation unit when identifying the position of the mobile body.

12. The mobile autonomous driving assistance system of claim 11, wherein the digital twin generation unit detects the moving body in real space using a learning model that has been machine-learned in advance to detect the moving body based on first image sensor data consisting of a point cloud in real space.

Images