Information processing systems and programs

Abstract

This enables pre-verification and high-precision verification of the physical compatibility between a user's specific mobile device and its environment. [Solution] Based on user operations, the information processing system acquires first 3D model data generated by measuring the 3D shape of a mobile object used by the user, acquires second 3D model data corresponding to the environment to which the mobile object moves, positions the mobile object model defined by the first 3D model data relative to the environment model defined by the second 3D model data in a virtual 3D space while maintaining dimensional consistency with the real space, moves the mobile object model in the virtual 3D space based on user operations, determines the mobility of the mobile object model, including whether it can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile object model and the environment model, and presents information indicating the determination result.

Description

Technical Field

[0001] The present disclosure relates to an information processing system and a program.

Background Art

[0002] In recent years, technologies (such as digital twin technology) that capture the physical environment and objects in the real space as three-dimensional (3D) data and perform reproduction or simulation in a virtual space have been developing. In particular, with the spread of AR (augmented reality) technology and VR (virtual reality) technology, consumers can use consumer devices such as smartphones and tablet terminals equipped with LiDAR (Light Detection and Ranging) scanners and ToF (Time of Flight) cameras without using dedicated equipment. It is possible to scan the real space and superimpose and display virtual objects (3D models) on the video, or to operate them in the virtual space.

[0003] For example, in the field of e-commerce (electronic commerce), "AR try-on" and "AR placement" systems that can place 3D models of products such as furniture and home appliances in the camera video of the user's room and confirm the size, color, and harmony with the interior have been put into practical use. In such a system, by placing a "ready-made product" 3D model prepared in advance by a manufacturer or a sales store on the video of the real space being photographed by the user, it supports consideration and simulation before purchase.

[0004] On the other hand, from the perspective of social infrastructure and welfare, the provision of barrier-free information is considered important. In particular, wheelchair users have a need to check the accessibility of facilities (ease of movement, width of passageways, presence of steps, size and layout of barrier-free toilets, etc.) in advance when using unfamiliar facilities or stores. Traditionally, this information has generally been provided as text information such as facility photos and equipment lists (e.g., presence of handrails, ramps), or as 2D map information. While panoramic image services like Street View exist, these are limited to visual confirmation and have limitations in understanding physical dimensions and three-dimensional structures. Furthermore, at certain event venues and large facilities (such as racetracks and stadiums), the availability of spectator seating, parking, and toilets can be a barrier to movement, and there has been a need for detailed means of checking this in advance. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] U.S. Patent No. 12346948 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Conventional technologies, such as those described in Patent Document 1, primarily focus on static furniture placement simulations and are insufficient for dynamic simulations that verify whether a "mobile object" like a wheelchair can physically pass through a specific environment. In particular, it is difficult to perform highly accurate compatibility verification that reflects the shape and movement characteristics of individually customized mobile objects, as well as three-dimensional elements such as the inclination of the floor surface, and thus has not been able to eliminate the physical and psychological barriers to user movement.

[0007] Therefore, this disclosure aims to provide an information processing system and program that enables the prior and highly accurate verification of the physical compatibility between a user's specific mobile device and its environment. [Means for solving the problem]

[0008] An information processing system according to one aspect of the present disclosure includes: a first acquisition unit that acquires first three-dimensional model data generated by measuring the three-dimensional shape of a mobile body used by a user based on the user's operation; a second acquisition unit that acquires second three-dimensional model data corresponding to the environment to which the mobile body is to move; a synthesis processing unit that arranges a mobile body model defined by the first three-dimensional model data in a virtual three-dimensional space with respect to an environment model defined by the second three-dimensional model data, while maintaining the consistency of dimensions in their respective real spaces; a movement control unit that moves the mobile body model in the virtual three-dimensional space based on the user's operation; a determination unit that determines the possibility of movement, including whether the mobile body model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile body model and the environment model; and a presentation processing unit that performs processing to present information indicating the determination result by the determination unit.

[0009] Programs relating to other aspects of this disclosure cause at least one computer to operate as the information processing system described above. [Effects of the Invention]

[0010] According to one aspect of this disclosure, an information processing system and program can be provided that enables prior and highly accurate verification of the physical compatibility between a user-specific mobile object and its environment. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing an example of the overall configuration of the information processing system according to the embodiment. [Figure 2] This figure shows an example of the functional block configuration of an information processing system according to an embodiment. [Figure 3] This is a flowchart showing the processing flow in the information processing system according to the embodiment. [Figure 4] This block diagram shows an example of the configuration of a user terminal according to the embodiment. [Figure 5] This block diagram shows an example of the server configuration according to the embodiment. [Figure 6] This figure shows an example of the database configuration of a server according to the embodiment, where (a) shows the data structure of the user information DB and (b) shows the data structure of the environment information DB. [Figure 7] This is a conceptual diagram showing the general operation of the information processing system according to the embodiment. [Figure 8] This is an explanatory diagram showing the procedure for scanning environmental data in an embodiment. [Figure 9] This figure shows an example of a simulation execution screen in the embodiment. [Figure 10] This is an explanatory diagram of the collision detection method in the embodiment, where (a) shows detection using a box collider and (b) shows detection using a mesh collider. [Figure 11] This figure shows an overview of the overall system integration and data flow in the example. [Figure 12] This diagram shows the processing flow of the application in the example and the branching based on the presence or absence of data. [Figure 13] This figure shows the flow of internal processing blocks and datasets during simulation execution in the example. [Figure 14] This figure shows a list of the main functions of the 3D wheelchair simulation according to the embodiment. [Figure 15] This figure shows the main parameter configuration of the wheelchair model (mobility model) in the embodiment. [Figure 16] This figure shows the main parameter configuration of the structural model (environmental model) in the example. [Figure 17] This diagram shows the management flow for registering, updating, and deleting 3D wheelchair data in the example. [Figure 18] This diagram shows the management flow for registering, updating, and deleting 3D facility data in the example. [Figure 19] This is a user interface transition diagram for registering and linking 3D wheelchair data in the embodiment. [Figure 20] It is a transition diagram of a user interface regarding registration and linkage of 3D facility data in an embodiment. [Figure 21] It is a user interface configuration diagram of the main screen during simulation execution, at the time of contact detection, and the setting screen in an embodiment. [Figure 22] It is a user interface configuration diagram regarding report display and SNS linkage after simulation in an embodiment. [Figure 23] It is a diagram showing an improvement plan of a user interface for viewpoint switching (first-person viewpoint · third-person viewpoint) according to a modification example. [Figure 24] It is a diagram showing an improvement plan of a controller (input interface) for a movement operation according to a modification example.

Mode for Carrying Out the Invention

[0012] Before explaining the details of the information processing system according to the embodiment, the specific problem background and the direction of solution approached by this system will be explained.

[0013] In a conventional AR placement system, a static confirmation of "can it be placed there" was possible, but in the case of a moving body, the possibility of continuous movement of "can it pass through there to reach the destination" is questioned. For example, moving bodies such as wheelchairs are often customized according to the user's physique, disability characteristics, or lifestyle, and there was a problem that it was difficult to judge the actual passability only by the catalog specifications (overall width, overall length, overall height) of off-the-shelf products.

[0014] For example, even with wheelchairs of the same model number, individual factors such as a backpack or medical equipment like an oxygen tank attached to the back, the way the user positions their feet and the angle of the footrest, and the increase in width due to the camber angle (wheel angle) to improve maneuverability can affect whether or not they can pass through a passage by a few centimeters, sometimes even a few millimeters. Furthermore, in the case of electric wheelchairs and mobility scooters, which have a large turning radius and are not very maneuverable, even if the passage is wide enough, there are cases where they cannot pass through crank-shaped passages or right-angle turns. In addition, three-dimensional terrain elements such as the slope of the floor (angle of the ramp) and the unevenness of the road surface are also important factors that directly affect the climbing ability and off-road capability of the mobility device, but there has been no widespread means of easily and accurately verifying these.

[0015] Furthermore, conventional barrier-free information primarily consists of two-dimensional information such as photographs, and intuitively grasping depth and three-dimensional structure from images relies on spatial awareness, making it difficult for many users. Moreover, even with existing 3D scanning technology, these are limited to simple shape data viewing (viewer) functions, and have not provided an integrated system that can verify high-precision "navigability" without actually going to the site, while reflecting the unique physical characteristics of individual users and the physical properties of their objects. As a result, users currently face physical failures such as "I went there but couldn't get in," and psychological barriers that cause them to hesitate to go out for fear of such failures.

[0016] The information processing system according to this embodiment is configured to solve these problems.

[0017] (1) Information processing system The information processing system (also simply referred to as the "system") according to the embodiment will be described below with reference to the drawings. In the drawings, identical or similar parts are denoted by the same or similar reference numerals.

[0018] (1.1) Example of overall structure Referring to Figure 1, an example of the overall configuration of the information processing system 10 according to this embodiment will be described in detail.

[0019] The information processing system 10 according to this embodiment includes a server 100 that is connected to a network 15 for communication and a plurality of user terminals 200 (200a, 200b, ...). This system dynamically integrates 3D data of real-world environments (commercial facilities, public transportation, roads, indoor spaces, seating areas and parking lots at event venues, etc.) and 3D data of mobile devices used individually by users (wheelchairs, strollers, mobility scooters, personal mobility devices, transport robots, etc.) on a cloud-based virtual space, and provides a platform for simulating whether or not the mobile devices can physically move within that environment.

[0020] Network 15 is an information transmission path and is a wide-area information communication network composed of, for example, the Internet, LAN (Local Area Network), dedicated lines, telephone lines, intranets, mobile communication networks (4G, 5G, LTE, etc.), Bluetooth®, Wi-Fi®, other communication lines, or combinations thereof. Network 15 may be wired or wireless and enables bidirectional transmission and reception of various data (large-capacity 3D model data, control signals, judgment results, metadata, etc.) between Server 100 and user terminals 200, or between user terminals 200 themselves. In particular, in this embodiment, rich 3D data such as point cloud data and high-resolution textured mesh data are frequently handled, so low-latency and high-bandwidth communication infrastructure such as 5G (fifth-generation mobile communication system) and high-speed optical lines are preferably used. However, use in narrow-bandwidth environments is also envisioned by using data compression technology and streaming technology.

[0021] Server 100 is a computer device (or a distributed collection thereof) that serves as the core of the information processing system 10 as both a backend and a data center. Server 100 may be implemented, for example, as a virtual server instance on a public cloud, or as an on-premises physical server within its own data center. Server 100 has the function of storing and managing environmental data (3D models of facilities, POI information, area map information, etc.) and user data (3D models of mobile objects, user profiles, etc.) uploaded from user terminals 200 in an appropriate database structure.

[0022] Furthermore, the server 100 may not only have a simple storage function, but also a CDN (Content Delivery Network) function to efficiently distribute the stored data in response to requests (location information and search queries) from the user terminal 200, a secure user authentication function, a billing management function as needed, and a data analysis function for usage trends. In addition, to reduce the processing load on the user terminal 200, it is also possible to configure the server 100 (cloud side) to perform some or all of the 3D data polygon reduction (lightening), format conversion, synthesis processing, or collision detection processing based on physics calculations (cloud rendering or cloud physics calculation). This makes it possible for users with entry-level smartphones with low processing power to enjoy the benefits of high-precision simulation.

[0023] User terminals 200a, 200b, ... (hereinafter collectively referred to as "User Terminal 200") are front-end information processing terminals used by each user (users of mobile devices, caregivers, facility managers, volunteers, etc.). User terminals 200 consist of, for example, smartphones, tablet devices, notebook PCs, HMDs (Head Mounted Displays), smart glasses, or wearable devices. In particular, in this embodiment, high-end smartphones and tablet devices equipped with LiDAR (Light Detection and Ranging) scanners or ToF depth cameras, which have become increasingly popular in recent years, are preferably used as devices that can complete both 3D data acquisition (scanning) and simulation execution (rendering and operation) in a single unit. This makes it possible for users to view simulation results while checking them on-site, or to acquire and check data on the spot.

[0024] The user terminal 200 can play two main complementary roles within the system. The first role is that of an active data "contributor." The user uses their own user terminal 200 to scan the mobile device they use on a daily basis (for example, themselves in their wheelchair with their usual belongings loaded) and generates a personalized 3D model (first 3D model data). The user also scans facilities and places they visit while out and about (accessible toilets, elevator lobbies, store entrances, ramped walkways, racetrack spectator stands, etc.) and generates a 3D model of that environment (second 3D model data), which is then linked with location information such as GPS and uploaded (posted) to the server 100. As a result, the entire system autonomously builds and updates a user-participatory (crowdsourced) and grassroots "3D barrier-free map."

[0025] The second role is a passive one as a "user" of the simulation. The user uses the user terminal 200 to select and download data for their intended destination or places of interest from the vast amount of environmental data stored on the server 100. Then, they place a 3D model of their own moving object as a digital twin within the virtual space of that environmental data, and operate the moving object on the screen of the user terminal 200 (virtual test drive) to simulate whether they can pass through obstacles, turn around, overcome steps, etc.

[0026] One of the key features of this information processing system 10 is that, rather than being a system where only government agencies or specific businesses unilaterally provide data, it forms a "mutual aid ecosystem" where an unspecified number of parties (users) contribute and share data with each other. For example, suppose a wheelchair user (User A) scans and uploads a model of a toilet in a restaurant near their home. Another wheelchair user (User B), living in a remote location, can download the 3D model of the toilet provided by User A when planning a trip and use their own wheelchair model (which differs from User A's in size, shape, and maneuverability) to verify whether they can enter that toilet.

[0027] Thus, the information processing system 10 according to this embodiment dynamically integrates 3D data of different entities (an individual's mobile body and a public environment) acquired at different temporal and spatial timings in a virtual space, enabling verification of physical interactions (contact detection, interference checks, etc.). This realizes a verification of suitability—specific, quantitative, and individualized suitability—that could not be determined from conventional photographs, 2D drawings, or subjective reviews from others (qualitative information such as "it was spacious" or "it was cramped"), such as "can I reliably use that place with my mobile body?"

[0028] Furthermore, this system can be a useful platform not only for wheelchair users, but also for families with young children using large strollers, delivery companies transporting large items, and businesses considering the introduction of autonomous robots to move around within facilities—in short, for any entity with physical mobility constraints or spatial requirements. Moreover, it has high social significance as it contributes to urban development and the development and evaluation of social infrastructure, such as verifying evacuation routes during disasters, verifying universal design during the architectural design phase, and simulating before renovation work.

[0029] (1.2) Functional Block Configuration and Operation Overview Referring to Figures 2 and 3, the functional block configuration and operation overview of the information processing system 10 according to this embodiment will be described below. The functions and operations corresponding to each embodiment will be described in detail along with specific processing content.

[0030] Each functional block shown in Figure 2 may be provided on the server 100 side or on the user terminal 200 side. However, the user interface 20 shown in Figure 2 is provided on the user terminal 200 side.

[0031] For example, the data management unit 7 may be provided on the server 100 side, and the first acquisition unit 1, second acquisition unit 2, synthesis processing unit 3, movement control unit 4, determination unit 5, presentation processing unit 6, and search processing unit 8 may be provided on the user terminal 200 side. At least one of the first acquisition unit 1, second acquisition unit 2, synthesis processing unit 3, movement control unit 4, determination unit 5, presentation processing unit 6, and search processing unit 8 may be provided on the server 100 side.

[0032] (1.2.1) First aspect As shown in Figures 2 and 3, the information processing system 10 according to the first embodiment includes: a first acquisition unit 1 that acquires first 3D model data generated by measuring the 3D shape of a mobile body used by the user based on the user's operation (step S1); a second acquisition unit 2 that acquires second 3D model data corresponding to the environment to which the mobile body is to move (step S2); a synthesis processing unit 3 that, in a virtual 3D space, positions the mobile body model defined by the first 3D model data relative to the environment model defined by the second 3D model data while maintaining the consistency of dimensions in their respective real spaces (step S3); a movement control unit 4 that moves the mobile body model in the virtual 3D space based on the user's operation (step S4); a determination unit 5 that determines the possibility of movement, including whether the mobile body model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile body model and the environment model (step S5); and a presentation processing unit 6 that performs processing to present information indicating the determination result by the determination unit 5 (step S6).

[0033] According to the first embodiment, a full-scale 3D model of a specific mobile object used by the user (first 3D model data) and a full-scale 3D model of the target environment (second 3D model data) can be combined and simulated within the same virtual space. This makes it possible to verify the mobility feasibility (passability, contact, turning capability) with high accuracy, reflecting the customization status of each user's mobile object, wear and tear, and the condition of temporary equipment (rain covers, luggage, etc.), compared to using general numerical data such as catalog specifications or off-the-shelf models. Users can check physical accessibility in advance as if they were testing it on-site, without actually going to the location, thus avoiding situations where they "go and find out it doesn't work," and significantly reducing psychological anxiety about travel and the risk of getting stuck.

[0034] The "first 3D model data" refers to data digitized by 3D scanning or other means of a specific mobile object that the user normally uses, and may include mesh data, point cloud data, voxel data, etc. It may also be interpreted as "mobile object model," "my car data," "avatar data," "user object," etc.

[0035] "Second 3D model data" refers to data that has been digitized by 3D scanning or other means of the space of the destination, such as a facility, room, or corridor, and may be interpreted as "environmental model," "facility data," "world data," or "spatial map."

[0036] The action of "placing while maintaining dimensional consistency" means, for example, referring to the scale information (scale factor) of both the moving object model and the environment model and placing them in a unified coordinate system that matches the physical units of real space (meters, centimeters, etc.). This can also be rephrased as "performing scale adjustment," "combining at real scale," or "normalizing physical size."

[0037] The action of "determining the possibility of movement" means, for example, calculating the geometric overlap and proximity of objects in 3D space and evaluating whether movement is possible or not, and can be rephrased as "performing collision detection," "performing interference checks," or "performing physical simulations."

[0038] The system in this embodiment is primarily intended to be implemented as an application running on a user terminal 200 (such as a smartphone or tablet), but a hybrid configuration is also possible in which some of the processing load is handled by a server 100. In this case, for example, complex physics calculations and high-resolution rendering processes may be performed on the server side, and the results may be streamed.

[0039] The first acquisition unit 1 imports or acquires 3D data of the moving object itself, scanned by the user using the LiDAR scanner or photogrammetry function of their device. In this process, not only the shape (geometry) but also texture information (surface color, pattern, texture) is acquired, enabling a visually realistic and immersive simulation. Furthermore, it may have a function to automatically measure the size during scanning and add the dimensions of the moving object (overall width, overall length, overall height) as metadata.

[0040] The second acquisition unit 2 acquires 3D data of a specific location where the user wants to perform a simulation (for example, a private room in a restaurant they are about to reserve, an accessible restroom at a train station, or a spectator area at a race track). This data may be data that the user has scanned and saved in the past, or it may be downloaded from publicly available data on server 100 (cloud database). A function that automatically presents available 3D data around the current location based on GPS information is also conceivable.

[0041] A technically important aspect of the synthesis processing unit 3 is the integration of two independent data sets (moving object and environment) that may have been acquired at different times, on different devices, and in different environments, while strictly matching their physical scale (size). For example, standard 3D formats such as GLB (glTF) and USDZ contain metadata (unit system) related to actual dimensions, and this is used to construct a virtual space with a unified standard such as "1 unit = 1 meter". For example, the ground contact reference point (tire contact point) of the moving object model may be matched with the floor surface detected by the plane recognition function of the real space provided by the AR framework, and a scaling process may be performed to match 1 unit distance in the virtual space to 1 meter in the real space by referring to the unit scale information contained in the 3D model file. It is also useful to have a function that automatically adjusts the origin position and orientation of the scan data so that the moving object is placed in the appropriate position (for example, just before the entrance) when the simulation starts. This prevents misidentification such as "being able to pass through on the screen when it is actually impossible (false negative)" or "being judged as unable to pass through when it is possible (false positive)" due to mismatch in scale.

[0042] The movement control unit 4 moves the moving object model within the environment model in response to input from a virtual joystick or touch controls on the screen, or from a physical controller. At this time, the moving object can not only move in parallel (slide), but also perform rotational movements such as turning in place or moving in curves. Furthermore, physical properties such as floors and walls (collision detection, friction coefficient) are set, and physical simulations are performed such as preventing further movement if there is an obstacle, or accelerating or decelerating on slopes.

[0043] The determination unit 5 calculates in real time whether the polygon meshes and bounding boxes of the moving object model and the environment model intersect (contact) in the virtual space. This determination is performed on a frame-by-frame basis and is constantly monitored during movement. It can also determine the slope of the floor surface (slope angle) and the height of any steps, and compare this with the capabilities of the moving object (climbing ability, step-climbing ability). This allows for the detection of situations such as "the object is small enough to pass through, but the step is too high to proceed."

[0044] The presentation processing unit 6 provides the user with easy-to-understand feedback on the judgment result. Specifically, it intuitively conveys the situation by combining visual means (lighting up the contact point red on the screen, displaying an NG mark, displaying the tilt angle with a heatmap-like color), auditory means (warning sounds, collision sounds, voice guidance), and tactile means (warnings through terminal vibration patterns).

[0045] (1.2.2) Second aspect In a second embodiment, the mobile body is a mobile body on which a user can ride and / or which can carry luggage, and the first acquisition unit 1 can acquire first three-dimensional model data based on three-dimensional data scanned with the mobile body with a user on board and / or luggage loaded.

[0046] According to the second embodiment, simulation becomes possible not only of the shape of the mobile device (hardware) alone, but also of the state in which a user is actually riding in it and the state in which luggage is routinely loaded (the combined "effective occupied volume" or "bounding volume"). In the case of wheelchairs, it is common for the user's toes to extend in front of the footrest, their knees to be spread apart, a large backpack hanging on the back to protrude, or for medical equipment such as suction devices or ventilators to be mounted. This configuration makes it possible to make precise contact judgments including these protruding parts, enabling safety checks that are more in line with real-world usage situations. This makes it possible to prevent accidents such as "the wheelchair itself can pass through, but the user's knees hit the door frame" or "the luggage on the back hits the wall, preventing turning."

[0047] Furthermore, "a mobile device on which a user can ride and / or carry luggage" refers to, for example, a device for transporting people or a device for carrying goods, and may be interpreted as "personal mobility," "transportation equipment," "mobility support equipment," etc.

[0048] "Boarding state" refers to, for example, the state in which a user is actually sitting or standing, and may be reinterpreted as "boarding state," "state of use," "seated posture," etc.

[0049] Typical examples of mobility devices include wheelchairs (self-propelled, assisted, electric, sports type, etc.), but are not limited to these. Other possible examples include strollers (Type A, Type B, twin buggies, etc.), mobility scooters (handlebar-type electric wheelchairs), electric kick scooters, stretchers, walkers, or even shopping carts, trolleys, and autonomous delivery robots.

[0050] In particular, for wheelchair users, seating posture and foot placement vary depending on the characteristics of their disability. Furthermore, they may carry medical equipment such as oxygen tanks or IV poles, which function as part of the mobile device. These "protruding" parts are irregular elements not included in the manufacturer's CAD data or catalog specifications. In this embodiment, a unique "My Mobile Device Model" is created by scanning the entire device with the user actually seated and all necessary equipment mounted, perfectly reflecting its state at that specific moment.

[0051] Specifically, with the cooperation of caregivers or family members, the user sits in their wheelchair in their usual posture while a 360-degree scan is performed. This allows for detailed verification of things like "whether their knees will hit the wall," "whether luggage on their back will get caught on doorknobs," and "whether the headroom is low (whether their head will hit the elevator control panel)." This is effective in identifying dynamic interference risks that cannot be determined by static dimensional measurements alone. Furthermore, if clothing changes with the season (e.g., thicker clothing in winter) or the amount of luggage changes, it is possible to create multiple "My Mobile Models" and use them according to the situation.

[0052] (1.2.3) Third aspect In a third embodiment, the first acquisition unit 1 and the second acquisition unit 2 may acquire the first 3D model data and the second 3D model data, respectively, based on data generated by an external 3D scanning application and output in a general-purpose 3D file format.

[0053] According to the third aspect, there is no need to independently develop and implement a scanning function dedicated to this system, and the ecosystem of high-performance and high-precision 3D scanning applications already widely available on the market can be fully utilized. This allows users to easily create high-quality 3D data using tools they are already familiar with, and the system can flexibly adapt to a variety of scanning devices and the latest scanning technologies. Furthermore, by adopting a standard general-purpose format, data compatibility and reusability are enhanced, and integration with other platforms and tools becomes easier.

[0054] Furthermore, "external 3D scanning applications" refer to, for example, third-party software that operates independently of this system and has photogrammetry or LiDAR scanning capabilities, and may be interpreted as "general-purpose scanning apps," "3D capture tools," "modeling software," etc.

[0055] "General-purpose 3D file format" means, for example, a standard format that can be read and written by many 3D software programs and web browsers, and may be interpreted as "GLB (glTF) format," "USDZ format," "OBJ format," "FBX format," "PLY format," etc.

[0056] Recent smartphones come standard with advanced depth measurement capabilities such as LiDAR sensors, enabling surprisingly accurate 3D scanning with free or inexpensive apps. This system does not incorporate its own scanning function, but rather is designed to import data generated by these external apps (file linking function). This is done seamlessly through the OS's "sharing" function and file system.

[0057] Specifically, users perform scans using their preferred external applications, export the generated data in a format such as GLB (GL Transmission Format Binary), and save it to their device's storage. GLB is a lightweight binary format that combines the shape (mesh), appearance (textures, materials), and scene structure of a 3D model into a single file, making it suitable for use in web and mobile applications. The first and second acquisition units of this system read this file and, as needed, perform data reduction (polygon reduction), coordinate system transformation (Y-up / Z-up adjustment), origin position correction, texture resolution adjustment, etc., before deploying it to the simulation space. It also has a function to analyze embedded position and scale information as metadata and perform automatic corrections. This enables an open and highly scalable system that can always incorporate the latest scan quality in line with technological advancements, without being locked into specific hardware or proprietary formats.

[0058] (1.2.4) Fourth aspect In the fourth embodiment, the second acquisition unit 2 may acquire, based on user operation, the second 3D model data corresponding to the environment to which the moving object is moving, from among a plurality of environmental data, each containing the second 3D model data, which is stored in a database (environmental information DB120) associated with geographical location information.

[0059] According to the fourth aspect, by using environmental data (3D models of facilities) linked to location information on a map, for example, users can intuitively select the location they want to simulate via a map application or similar. This makes it possible to pinpoint and retrieve data not only for the current location but also for remote locations that the user plans to visit on trips or business trips, allowing for pre-trip rehearsals. This makes it easier to check accessibility information during the travel planning stage, significantly reducing the risk of travel failures such as "I went there but couldn't get in." Furthermore, by linking with indoor floor information (floor maps) and facility area map functions, it becomes possible to obtain accurate location data even in indoor facilities where GPS signals are difficult to receive or in large areas (such as racetracks or parks).

[0060] Furthermore, "geographic location information" refers to relative location information such as floor number, beacon ID, and Wi-Fi fingerprint, in addition to absolute coordinate information such as latitude, longitude, and altitude, and may be interpreted as "GPS data," "location coordinates," "geotag," or "indoor positioning information."

[0061] "Environmental data" refers to a dataset in which attribute information such as name, address, category, business hours, congestion level, and accessibility information is added to 3D model data, and may be interpreted as "spot information," "POI (Point of Interest) data," "facility card," or "area map information."

[0062] The database on server 100 stores a large amount of environmental data. Each data entry is accompanied by latitude and longitude information, making it possible to map it onto a general-purpose map service or a proprietary map platform (such as a facility area map). By tapping a pin (spot) on the map, users can access a 3D model associated with that location (for example, a model of the steps at the entrance of the building, the entrance hall, a model of an accessible toilet inside, or the layout of the seating area).

[0063] The second acquisition unit 2 downloads 3D model data corresponding to the spot ID selected by the user from the server 100 and loads it into memory. In this case, if the data size is large, Level of Detail (LOD) technology may be used to initially display a low-resolution model (for preview) and then acquire high-resolution data when a detailed simulation is required. Furthermore, by temporarily caching (saving) the downloaded data on the terminal, the simulation can be run even in locations with unstable communication environments or offline environments. This links "location" and "spatial shape," enabling more practical and comfortable navigation support. In addition, by providing a UI that seamlessly switches between AR view and map view, users can intuitively find their desired location.

[0064] (1.2.5) Fifth aspect In the fifth embodiment, a server 100 capable of communicating with multiple user terminals 200 via a network 15 includes a data management unit 7 that registers second three-dimensional model data obtained from each user terminal 200 in a database (environmental information DB) in association with location information of the place where the data was measured, and a second acquisition unit 2 may acquire second three-dimensional model data registered by any user from the database.

[0065] According to the fifth aspect, by incorporating so-called CGM (Consumer Generated Media) and crowdsourcing mechanisms, it becomes possible to accumulate and utilize environmental data captured and generated not only by specific administrators or local governments, but also by ordinary users as "shared knowledge (commons)." This improves the comprehensiveness and update frequency of barrier-free information, enabling the creation of a "barrier-free map created by everyone." A mutually supportive ecosystem is built where the actions of one user help many other users, and it is expected that this will accelerate the barrier-free transformation of society as a whole from an information perspective.

[0066] Traditional barrier-free map creation requires on-site surveys by local governments or specialized surveyors, which is extremely costly and time-consuming, often resulting in delayed information updates and many gaps in the information. This embodiment allows users to scan and upload locations they visit using their smartphones. This enables real-time reflection of rapidly changing information, such as temporary event venues and detours during construction, in addition to locations used on a daily basis.

[0067] The data management unit 7 of server 100 receives 3D model data transmitted from user terminals 200 and stores it in a database along with location information, date and time of shooting, poster information, and type of device used (information related to scan accuracy). At this time, the data management unit 7 may perform automatic filtering of inappropriate data (data containing infringing images that violate privacy, data that is contrary to public order and morals, or data with low scan accuracy) and quality control through review and reporting functions by other users in order to ensure data quality. In addition, version control of the data may be implemented to always provide the latest data while also making past data accessible, so that users can check the history of changes in the environment. The data accumulated in this way is made available as an open resource to all users (or authorized users), and anyone can freely use it for simulations.

[0068] (1.2.6) Sixth aspect In the sixth embodiment, the data management unit 7 may manage the first 3D model data as private, accessible only to the user, and the second 3D model data as public, accessible to other users, based on the user's operation.

[0069] According to the sixth aspect, granular access control (privacy management) tailored to the nature of the data becomes possible. Mobile data (first 3D model), which is personal property and may reflect physical characteristics and lifestyles, can be kept strictly confidential, while facility data (second 3D model), which is highly public, can be widely shared. This allows for flexible operation. As a result, users can register and use their own real data in the system with peace of mind, without worrying about the risk of privacy leaks. This is an essential function for gaining user trust in this system, where personal and public data coexist.

[0070] Note that "private settings" means, for example, limiting data access to specific users (such as the user themselves), and may be interpreted as "private mode," "local storage," "secret settings," or "access restrictions."

[0071] "Publication settings" means, for example, opening up access permissions to data to the general public (or a specific group), and can be interpreted as "public mode," "sharing settings," "open data," or "fully public."

[0072] Mobile model data may include personal belongings such as wheelchairs and, in some cases, the user's own image, making it highly private data. Unrestricted public release of this data carries the risk of identifying individuals and inferring their lifestyle patterns. On the other hand, environmental model data concerns public places such as train station restrooms, shop entrances, and park pathways, and its widespread sharing creates social value.

[0073] The data management unit 7 strictly manages access control lists (ACLs) and public flags in the database for each data type or according to user settings. For example, mobile data is encrypted and stored in association with the user ID, and can only be downloaded and decrypted on devices that have passed user authentication. On the other hand, environmental data is placed in the public database in a searchable state. Furthermore, by providing intermediate settings such as "share only with friends," "share for a limited time," and "share only within a specific community," it is possible to promote information sharing within travel groups and clubs. It is also important to allow users to choose to keep data private, even for environmental data, for locations related to privacy, such as around their home.

[0074] (1.2.7) Seventh aspect In the seventh embodiment, if the data management unit 7 obtains multiple different 3D model data for the same location, it may perform a process to integrate the multiple 3D model data into a single 3D model data.

[0075] According to the seventh aspect, when multiple users scan the same location at different times and angles, merging that data can generate a more comprehensive, higher-resolution, and more complete environmental model with fewer blind spots and occlusions. Even if individual scans fail to capture areas such as behind furniture or high places, this information is supplemented by the scan data of other users, significantly improving the reliability of the simulation and the accuracy of spatial reproduction. Furthermore, the elimination of duplicate data is expected to improve storage efficiency.

[0076] The term "integration process" refers to actions such as aligning and combining multiple point cloud or mesh data sets, or selecting the highest quality model, and may be interpreted as "data fusion," "registration," "combination process," or "selection process."

[0077] "The same location" refers to a location where, for example, the location information matches within a predetermined range, or where the location is determined to be the same space through matching of image feature points or shape features.

[0078] For example, if one user focuses their scan on the area near the entrance of a barrier-free toilet, while another user scans the area around the toilet inside the stall in detail, integrating these scans creates a model that can seamlessly simulate the entire movement path from the entrance to the inside of the stall, and then to turning and transferring.

[0079] The data management unit 7 identifies the relative positional relationships of multiple data points based on GPS information, Wi-Fi information, and image feature points (SIFT, ORB, etc.), and performs high-precision positioning of point clouds using algorithms such as ICP (Iterative Closest Point). In addition to identifying approximate positions using GPS, it may also perform self-position estimation with centimeter-level accuracy by matching feature point clouds extracted from camera images with a 3D point cloud map registered in the database in advance (VPS technology), thereby aligning the environmental model with the real space. Then, it performs averaging of overlapping parts, noise reduction, and mesh reconstruction to generate a single high-quality integrated model. Furthermore, if the environment changes over time (e.g., changes in store layout or renovation work), a "time-series merging" mechanism is introduced that prioritizes data with more recent shooting dates and updates the model, ensuring that the latest on-site conditions are always reflected. In addition, the accuracy of the model generated by the integration process can be scored and presented to the user, providing a basis for judging the reliability of the simulation.

[0080] (1.2.8) Eighth aspect In the eighth embodiment, the synthesis processing unit 3 may simultaneously arrange multiple mobile model units corresponding to different users within the same environment model, and the determination unit 5 may determine contact between multiple mobile model units in addition to contact with the environment model.

[0081] According to the eighth aspect, it becomes possible to simulate situations where multiple moving objects exist in the same space (multi-user simulation). This allows for the verification of complex events that cannot be verified by a single simulation, such as whether wheelchairs can pass each other in a narrow corridor, whether multiple wheelchairs or strollers can fit in an elevator simultaneously, or whether congestion or getting stuck will occur in corridors during a disaster evacuation. This is useful in planning group outings, developing evacuation plans for facilities, and pre-verifying event operations.

[0082] "Multiple mobile model" refers to, for example, the mobile model of oneself and a friend, or a virtual other (AI agent), and may be rephrased as "other vehicle model," "NPC (Non-Player Character) mobile model," "companion model," etc.

[0083] "Contact between mobile model entities" refers to, for example, collisions or interference between users, or approaching each other at close range, and may be reinterpreted as "mutual interference," "passing each other detection," or "person-to-person distance confirmation."

[0084] This feature could be implemented as a "virtual tour" where friends in remote locations can preview facilities in a virtual space, similar to a real-time online multiplayer game, or it could be a format where a single user places and controls their own and their friend's moving models.

[0085] For example, when two wheelchair users go out to eat together, they might want to check beforehand whether two wheelchairs can fit into a private room at a restaurant and sit at a table. Using this function, they can place their own and their friend's wheelchair models within a model of the restaurant's environment and verify their movement. The judgment unit 5 detects not only collisions with the environment (walls, pillars) but also collisions between wheelchair A and wheelchair B, and issues a warning if passing is difficult or if there is insufficient space to turn. Furthermore, it can be applied to crowd simulations to identify bottlenecks in evacuation routes during disasters and to formulate optimal evacuation guidance plans. This can contribute not only to individual mobility support but also to improving the safety of society as a whole.

[0086] (1.2.9) The ninth aspect In the ninth embodiment, the movement control unit 4 may set motion characteristic parameters for the moving body model, including the minimum turning radius of the corresponding real-world moving body, the pivot axis based on the wheel arrangement, and / or the maximum inclination angle that can be climbed, and move the moving body model in a virtual three-dimensional space under constraints based on the motion characteristic parameters.

[0087] According to the ninth aspect, it is possible to reproduce a realistic feel of operation and movement trajectory that reflects the actual physical behavior and motion performance (kinematics / dynamics) of the moving object, rather than simply translating the shape. For example, wheelchairs that cannot turn in place, electric carts with a large turning radius, or differences between front-wheel drive and rear-wheel drive may prevent passage through a gap that appears to be passable in terms of shape, even if it is not actually possible to turn through. This configuration allows for accurate detection of the risk of impassability due to such "movement constraints" and enables more realistic judgments. Furthermore, by setting the climbing ability against the slope of the floor as a parameter, it is possible to know in advance which slopes cannot be climbed due to insufficient power.

[0088] Furthermore, "motion characteristic parameters" refer to, for example, physical numerical values ​​that define the movement of a moving object, and may be interpreted as "physical parameters," "behavior setting values," "spec data," "wheelbase / tread information," etc.

[0089] The movement control unit 4 calculates the behavior of the moving object using a physics engine. Depending on the type of moving object (self-propelled wheelchair, assisted wheelchair, electric wheelchair, mobility scooter, omni-wheel equipped machine, etc.), it sets parameters such as the positions of the front and rear wheels, drive wheels, steering wheels, wheelbase, tread, and center of gravity in detail.

[0090] For example, it can simulate the movement of the front casters and the turning behavior caused by the difference in rotation between the left and right tires. It can also take into account the friction coefficient and gradient information of the floor surface in the environmental model, so that if the slope is too steep it will "slip and be unable to climb," it will stop, or if the step is too high it will reproduce the behavior of the casters getting stuck and being unable to overcome it. This can prevent situations such as "the size is OK, but it cannot enter due to insufficient power or poor maneuverability." Furthermore, physical simulations of movement in 3D space, such as the angle of inclination and the height of the step that can be passed over, can be visually confirmed. In addition, these parameters can be customized not only based on catalog values ​​but also on the user's actual measurements and subjective feelings.

[0091] (1.2.10) Tenth aspect In the tenth embodiment, the determination unit 5 may set a bounding box and / or polygon mesh for the moving object model and detect contact by performing intersection determination with the environment model using the bounding box and / or polygon mesh.

[0092] According to the tenth aspect, appropriate collision detection methods can be flexibly selected and combined depending on the processing power of the user terminal and the simulation conditions (whether it is a large or small area). Using bounding boxes reduces the computational load and enables smooth operation, while using polygon meshes (the shapes themselves) allows for precise detection of contact between complex shapes. As a result, even in resource-limited environments such as mobile terminals, simulations can be performed with both practical speed and accuracy without freezing. In particular, by setting detailed collision detection for floors and walls, realistic physical simulations can be achieved.

[0093] The term "bounding box" refers to a simple rectangular prism or sphere that encloses an object, and can be interpreted as a "collision detection box," "boundary volume," or "simple collider."

[0094] "Polygon mesh" refers to a collection of polygons that make up the surface of an object, and can be interpreted as "mesh collider," "detailed shape," or "concave collider," among others.

[0095] The determination unit 5 may perform LOD (Level of Detail) control, which normally uses a bounding box (Box Collider) for determination, and automatically switches to a mesh collider only when detailed determination is required, such as when approaching an obstacle within a certain distance or passing through a narrow space.

[0096] For example, with simple box detection, details such as the footrests, handrims, and foot space of a wheelchair can be misinterpreted as "objects present" even in "empty spaces," potentially leading to false positives where passage is impossible. By using mesh detection, these gaps can be accurately recognized, enabling precise passage detection down to the millimeter, such as whether knees can fit under a table or whether a door frame protrusion can be avoided. Furthermore, processing efficiency can be optimized by adjusting the detection frequency according to the speed of the moving object or by omitting detection of less important objects (such as curtains).

[0097] (1.2.11) Eleventh aspect In the eleventh embodiment, the presentation processing unit 6 may perform a process to visually highlight the contact area in the moving object model and / or environment model when contact is detected by the determination unit 5.

[0098] According to the 11th aspect, the user can intuitively understand "where" and "what" they are bumping into. Rather than simply stating that they "cannot pass," the specific point of contact (for example, "the right footrest is hitting the door frame" or "the luggage behind is touching the wall") is clearly indicated, providing the user with specific and useful information to try avoidance maneuvers (such as moving a little to the left to pass, or moving the luggage to their knees) or to determine whether they can pass with assistance.

[0099] Furthermore, "highlighting" refers to actions such as changing the color, making it blink, displaying a frame, or making a silhouette stand out, and may be interpreted as "highlight display," "warning color display," "effect display," or "visualization."

[0100] The term "contact area" refers to, for example, the region or point where two objects overlap, and can be reinterpreted as "interference point," "collision point," or "intersection region."

[0101] The display processing unit 6 changes the rendering color of the detected polygon or part to red, or displays a semi-transparent spherical marker. It may also change the intensity of the color according to the depth and strength of the contact, or differentiate the display of contact with "hard objects" such as walls and contact with "soft objects (passable)" such as curtains using different colors. Furthermore, it is effective to display text messages or icons on the screen, such as "front right contact," or to provide voice guidance for visually impaired users (e.g., "the right tire is hitting something"). This allows users who have difficulty understanding spatial relationships in 3D space to easily understand the situation and take countermeasures. Additionally, by recording the contact history as a log and reviewing it later, it is possible to identify areas where movement is difficult and use this information for operation practice.

[0102] (1.2.12) The twelfth aspect In the twelfth embodiment, the information processing system 10 may further include a search processing unit 8 that receives the setting of a starting point and / or destination point for the movement of a mobile model, and searches for and presents a path that the mobile model can traverse based on the size and / or motion characteristics of the mobile model.

[0103] According to the twelfth aspect, the system can automatically find a passable route and present it as a navigation path without the user having to manually operate and confirm it. This significantly reduces the effort required when planning movement within a large facility, and allows even users unfamiliar with the system to instantly determine whether they can proceed. Furthermore, since a personalized optimal route tailored to the user's vehicle specifications is proposed, the efficiency and safety of movement are improved.

[0104] Furthermore, the term "search processing unit" refers to a function that executes a route search algorithm, and may be interpreted as "navigation engine," "route planner," "pathfinder," etc.

[0105] "Passable route" refers to a trajectory that does not physically interfere with a moving object and is traversable in terms of its motion capabilities, and may be interpreted as "accessible route," "movable path," or "recommended route."

[0106] The search processing unit 8 performs spatial analysis of the environment model (e.g., generation of a navigation mesh and voxel analysis) and searches for a path using algorithms such as A* (A*) algorithm, Dijkstra's algorithm, and RRT (Rapidly-exploring Random Tree). In this process, physical conditions such as whether the path width is greater than or equal to the width of the moving object model plus a safety margin, whether steps and gradients are within the climbing ability, and whether there is space necessary for turning are incorporated as cost functions, not just distance. The resulting path is drawn as lines and arrows in 3D space and can also be displayed overlaid on the real space using AR display. In addition, multiple path candidates (shortest route, safe route, covered route, etc.) may be presented and the user can select one.

[0107] (1.2.13) The 13th aspect According to the 13th embodiment, if the determination unit 5 determines that the mobile object model cannot pass through, the search processing unit 8 may perform a process of searching for and presenting alternative facilities or route information that the mobile object model can pass through.

[0108] According to the 13th aspect, if the simulation results show that the user cannot go to their desired destination, instead of simply discouraging them, constructive alternatives (such as "this route is longer, but it will get you there" or "the accessible restroom in the next building is spacious enough") are immediately presented, thereby expanding the user's options for action and maintaining or improving their quality of life (QOL). This provides a sense of psychological security regarding potential problems while out and about.

[0109] Furthermore, "impossible to pass" refers to a state in which, for example, no route to the destination is physically passable or deemed to be highly dangerous, and may be interpreted as "inaccessible," "NG judgment," "stuck," etc.

[0110] "Alternative facility or route information" means, for example, another location with similar functionality or an alternative route, and may be interpreted as "detour route," "alternative spot," "second option," etc.

[0111] For example, if the entrance to the target accessible toilet is too narrow to enter, the search processing unit 8 searches the database for data on other accessible toilets in the vicinity of the current location and lists toilets that are passable (simulated or have sufficient specifications) using the user's mobile model, presenting them on a map or in a list format. It can also provide suggestions that adapt to dynamic situation changes, such as suggesting an alternative route using a ramp if the elevator is under maintenance and unusable. This allows the user to smoothly change their destination or means of transportation, enabling them to achieve their goal without giving up on going out. Furthermore, when presenting alternatives, the system also displays the increase in travel distance and time to support the user's decision-making.

[0112] (2) Example of device configuration The following describes examples of the configurations of each device according to this embodiment.

[0113] (2.1) Example of user terminal configuration Referring to Figure 4, an example of the configuration of the user terminal 200 will be described in detail. In this embodiment, the user terminal 200 functions as an edge device equipped with advanced sensing and processing capabilities, and is responsible for data acquisition, generation, and simulation execution.

[0114] The user terminal 200 includes an image input unit 210, a LiDAR sensor unit 220, an operation input unit 230, an image output unit 240, an audio / vibration output unit 250, a communication unit 260, a storage unit 270, and a control unit 280, all of which are interconnected via a bus or the like. These components are generally integrated into the casing of a smartphone or tablet terminal, but some (for example, a LiDAR sensor or a high-precision GPS module) may be externally connected as accessories via USB-C or Bluetooth.

[0115] The image input unit 210 is a camera module that captures images (still images, videos) of the real world. For example, a multi-camera configuration with a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera equipped with a CMOS image sensor is used. The control unit 280 controls the image input unit 210 to acquire color information (texture) of the subject during scanning and background video for AR display during simulation. It also works in conjunction with SLAM (Simultaneous Localization and Mapping) technology to track feature points in the camera video, enabling highly accurate self-position estimation of the terminal. Furthermore, it is used as input data for AI-based object recognition processing (semantic segmentation of doors, chairs, steps, etc.), supporting the assignment of attributes to the environment model.

[0116] The LiDAR sensor unit 220 (Light Detection and Ranging) is a distance measuring sensor that measures the distance and shape of an object with high precision and speed by irradiating the object with pulsed infrared laser light and measuring the time it takes for the reflected light to return (ToF: Time of Flight). In this embodiment, the LiDAR sensor unit 220 acquires several thousand to several million point cloud data points per second based on instructions from the control unit 280. The control unit 280 integrates this point cloud data with image data from the image input unit 210 in real time to generate actual-scale mesh data. This process is updated sequentially as the terminal moves during scanning, and the scanned area is fed back to the user in AR display. Accurate acquisition of depth information is important for ensuring the reliability of collision detection in simulations. In the case of terminals that are not equipped with a LiDAR sensor, a configuration that acquires and supplements depth information using a ToF sensor, stereo camera, or depth estimation AI using a monocular camera is also acceptable.

[0117] The operation input unit 230 is an interface that receives operation instructions from the user. Specifically, this includes capacitive touch sensors on the touch panel display, physical buttons for power and volume control, and a microphone array for voice input. During simulation, the control unit 280 displays a virtual joystick and buttons on the screen and converts the user's touch operations into control signals for forward / backward movement, turning, and viewpoint changes of the mobile model. It also accepts intuitive input methods such as "tilting the terminal" using gyro sensors and accelerometers, as well as voice commands ("forward," "stop," etc.). Furthermore, by supporting NUI (Natural User Interface) such as eye tracking and hand tracking, the convenience for users with upper limb disabilities can be enhanced.

[0118] The image output unit 240 is a display that shows various screens and simulation results. High-resolution and high-brightness liquid crystal displays (LCDs), organic light-emitting diodes (OLEDs), micro-LED displays, etc., are used. The image output unit 240 displays AR images, which are created by combining a virtual moving object model with images of the real world captured by a camera, and simulation images, which are created by combining a moving object model and an environment model in a virtual space (VR space), after rendering processing by the control unit 280. At this time, the control unit 280 performs LOD (Level of Detail) control to dynamically adjust the number of polygons and texture resolution of the displayed objects in order to maintain the rendering frame rate. Furthermore, it is desirable to have a high-brightness mode that ensures visibility even in ambient light, and a display mode that supports color universal design that takes into account color vision diversity, in anticipation of outdoor use.

[0119] The audio / vibration output unit 250 is a device that provides auditory and tactile feedback to the user. This includes stereo speakers and linear resonant actuators (haptic devices). The control unit 280 drives the audio / vibration output unit 250 based on collision detection signals from the judgment unit 5 during the simulation. For example, if the moving object model collides with a wall, the speaker outputs a collision sound or warning sound, and at the same time vibrates the vibrator in a specific pattern, intuitively conveying to the user the sensation of "collision." In particular, tactile feedback is an effective means of conveying the presence of obstacles or unevenness in the road surface (such as a bumpy road) to users who cannot focus on the screen or who have visual impairments.

[0120] The communication unit 260 is a communication interface for communicating with the server 100 and other devices via the network 15. It includes a 5G / 4G (LTE) module, a Wi-Fi 6 / 6E / 7 module, and a Bluetooth 5.x / LE module for short-range wireless communication, all capable of high-speed, high-capacity communication. Based on instructions from the control unit 280, the communication unit 260 asynchronously uploads the generated large-capacity 3D model data to the cloud and downloads the simulation target environment data in the background. This ensures that user operation is not interrupted even during data transfer. Furthermore, during multi-user simulation, it synchronizes location information and operation information with other user terminals in real time (using WebSocket, etc.) to achieve low-latency interaction.

[0121] The memory unit 270 is a storage device that stores various programs and data. It consists of an NVMe-connected SSD or UFS (Universal Flash Storage) to speed up the startup of the OS and applications, and RAM such as LPDDR5 that functions as an execution area. The memory unit 270 stores application programs for operating this system (such as the "WheeLog!" application), 3D model data of the user's mobile body (GLB / USDZ files, etc.), environment data downloaded from the server 100 and cached, user profile information (physical characteristics, mobile body specifications), and temporary data during scanning. The control unit 280 manages access to this data and protects data related to privacy in particular through encryption and sandboxing.

[0122] The control unit 280 is often implemented as a SoC (System on a Chip) and is composed of processor cores such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), and ISP (Image Signal Processor), and comprehensively controls the operation of the entire user terminal 200. In this embodiment, the control unit 280 functions as each of the functional units shown in Figure 2 (first acquisition unit 1, second acquisition unit 2, synthesis processing unit 3, movement control unit 4, determination unit 5, presentation processing unit 6, and search processing unit 8) by executing a program stored in the storage unit 270.

[0123] In particular, processing 3D data (point cloud processing, mesh generation), physics calculations (rigid body simulation), and rendering (shading, lighting) place a high computational load on the system. Therefore, the control unit 280 utilizes dedicated processors such as GPUs and NPUs to efficiently perform parallel processing. This enables a processing method that balances lightweight design and accuracy when executing real-time 3D simulations on general-purpose devices. Alternatively, these advanced functions may be implemented using game engines such as Unity® or Unreal Engine®, or OS-standard AR frameworks such as ARKit® or ARCore®.

[0124] (2.2) Example Server Configuration Referring to Figures 5 and 6, an example of the configuration of Server 100 will be described in detail. Server 100 functions as a backend infrastructure that aggregates and manages data for the entire system and provides stable services.

[0125] Server 100 comprises a communication unit 110, a storage unit 120, and a processing unit 130, which are implemented as rack-mount servers in a data center or as virtual resources in the cloud.

[0126] The communication unit 110 is a high-bandwidth interface that communicates simultaneously with numerous user terminals 200 via the network 15, receiving requests and sending / receiving data. The communication unit 110 uses secure protocols such as HTTPS to prevent data eavesdropping and tampering. In addition, to stably process a large volume of access and transfer of large amounts of data, it has the function of appropriately distributing and controlling traffic in cooperation with load balancers and CDNs (Content Delivery Networks).

[0127] The storage unit 120 is a large-scale database group that stores and manages various types of data used in this system. It is composed of relational databases (RDBs), NoSQL databases, object storage, etc., combined in appropriate ways. As shown in Figure 6, the storage unit 120 mainly includes a "user information DB (database)" and an "environment information DB".

[0128] The "User Information DB" (Figure 6(a)) is a database that manages each user's personal information. In addition to the user ID, encrypted password, and profile information (age, language used, etc.), it stores the "Mobile Model ID" registered by the user and the corresponding physical data, the "First 3D Model Data (Mobile Model Data)." Data disclosure settings (public / private / limited access), mobile body motion characteristic parameters (minimum turning radius, climbing ability, overall width / length, etc.), and accessibility settings (color vision correction, font size, etc.) are also stored here and used by the processing unit 130 to provide personalized services.

[0129] The "Environmental Information DB" (Figure 6(b)) is a geospatial database that manages environmental data submitted by users or provided by system administrators. It stores environmental model IDs, precise location information (latitude, longitude, altitude), addresses, spot names, categories (barrier-free toilets, elevators, ramps, shops, stations, seating areas, parking lots, etc.), and contributor user IDs, as well as the actual data, the "second 3D model data (environmental model data)". Metadata such as data evaluation (number of likes, reviews, number of "attended / not attended" reports), data accuracy information (scan quality, presence or absence of LiDAR), shooting date and time, and tag information ("spacious," "with handrails," etc.) is also managed and used for searching and filtering by the processing unit 130.

[0130] The processing unit 130 is a high-performance processor (multi-core CPU, GPU, etc.) that performs calculations for the server 100. In addition to performing the functions of the data management unit 7 in Figure 2, it is also responsible for various backend processes. The processing unit 130 performs the following processes in response to API requests from the user terminal 200.

[0131] • Data registration and purification processing: The received 3D model data is subjected to inappropriate content detection using machine learning models, automatic masking of personal information (such as blurring faces), standardization of data formats, and optimization by polygon reduction. A unique ID is then assigned, and the data is stored in the storage unit 120.

[0132] • Data retrieval and distribution processing: Based on location information, search keywords, and filter conditions transmitted from the user terminal 200, the system rapidly searches the environmental information database using a spatial index (such as R-Tree). Then, it streams or downloads the list of relevant environmental data, thumbnails, and 3D model data to the user terminal 200, optimizing them according to the network bandwidth.

[0133] • Data Integration Processing (Spatial Merging): When multiple 3D data (point cloud data, etc.) exist for the same location, batch processing is performed periodically or on demand to align and merge these data to generate higher-quality data that covers a wider area and has fewer data gaps. During this process, data integrity is checked, and mutual exclusion control is implemented, such as prioritizing the newer data if inconsistencies are found.

[0134] • Authentication and Permission Management Process: This process ensures secure data access by performing user login authentication using OAuth, etc., and managing data access permissions (ACLs).

[0135] • Analysis Processing: Analyzes accumulated data to generate maps showing the completeness of accessibility information, analyze trends in frequently searched locations, and generate recommendations for users.

[0136] Furthermore, server 100 does not necessarily have to be a single physical server; it could be a distributed system where each function is microserviced and managed by a container orchestration tool, or a serverless architecture composed of a combination of cloud storage, cloud databases, and serverless computing functions. This allows for flexible scaling (auto-scaling) in response to increases or decreases in access numbers and data volume, achieving both system availability and cost efficiency.

[0137] (3) Examples of operation Referring to Figures 7, 8, 9, and 10, the specific operation flow using the information processing system 10 according to this embodiment will be explained in detail, along with the user experience (UX) and the internal processing of the system. Here, we take the example of a user of an electric wheelchair verifying at home in advance whether they can comfortably use the barrier-free restrooms of a commercial facility they are visiting for the first time. Another example is to check access to spectator seats and parking lots at a specific large-scale facility (for example, a race track or stadium).

[0138] (3.1) Phase 1: Generation of 3D data (scanning) First, as a preparatory step for the simulation, we will explain the process of generating 3D data of the moving object and the environment. This is the "data accumulation phase" that forms the foundation of this system.

[0139] (3.1.1) Scanning of mobile data Figure 7 is a conceptual diagram showing the scanning process for mobile object data.

[0140] Users scan their wheelchair (mobility device) to create a 3D model (first 3D model data). This process usually only needs to be done once, and only needs to be updated when there are changes in the model or equipment (addition of a new cushion, change of backpack, etc.).

[0141] As shown in Figure 7, the user (or caregiver) holds the user terminal 200 (a LiDAR-equipped smartphone or tablet) in their hand and scans the target wheelchair while moving around it. The app automatically starts scanning when it recognizes the object and overlays the acquired areas on the screen in a mesh pattern. The user moves the terminal from various angles to take pictures so that this mesh covers the entire wheelchair. If the movement is too fast during scanning or if feature points cannot be obtained due to insufficient lighting, guidance such as "Please move more slowly" or "Please move to a brighter location" is displayed in real time to prevent failures.

[0142] The key point here is to scan the wheelchair not in isolation, but in a state that faithfully replicates the user's usual mobility style. Specifically, this includes a backpack on the back, luggage under the seat, drink holders, cane holders, and any medical equipment such as ventilators or suction devices. These attachments can cause unexpected interference when turning or passing through narrow spaces.

[0143] Furthermore, if possible, it is desirable for the user to sit in the vehicle with the help of an assistant and perform the scan in their usual posture (feet on the footrest, recline angle adjusted, knees spread, etc.). This generates the "actual shape that occupies space (bounding volume)," including the user's knee protrusion, head height, elbow position, etc. The generated 3D model can be previewed simply within the app, and after trimming and editing out unnecessary noise (floor and background), it is saved. Because this data includes the individual's physical characteristics, for privacy reasons, it is stored only in a secure area on the user terminal 200 (or a private area on the server 100) and is set by default not to be made public to others.

[0144] Furthermore, any missing parts (occlusion regions) that could not be obtained during scanning may be subsequently filled in with simple primitive shapes (such as cylinders) or approximated using bounding boxes.

[0145] (3.1.2) Scanning facility data Figure 8 is a conceptual diagram showing the scanning process of facility data.

[0146] Information providers (users, facility managers, volunteers, etc.) use the user terminal 200 to scan the environment of the facility, such as the toilets. The provider launches a dedicated scanning app and slowly walks around the toilet while checking the AR feedback (mesh display, etc.) superimposed on the camera image, taking pictures of the walls, floor, toilet bowl, handrails, sink, waste bin, baby changing table, and other equipment from various angles.

[0147] In particular, to ensure that critical elements for wheelchair maneuverability, such as "effective width of the entrance (including the thickness of the door stopper)," "space next to the toilet (lateral approach space for transfer)," "height and position of handrails (movable or fixed)," and "obstacles at foot level (location of pipes, pedals, trash cans)," are scanned without blind spots, the device is moved up, down, left, and right to acquire data. For example, the back of the toilet and under the sink are points that are often overlooked, but the app highlights unscanned areas to prevent them from being missed.

[0148] The user terminal 200 integrates high-precision depth information acquired by the LiDAR sensor with camera footage in real time to generate a full-scale textured mesh model (second 3D model data). After the scan is complete, unnecessary parts (passersby, temporarily placed cleaning equipment, extraneous background, etc.) are trimmed and edited on the app, and the model is exported in a general-purpose format such as GLB.

[0149] The data is then imported into the system's app, linked to point-of-interest (POI) information on the map and facility area map information, and uploaded (registered) to server 100 as "○○ Building 1F Barrier-Free Toilet". At this time, server 100 performs image analysis on the uploaded data, and if it contains personally identifiable information such as people's faces or car license plates, it automatically blurs (masks) the image to ensure privacy protection. In addition, the data quality (polygon count, texture resolution) is checked, and optimization processing is performed as needed to reduce the communication load during download.

[0150] (3.2) Phase 2: Run the simulation Next, we will explain the operations performed by the user (information user) who wants to perform the simulation.

[0151] The user selects the desired toilet location on the app on user terminal 200 and taps the "View in 3D" button on the details screen. The 3D model data (second 3D model data) of that toilet is then streamed or downloaded from server 100. The downloaded data is cached (temporarily stored) on the terminal for a certain period, reducing the amount of data transmitted when viewing the same location again. The app loads the downloaded environment model (toilet) and the user's pre-registered mobile device model (wheelchair) and places them in the virtual space. At this time, the scale of both is precisely adjusted to 1:1 (the same ratio as in real space) based on metadata, so 1 centimeter on the screen corresponds to 1 centimeter in reality. The initial placement is automatically set to a natural location where the user would begin their approach, such as just before the toilet entrance.

[0152] Figure 9 shows the simulation screen.

[0153] The top of the screen (main view) displays the virtual space. In this example, a mobile device model (wheelchair) is placed within an environment model (toilet). The viewpoint can be flexibly switched according to the user's preference. In addition to a third-person view (TPS) from above and behind the mobile device model as shown in the illustration, users can choose from a first-person view (FPS) from the user's eye level, an overhead view (top view) from directly above, or a close-up view focusing on a specific obstacle. In the FPS view, users can realistically check the feeling of confinement when actually entering the toilet and the ease of reaching handrails and switches, while the top view allows users to grasp the overall amount of turning space at a glance.

[0154] At the bottom of the screen, a virtual controller for operating the mobile device is displayed. The joystick on the left controls forward and backward movement and turning, while the slider and swipe gestures on the right rotate and zoom the camera view. The control sensitivity can be adjusted to the user's preference, recreating a feel close to that of operating a real electric wheelchair's joystick.

[0155] While viewing this screen, users operate a virtual controller to verify a series of usage actions, such as "Is there enough clearance for the width of the entrance door?", "Can I move to the side of the toilet and turn 180 degrees?", and "Can I get to the transfer position without the handrail getting in the way?". The wheelchair on the screen is the user's own wheelchair, so they can operate it with their "usual feeling" and experience a realistic sense of size and confinement. The screen also features an AR measuring function that displays numerical values ​​such as "the distance from the current location to the toilet" and "the width of the passageway," supporting more quantitative decision-making.

[0156] (3.3) Phase 3: Collision detection and feedback During the simulation, the system is constantly performing high-speed collision detection in the background. Figure 10 is a diagram illustrating contact detection.

[0157] Figure 10(a) shows an example of a simplified detection method using a "Box Collider." A simple rectangular prism (dotted outline) is set up to cover the entire wheelchair, and the system determines whether this box touches a wall or obstacle. While the processing load is light, it may fail to recognize "gaps" such as under the armrests or at the feet, potentially resulting in a false positive (a collision being detected even when the knees are actually under the table).

[0158] Figure 10(b) shows an example of high-precision detection using a "Mesh Collider." The detection is performed using the polygon mesh itself that conforms to the complex shape of the wheelchair (mesh of the pipes, unevenness of the footrests, shape of the tires, bulge of the backpack). Although the processing load is heavy, it can accurately reflect the gaps in the footrests and the shape of the handle, so it can accurately determine whether turning or approaching in a tight space is possible. In this embodiment, a hybrid control system is used, which normally uses a box collider and automatically switches to a mesh collider when approaching an obstacle or stopping, thereby achieving both performance and accuracy.

[0159] In this embodiment, based on this contact detection, if the wheelchair model comes into contact with a wall, door frame, handrail, or other protruding object such as a paper holder, the system immediately provides multi-sensory feedback.

[0160] • Visual feedback: The contact area (e.g., the right footrest) and the surrounding environment (e.g., the door frame) are illuminated in red to highlight them. As the remaining distance to the contact point decreases, the display color changes from green to yellow to red, intuitively conveying the level of danger.

[0161] • Haptic feedback: The device vibrates (haptics). By changing the vibration pattern (intensity, rhythm) according to the strength of the contact and whether it is a "rub" or a "collision," the user can understand the contact situation even without looking at the screen.

[0162] • Auditory feedback: Plays a collision sound such as "thud" or an audio guide saying "the right side is hitting."

[0163] • Reproduction of physical behavior: The physics engine reproduces behavior that follows real-world physical laws, such as being unable to move further after a collision (being blocked by a wall), or changing direction as if sliding when hitting at an angle. This allows players to intuitively understand that "you can't force your way through."

[0164] Furthermore, the system not only alerts the user to a collision but also provides advice for avoidance. For example, it assists the user with messages such as "You can pass if you move 5 centimeters to the left" and a guide function that displays a recommended route with arrows.

[0165] This allows you to identify specific and critical problems before going to the site, such as "the backpack hits the wall when you go too far in, preventing you from turning" or "the fixed handrail gets in the way, preventing you from getting close to the ideal transfer position," and then take countermeasures (such as putting down luggage, finding another toilet, or having a caregiver hold the door open).

[0166] (4) Examples Referring to Figures 11 to 24, a more specific implementation example of the information processing system 10 according to this embodiment, the data processing flow, and the configuration of the user interface (UI) will be described in detail. In this embodiment, the main description will be of a configuration in which an application (hereinafter referred to as "3D simulation app" or simply "app") that runs on a mobile terminal such as a smartphone or tablet equipped with a LiDAR scanner and a high-performance computing processor, and a barrier-free map app (e.g., "WheeLog!") operate in seamless cooperation via APIs, URL schemes, etc.

[0167] In this embodiment, the system may further include a result management unit that generates simulation result data including the result of the determination by the determination unit or the movement history by the movement control unit, and stores the simulation result data in a server in association with the environment model. The second acquisition unit may acquire the simulation result data by other users stored in the server, and the presentation processing unit may present the simulation result data by other users as reference information.

[0168] Furthermore, this embodiment may also include a viewpoint control unit that controls the viewpoint when displaying the virtual three-dimensional space. The viewpoint control unit may switch between displaying a third-person viewpoint that provides an overview of the space including the mobile model, and a first-person viewpoint that reproduces the view of a user riding in the mobile model, based on user operation.

[0169] Furthermore, in this embodiment, the synthesis processing unit may detect the floor surface in the environment model and automatically adjust the height position of the mobile body model so that the contact surface of the mobile body model is in contact with the floor surface.

[0170] (4.1) System Integration and Data Structure Figure 11 shows an overview of the system integration and data flow related to this embodiment.

[0171] This system employs a distributed architecture in which three elements work together, primarily with a division of functional roles: "3D simulation application (client-side)," "server (spot information and user information management)," and "cloud storage (physical file group management)."

[0172] Server 100 (see Figure 1) manages "spot information," which is text-based metadata about specific locations (spots) on the map, such as their name, address, coordinates, category, and accessibility attributes (e.g., availability of accessible toilets, elevator size), as well as "user information," which manages the user's profile, login credentials, and application settings. This server is responsible for responding quickly to lightweight requests from clients.

[0173] On the other hand, the 3D model data itself, which forms the core of the simulation (GLB files, FBX files, USDZ files, point cloud data, etc.), has a data capacity ranging from several megabytes to several hundred megabytes. Therefore, it is stored as a "collection of physical files" in cloud storage (object storage, etc.) that is physically or logically separated from the relational database of Server 100. This makes it possible to efficiently and quickly deliver large amounts of 3D data via a CDN (Content Delivery Network) while maintaining the response performance of the database during metadata searches.

[0174] As shown in Figure 11, the 3D simulation application first obtains reference information for the target spot (URL on storage, hash value, metadata) from the server. Then, based on this information, it directly downloads the actual files of the "3D structure model (environment model)" and "3D wheelchair model (mobility model)" from cloud storage as needed, expands them into memory in the local environment (within user terminal 200), and runs the simulation.

[0175] In this case, the 3D structural models are captured and cropped by other users using general-purpose 3D scanning applications, edited on a PC or similar device (such as deleting unnecessary polygons or filling in holes - splitting and merging), and then uploaded. They are managed as shared resources linked to a spot ID. On the other hand, 3D wheelchair models are either created by the user themselves through photography, or are provided by manufacturers or created as presets by volunteers. Individual physical properties (mass, center of rotation, drive wheel position, caster angle, etc.) can be set.

[0176] The simulation results (movement routes, contact points with walls and obstacles, user feedback and evaluations, etc.) are sent to the server as feedback information, stored as comments on spot information and "driving performance data," and shared with other users, contributing to the qualitative improvement of barrier-free information.

[0177] (4.2) Application Processing Flow Figure 12 shows the overall processing flow of the application according to this embodiment.

[0178] First, the user launches the app (for example, the linked barrier-free map app) and performs a normal spot information search. They select a specific facility (spot) they are considering using from the map's pin display or a list display based on keyword search, and the detailed information screen for that spot is displayed. At this point, the system queries the server in the background to determine if a 3D model (3D structure model) associated with that spot exists, and displays it on the UI as an icon or similar.

[0179] If a 3D model exists (the "Yes" route), the user selects the 3D structural model to use in the simulation. For a single location (e.g., "XX Station"), multiple 3D models may be registered (e.g., "From the East Exit ticket gate to the elevator," "Accessible toilet A (1st floor)," "Accessible toilet B (2nd floor)," "Waiting room on the platform," etc.). The user selects the appropriate one from the list, referring to the thumbnail image, date and time of shooting, and ratings from other users (number of stars, etc.).

[0180] Next, the user selects a "3D wheelchair model" to use in the simulation. This can be chosen from the user's own "My Model," which is pre-registered and saved, or from "Preset Models" of representative models included as standard in the app. Multiple My Models can be registered depending on whether luggage is present or the type of wheelchair used (manual / electric), allowing users to switch between them as needed. Customization, such as fine-tuning the size, can also be done immediately before the simulation if necessary.

[0181] Once these selections are complete, the 3D simulation function is invoked, and after a loading screen, the simulation begins. The simulation function launches as a separate window (or separate process, or submodule implemented in Unity, etc.) from the map application, providing an immersive user experience. After the simulation is complete (after the report is generated), the user seamlessly returns to the original map application screen, ensuring that the user experience is not interrupted.

[0182] On the other hand, if a 3D model does not exist (the "none" route), the user can become a data provider (contributor). The user launches a compatible 3D scanning app and takes a picture of the structure at the location. After taking the picture, they crop out unnecessary backgrounds and process the image for privacy protection, then enter metadata (spot name, additional information about the shooting location, shooting date, etc.) and upload it. The uploaded data is linked to the spot information immediately, or after an administrator approval process (checking for inappropriate content, etc.), and becomes available for the user and other users to use for simulations in the future.

[0183] (4.3) Internal processing of the simulation engine Figure 13 shows the internal processing blocks and data flow during simulation execution.

[0184] The simulation process, in order to ensure the stability and realism of the system, is broadly composed of five stages: "data setup," "initial position determination," "model grounding processing," "3D simulation execution," and "SIM report presentation."

[0185] During the "data setup" phase, the selected 3D structure models and 3D wheelchair models are loaded from the cloud or local cache and placed in GPU memory for rendering preparation. At this stage, data used once is cached on the device, significantly reducing loading time for subsequent uses.

[0186] In the "Initial Position Determination" stage, you decide where in the 3D space within the structure model the wheelchair model will appear (spawn). Basically, it will be placed in the "default position (near the entrance or the start of a passageway, etc.)" and "default orientation" saved as metadata in the structure data, but it is also possible for the user to manually change the placement position or start from any position.

[0187] In the "Model Grounding Processing" stage, physics calculations or raycasting are used to detect the floor surface (mesh) of the structural model and automatically snap the wheelchair's tire contact points to prevent the wheelchair model from floating in the air or sinking into the floor. This allows the simulation to start with the correct orientation relative to gravity, even if the coordinate axes of the scan data are tilted and off-horizon.

[0188] The "3D simulation execution" phase is the core of this system, and the following functions operate in parallel on a frame-by-frame basis.

[0189] • Movement SIM (Free Operation): A movement control system that calculates the acceleration, velocity, and angular velocity of the wheelchair based on user input (joystick operation, etc.) and updates its position coordinates.

[0190] • Two-model collision detection: Interference checks at the polygon level or bounding box level between structures (static colliders) and wheelchairs (dynamic colliders).

[0191] • Arbitrary viewpoint / First-person viewpoint: Real-time calculation and rendering of camera angles that follow user actions and wheelchair movement.

[0192] • Multiple wheelchair placement: For multiplayer and crowd simulations, multiple models can be placed in the same space and synchronized.

[0193] • Physical SIM (Slope Behavior): Physical calculations such as acceleration / deceleration based on the slope (normal vector) of the floor surface, the effects of friction and gravity, and determination of whether or not it is possible to climb the slope.

[0194] • Contact and movement path recording: The coordinates of the movement path, as well as the coordinates and number of times contact occurred, are temporarily saved as a log.

[0195] After these processes, the user is presented with and saved a "SIM report" that visualizes the total distance traveled during the simulation, the number of contacts, and particularly dangerous areas that were difficult to pass through.

[0196] (4.4) Function Details and Parameter Settings Figure 14 is a diagram showing a list of the main functions of the 3D wheelchair simulation according to this embodiment.

[0197] On the system side (SYS), instead of developing its own scanner function, it incorporates the ability to import files in general-purpose formats (GLB, USDZ, etc.) output from existing advanced 3D scanning apps (e.g., iOS's "Scaniverse" and "Polycam," Android's ARCore-compatible apps, etc.). This allows for the immediate adoption of the latest scanning technologies that improve in line with the evolution of the OS and devices (improved LiDAR accuracy and faster photogrammetry processing). Furthermore, the 3D data is stored in external cloud storage, and the loosely coupled design, which only exchanges reference information (links) with the app itself, enhances the system's scalability and maintainability.

[0198] In terms of simulation (SIM), preset data for commonly available wheelchairs is provided, and by setting parameters based on actual measurements (such as the center of rotation, drive wheel position, and seat height) in addition to the size, realistic behavior that cannot be determined from catalog specifications alone is reproduced. Furthermore, considering use in locations with poor communication environments and for demonstration experiences, some representative data is pre-installed or persistently cached on the device, allowing simulations to be experienced even in offline environments.

[0199] Figure 15 shows a list of the main parameters set for the wheelchair model (mobility model).

[0200] In addition to basic physical quantities such as "size (overall width, overall length, overall height)", "tire diameter (front and rear wheels)", and "mass (total weight including rider)", parameters that directly affect the feel of operation, such as "center of rotation (position of the pivot axis)" and "drive wheel position", are defined. For example, the turning trajectory (inner wheel difference, outer wheel difference) differs greatly depending on whether the center of rotation is in the center of the seat or on the rear wheel axle, affecting the maneuverability in narrow spaces, so this parameter is extremely important. Furthermore, by specifying the "forward direction", the system can correctly recognize and correct the direction of travel even if the orientation of the local coordinate system of the imported 3D model (whether the Z axis is forward or the Y axis is forward, etc.) is different. In addition, the presence or absence of a "caregiver" can be set as an optional item, and simulations that take into account the force (propulsion) applied by the caregiver, as well as scan quality information such as "whether LiDAR is used", are stored as metadata.

[0201] Figure 16 shows a list of the main parameters set in the structural model (environmental model).

[0202] The "starting position" defines the spawn point (X, Y, Z coordinates and orientation) at the start of the simulation. This ensures that users can start from the appropriate starting point each time. Additionally, the "contact and movement log" stores a history of past simulations at that location (where other users bumped into each other, what routes they took to reach their destinations) in the cloud. This history is then preserved and visualized as a heatmap (highlighting areas with frequent contact in red, etc.) and recommended route line data, transforming individual experiences into collective intelligence for accessibility information.

[0203] (4.5) Data Management Lifecycle Figure 17 shows the management flow of 3D wheelchair data.

[0204] Users register their 3D wheelchair data from their profile screen. Here, they select scanned real data saved on their device, enter a name (e.g., "Work Electric Wheelchair"), model number, comments, etc., and upload it. Registered data can be viewed on the "3D Wheelchair Data Details" screen, where information can be updated (parameter correction, etc.) or deleted. Administrators can search and view all user data from the administration screen and have the authority to forcibly make private or delete inappropriate data that violates the terms of service (e.g., data containing personal information, data contrary to public order and morals). Furthermore, a maintenance flow is defined for maintaining quality, in which administrators download uploaded 3D data, make corrections such as noise reduction, mesh reduction, and texture correction using high-performance 3D editing software on a PC, and then re-upload (update) the data.

[0205] Figure 18 shows the management flow of 3D facility data (environmental data).

[0206] Facility data can be registered via the "Spot Registration" flow on the map or through the existing "Spot Details" screen. Submitted data is managed in a tree structure as child data linked to that spot. Facility data can be updated and deleted by the contributor (owner), and strict quality control is performed by administrators. Facility data, in particular, is highly public and used by many users, so inaccurate information or data with low scan quality can have a significant impact. Therefore, curation by administrators (selection, duplicate removal, and quality improvement) is crucial for maintaining a reliable platform.

[0207] (4.6) Example of User Interface (UI) Implementation Figure 19 shows the UI transitions related to the registration and linking of 3D wheelchair data.

[0208] The profile screen displays the user's avatar and current level (contribution) along with registered "wheelchair information" in card format. Tapping the "3D Registration" button takes the user to the posting screen, where they can enter a name, date, and detailed comments, as well as set a "thumbnail" image to improve data visibility. Users upload and set thumbnails, such as screenshots that show the overall appearance of the 3D model. Registered wheelchair data is displayed as a list, and the model to be used when starting a simulation can be selected (checked) using radio buttons, etc.

[0209] Figure 20 shows the UI transitions related to the registration and linking of 3D facility data.

[0210] The spot details screen displays an indicator (such as a "3D Available" icon or label) that allows users to easily determine whether 3D data is available. Tapping this indicator displays a list of 3D data associated with the spot. Users can then select the data they want to view (e.g., "Multifunctional Toilet in General Restroom," "Entrance Ramp," etc.) and press the "3D SIM" button to immediately begin the simulation. The same screen also includes a "3D Data Submission" button, from which new data can be uploaded. When submitting data, users can add information such as a name, date, comments about the location's condition, and a thumbnail, similar to wheelchair data, to make it easier for other users to select the data.

[0211] Figure 21 shows the UI configuration of the main screen during simulation execution.

[0212] In the example screen (immediately after moving), the main view (third-person perspective, etc.) is displayed in the center of the screen, and a semi-transparent "Top View (minimap)" showing an overview of the current location is overlaid in the upper left. This Top View is useful for understanding what's beyond walls and the overall structure, and can be minimized (stored) to the edge of the screen when not needed, ensuring that it does not obstruct the visibility of the main view.

[0213] At the bottom of the screen is a "controller (virtual joystick)" for movement. This controller is displayed as a floating UI, allowing users to drag it to any position on the screen that is easily accessible to their fingers. The screen also features buttons to access debugging options that visualize the "bounding box" (boundary frame) used as the basis for collision detection, as well as functions that overlay the movement "trajectory" and "contact points" in real time.

[0214] In the example screen for (contact), when the moving object model comes into contact with a wall or obstacle, the contact point (e.g., the orange circle in the diagram) and the wall being contacted are visually highlighted. At the same time, the device vibrates (haptic feedback) and a warning sound is emitted, allowing the user to recognize the contact in ways other than visual.

[0215] Furthermore, the settings screen (pictured on the right), accessible via the "Settings" button, allows you to configure settings such as showing / hiding the on-screen UI (immersive mode), dynamically changing the wheelchair model (a function that allows you to switch to and try out different wheelchairs during the simulation), and adjusting the control sensitivity and viewpoint movement speed.

[0216] Figure 22 shows the UI for reporting and sharing functions after the simulation is completed.

[0217] When the simulation ends (or is interrupted), a report screen is displayed. This screen displays information about the location where the simulation was performed, information about the wheelchair used, and automatically generated thumbnails of the simulation results (such as an overhead image with the trajectory and contact points drawn like a heatmap). Users can add comments to this report, such as their impressions and points to note from the simulation (e.g., "The entrance was narrow, but the inside was spacious," "Turning required multiple maneuvers"), and save them.

[0218] Furthermore, the "sharing" function (SNS integration) allows users to post the report content to external SNS or messaging apps. In this process, image data is shared in a format accessible as a URL on the cloud, and text information is copied to the clipboard. This goes beyond simple individual verification, promoting the dissemination and sharing of accessibility information among users.

[0219] (4.7) Improvements to viewpoint control and controller (UI variations) Figure 23 shows proposed improvements to the UI related to changing the viewpoint.

[0220] In simulations, the controllability of the viewpoint (camera) is a crucial element directly related to the ease of spatial awareness and immersion. In this modified version, an intuitive "viewpoint switching icon" is placed on the screen, allowing seamless switching between "first-person view (FPS)" and "third-person view (TPS)" by tapping it.

[0221] • Icon 1 (human shape): Switches to first-person view mode. When selected, it lights up and the camera position moves to eye level above the seat of the wheelchair model (for example, about 110cm to 130cm above the ground). This reproduces the view from the perspective of an actual user, allowing you to check the sense of confinement and the ease of reaching switches.

[0222] • Icon 3 (shaped like a video camera): Switches to third-person view mode. It lights up when selected. Furthermore, the icon's lighting / blinking state indicates whether the camera follows the wheelchair's movement in third-person view (tracking mode) or remains fixed in a specific position (fixed-point mode).

[0223] This UI allows users to intuitively switch between perspectives depending on the situation: "third-person view (following) for an overview while moving," "first-person view for checking clearance in narrow spaces," and "fixed viewpoint for closer inspection of specific locations."

[0224] Figure 24 shows a proposed improvement to the controller (input interface).

[0225] In addition to the conventional circular analog joystick, a directional pad-style "button controller" will be offered. This is an accessibility feature for users who have difficulty with analog operations such as swiping, or for users who prefer precise directional input.

[0226] • Clarifying input direction: To prevent buttons (arrows) from being obscured by fingers, the tap area (rectangle) is made larger than the actual display and is semi-transparent. The button corresponding to the input (tap) direction lights up brightly, providing visual feedback that the operation has been accepted.

[0227] • Visualization of input vectors: The controller's central section displays an indicator (such as a line connecting "●" and "○", or the length of a vector) showing the direction and strength (analog value) of the input, making it easier to check the analog input status.

[0228] • Size adjustment function: The controller size (large, medium, small) can be changed from the settings screen to suit the user's hand size, range of motion, and preference.

[0229] (5) Variations and extensions In addition to the examples described above, the system's social usefulness and entertainment value can be further enhanced by implementing the following functional enhancements and specification changes.

[0230] (5.1) Next-generation 3D simulation function This system is not merely a tool for determining whether passage is permitted or not; it has the potential for multifaceted development in the fields of education, entertainment, and VR.

[0231] • Features for barrier-free education: Provides an editor function that allows users to easily create their own "miniature city" by freely placing obstacles (steps, poles, abandoned bicycles, etc.) and barrier-free facilities (ramps, handrails, etc.) within the city. This can be used to conduct "barrier-free experience simulations" where able-bodied individuals and students learn about the difficulties and ingenuity involved in wheelchair mobility, or to add elements of a "serious game" with game-like features such as aiming to reach a destination within a time limit, thereby promoting its use in educational settings and training.

[0232] • Barrier-free transportation and tourism support functions: By incorporating 3D terrain data of major tourist destinations and urban areas (for example, 3D urban model open data such as PLATEAU from the Ministry of Land, Infrastructure, Transport and Tourism), it will be possible to simulate travel not only to individual facilities but also to wide areas including travel from stations. In addition, it will promote entertainment use (metaverse use) such as a "virtual group photo" function that allows multiple users to gather in the same 3D space as avatars to communicate and take commemorative photos. Furthermore, it will enhance AR functionality that overlays 3D models onto real space, and improve on-site AR navigation and guide functions.

[0233] • VR Compatible: We provide apps that support not only smartphone screens but also PC-connected high-end VR headsets and standalone VR devices such as MetaQuest®. This enables simulations that allow users to experience more realistic viewpoint movement, distance perception, and elevation differences in an overwhelmingly immersive environment.

[0234] • Support for simulation peripherals: In addition to on-screen touch controls, the system will allow users to connect and operate joystick controllers used in actual electric wheelchairs via Bluetooth, USB, etc. Alternatively, existing gamepads can be customized for wheelchair operation, enabling rehabilitation training of wheelchair operation and verification that more closely resembles actual driving.

[0235] (5.2) Improvement and optimization of specifications To improve usability and system quality, the following minor specification improvements may be applied.

[0236] • Defaulting mobility selection: To eliminate the need to select a wheelchair each time a simulation starts, a specific mobility device (the one used last time or a registered device set as the user's main device) will be automatically selected by default. If there are multiple options, the selected device will be clearly indicated with a bounding box or highlight.

[0237] • Simplified initial positioning: When manually placing the wheelchair within a structural model, rotation of the wheelchair is restricted to the "yaw axis (Y-axis, rotation around the vertical axis)" only. Allowing rotation on all three axes could cause the wheelchair to tilt unintentionally and tip over, so this is prevented, and anyone can intuitively position the wheelchair.

[0238] • Icon behavior streamlined: The roles and behaviors of the "back button (previous screen)," "x (close) button," and "redo (reset) button" on the simulation screen will be clarified, and confirmation dialogs will be added to prevent unintended simulation interruptions due to accidental operation.

[0239] • Mobility changes and additions during simulation: Provides a flexible UI that allows you to dynamically change or add wheelchair models (such as verifying passing maneuvers by placing a second or subsequent wheelchair) from the overlay menu without interrupting the simulation.

[0240] • Improved camera control: In addition to panning the camera with a two-finger swipe, the rotation axis movement has been optimized to allow for smoother viewing of desired locations. A "▼" marker is now displayed above the currently controlled mobility to prevent confusion when multiple mobility units are deployed.

[0241] • Ground data completion: If the floor surface is missing (has a hole) in the scan data, the wheelchair will fall in the physics calculation. Therefore, a process is implemented to estimate the plane from the surrounding mesh and automatically generate an invisible collider (mesh for physics detection) to fill the hole. This ensures the continuity of the simulation even with incomplete scan data.

[0242] (6) Variant Modifications of the above-described embodiments and examples will now be explained.

[0243] (6.1) Variations and extensions of mobile units Although the above embodiment mainly uses a wheelchair as an example, the "mobile body" in the present invention is not limited to this and can be applied to various "means of transportation with spatial constraints."

[0244] • Strollers: For users of twin strollers or large buggies from overseas, the ability to pass through train station ticket gates, elevators, and supermarket checkout aisles is a critical issue. Changes in the center of gravity when carrying luggage on the stroller should also be considered.

[0245] • Senior scooters (handlebar-type electric wheelchairs): Due to their large size and limited maneuverability, they are useful for testing the spacing of bollards on sidewalks and for navigating crank-shaped entrances.

[0246] • Stretchers and medical carts: Allows for simulation of movement within hospitals and during emergency transport, including the passage of ancillary equipment such as IV stands.

[0247] • Delivery robots, security robots, cleaning robots: Before introducing autonomous robots, this can be used as an "implementation simulation tool" to verify whether the robot can navigate existing buildings (passageway width, steps, turning space, sensor blind spots, etc.) using 3D data without bringing in the actual robot. It is also possible to check the viewpoint from the robot's sensor position (LiDAR and camera height).

[0248] (6.2) Hybrid use of AR and VR In the above embodiment, we mainly described a VR (Virtual Reality) type simulation that is completed in a virtual space on the screen, but it may also be combined with AR (Augmented Reality) which utilizes images from the real world.

[0249] For example, in "reverse AR (on-site AR)," a 3D model of the toilet the user intends to go to is displayed in life-size using AR in a large, empty room where the user is currently located (such as a living room or gymnasium). The user then actually moves around within this virtual toilet image in their wheelchair. This allows for a realistic rehearsal that involves not only on-screen fingertip operations but also actual physical sensations (the feeling of turning the handrims, the G-force during turns, and the height of the line of sight). Conversely, it is also conceivable that a model of the wheelchair the user intends to purchase could be displayed in AR in an empty space on-site to check if it can pass through.

[0250] (6.3) Automatic route planning and difficulty heatmap The search processing unit 8 may not only show passable routes but also spatially visualize the "difficulty" of movement. A sweep test is performed on the environment model using the size and motion characteristics (turning radius, climbing ability) of the moving object model to calculate the passability margin.

[0251] The floor surface is color-coded using a heatmap: "blue" for areas with ample space, "yellow" for areas that are just barely passable, and "red" for areas that are impassable or have a high risk of contact. This allows users to choose routes based on their skill level; for example, beginners can choose the safer "blue" route even if it's longer, while experienced users can choose the shortest "yellow" route even if it's narrower. Furthermore, by accumulating this data, facility managers can be shown "barrier hotspots" and encouraged to make improvements.

[0252] (6.4) Simultaneous simulation by multiple users In the future, it will be possible to implement a function that allows multiple people to enter the same 3D space and perform simulations simultaneously. This will enable the simulation of crowded conditions, the possibility of passing each other, and group movement, allowing for the analysis and visualization of interactions and interferences.

[0253] (6.5) Applications to e-commerce sites and home renovation / design This system can be applied not only to verifying existing facilities, but also to verifying future acquisitions and environments.

[0254] • E-commerce: The system uses environmental data such as a scan of the home (entrance, hallway, toilet) and 3D models (manufacturer-provided data, etc.) of items such as a wheelchair to be purchased and caregiving equipment (care bed, lift, etc.) as mobile data. This allows customers to check how the items will be handled and placed in their home before purchasing, reducing the risk of returns.

[0255] • Home renovation and architectural design: When renovating or building a new home, we incorporate design drawings (BIM data, etc.) as an environmental model to verify whether it is possible to live comfortably in a wheelchair. By allowing the homeowner to virtually experience the flow of movement, which is difficult to visualize from drawings alone, we can prevent design errors and rework such as "being unable to turn corners in hallways" or "being unable to pass through when a door is open."

[0256] (7) Other embodiments Although embodiments have been described in detail above, these embodiments are not limited to the examples described above.

[0257] In the above embodiment, at least a portion of the processing performed by the server 100 may be modified to be performed on the user terminal 200 side. Alternatively, in the above embodiment, at least a portion of the processing performed by the user terminal 200 may be modified to be performed on the server 100 side.

[0258] The operation flow and operation examples in the above-described embodiments do not necessarily have to be executed chronologically in the order shown in the flowchart. For example, the steps in the operation may be executed in a different order than that shown in the flowchart, or they may be executed in parallel. Also, some of the steps in the operation may be deleted, or further steps may be added to the process.

[0259] A program may be provided that causes a computer (information processing device) to perform the operations according to the above embodiment. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient storage medium. The non-transient storage medium is not particularly limited, but may be a storage medium such as a CD-ROM or DVD-ROM.

[0260] The functions realized by the information processing system according to this embodiment may be implemented in a circuit or processing circuitry, including a general-purpose processor, an application-specific processor, an integrated circuit, an ASIC (Application Specific Integrated Circuit), a CPU (a Central Processing Unit), conventional circuits, and / or a combination thereof, which are programmed to realize the described functions. A processor, including transistors and other circuits, is considered a circuit or processing circuitry. A processor may be a programmed processor that executes a program stored in memory. In this specification, a circuit, a unit, or a means is hardware programmed to realize or execute the described functions. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to realize or execute the described functions. If the hardware is a processor which is considered a type of circuit, then the circuit, a means, or a unit is a combination of hardware and software used to constitute such hardware and / or processor.

[0261] As used herein, the terms "based on" and "responsive to" do not mean "only based on" or "only responsive to" unless otherwise specified. The term "based on" means both "only based on" and "at least partially based on". Similarly, the term "responsive to" means both "only responsive to" and "at least partially responsive to". Also, the terms "include", "comprise", and their variants do not mean to include only the recited items, but may include only the recited items or may further include additional items in addition to the recited items. Further, the term "or" used herein is not intended to be exclusive disjunction. In this specification, for example, when articles are added by translation, such as a, an, and the in English, these articles are assumed to include plural ones unless the context clearly indicates otherwise.

[0262] As described above, the embodiments have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist.

Description of Reference Numerals

[0263] 1 First acquisition unit 2 Second acquisition unit 3 Composition processing unit 4 Movement control unit 5 Determination unit 6 Presentation processing unit 7 Data management unit 8 Search processing unit 10 Information processing system 15 Network 20 User interface 100 Server 110 Communication unit 120 Storage unit (Environment information DB, User information DB) 130 Processing unit 200 User terminal 210 Image input unit 220 LiDAR sensor section 230 Operation Input Section 240 Image Output Unit 250 Audio / Vibration Output Unit 260 Communications Department 270 Storage section 280 Control Unit

Claims

1. A first acquisition unit that acquires first 3D model data generated by measuring the 3D shape of a mobile object used by the user based on the user's operation, A second acquisition unit acquires second three-dimensional model data corresponding to the environment in which the moving object is moving, A synthesis processing unit that, within a virtual three-dimensional space, positions a moving object model defined by the first three-dimensional model data relative to an environment model defined by the second three-dimensional model data, while maintaining dimensional consistency with the respective real spaces, A movement control unit that moves the mobile object model in the virtual three-dimensional space based on user operations, A determination unit that determines the mobility of the mobile model, including whether the mobile model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile model and the environment model, The system includes a presentation processing unit that performs processing to present information indicating the determination result by the determination unit, The movement control unit records the movement history of the moving object model in the virtual three-dimensional space based on the user's operation, and the presentation processing unit performs a process to present the simulation results, including the movement history and the determination result of the possibility of movement for that movement history, as the determination result by the determination unit. Information processing system.

2. A first acquisition unit that acquires first three-dimensional model data generated by measuring the three-dimensional shape of a mobile object used by the user based on the user's operation, A second acquisition unit acquires second three-dimensional model data corresponding to the environment in which the moving object is moving, A synthesis processing unit that, within a virtual three-dimensional space, positions a moving object model defined by the first three-dimensional model data relative to an environment model defined by the second three-dimensional model data, while maintaining dimensional consistency with the respective real spaces, A movement control unit that moves the mobile object model in the virtual three-dimensional space based on user operations, A determination unit that determines the mobility of the mobile model, including whether the mobile model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile model and the environment model, The system includes a presentation processing unit that performs processing to present information indicating the determination result by the determination unit, The aforementioned mobile body is a mobile body on which a user can board and / or which can carry luggage. The first acquisition unit acquires the first three-dimensional model data based on the three-dimensional data scanned while the mobile vehicle is carrying a user and / or loaded with luggage. Information processing system.

3. The first acquisition unit and the second acquisition unit acquire the first 3D model data and the second 3D model data, respectively, based on data generated by an external 3D scanning application and output in a general-purpose 3D file format. The information processing system according to claim 1.

4. A first acquisition unit that acquires first three-dimensional model data generated by measuring the three-dimensional shape of a mobile object used by the user based on the user's operation, A second acquisition unit acquires second three-dimensional model data corresponding to the environment in which the moving object is moving, A synthesis processing unit that, within a virtual three-dimensional space, positions a moving object model defined by the first three-dimensional model data relative to an environment model defined by the second three-dimensional model data, while maintaining dimensional consistency with the respective real spaces, A movement control unit that moves the mobile object model in the virtual three-dimensional space based on user operations, A determination unit that determines the mobility of the mobile model, including whether the mobile model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile model and the environment model, The system includes a presentation processing unit that performs processing to present information indicating the determination result by the determination unit, The second acquisition unit acquires the second 3D model data corresponding to the environment to which the moving object is moving, based on user operation, from among a plurality of environmental data, each containing the second 3D model data, which are stored in a database associated with geographic location information. Information processing system.

5. A first acquisition unit that acquires first three-dimensional model data generated by measuring the three-dimensional shape of a mobile object used by the user based on the user's operation, A second acquisition unit acquires second three-dimensional model data corresponding to the environment in which the moving object is moving, A synthesis processing unit that, within a virtual three-dimensional space, positions a moving object model defined by the first three-dimensional model data relative to an environment model defined by the second three-dimensional model data, while maintaining dimensional consistency with the respective real spaces, A movement control unit that moves the mobile object model in the virtual three-dimensional space based on user operations, A determination unit that determines the mobility of the mobile model, including whether the mobile model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile model and the environment model, A presentation processing unit that performs processing to present information indicating the determination result by the determination unit, It has a server that can communicate with multiple user terminals via a network, The server includes a data management unit that registers the second three-dimensional model data obtained from each user terminal in a database, associating it with location information of the place where the data was measured. The second acquisition unit acquires the second 3D model data registered by any user from the database. Information processing system.

6. Based on user operations, the data management unit manages the first 3D model data as private, accessible only to the user in question, and the second 3D model data as public, accessible to other users as well. The information processing system according to claim 5.

7. The data management unit, when it obtains multiple different 3D model data for the same location, performs a process to integrate these multiple 3D model data into a single 3D model data. The information processing system according to claim 5.

8. A first acquisition unit that acquires first three-dimensional model data generated by measuring the three-dimensional shape of a mobile object used by the user based on the user's operation, A second acquisition unit acquires second three-dimensional model data corresponding to the environment in which the moving object is moving, A synthesis processing unit that, within a virtual three-dimensional space, positions a moving object model defined by the first three-dimensional model data relative to an environment model defined by the second three-dimensional model data, while maintaining dimensional consistency with the respective real spaces, A movement control unit that moves the mobile object model in the virtual three-dimensional space based on user operations, A determination unit that determines the mobility of the mobile model, including whether the mobile model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile model and the environment model, The system includes a presentation processing unit that performs processing to present information indicating the determination result by the determination unit, The synthesis processing unit simultaneously arranges multiple mobile model corresponding to different users within the same environment model. The determination unit determines contact not only with the environment model but also with each other among the multiple mobile model entities. Information processing system.

9. The movement control unit sets motion characteristic parameters for the moving body model, including the minimum turning radius of the corresponding real-world moving body, the pivot axis based on the wheel arrangement, and / or the maximum inclination angle at which it can climb, and moves the moving body model in the virtual three-dimensional space under constraints based on the motion characteristic parameters. The information processing system according to claim 1.

10. The determination unit sets a bounding box and / or polygon mesh for the moving object model and detects the contact by performing an intersection determination with the environment model using the bounding box and / or polygon mesh. The information processing system according to claim 1.

11. When the contact is detected by the determination unit, the display processing unit performs a process to visually highlight the contact area in the moving object model and / or the environment model. The information processing system according to claim 1.

12. The system further includes a search processing unit that receives the setting of the starting point and / or destination point for the movement of the mobile model, and searches for and presents a path that the mobile model can traverse based on the size and / or motion characteristics of the mobile model. The information processing system according to claim 1.

13. If the determination unit determines that the mobile object model cannot pass through, the search processing unit performs the process of searching for and presenting alternative facilities or route information that the mobile object model can pass through. The information processing system according to claim 12.

14. A first acquisition unit that acquires first three-dimensional model data generated by measuring the three-dimensional shape of a mobile object used by the user based on the user's operation, A second acquisition unit acquires second three-dimensional model data corresponding to the environment in which the moving object is moving, A synthesis processing unit that, within a virtual three-dimensional space, positions a moving object model defined by the first three-dimensional model data relative to an environment model defined by the second three-dimensional model data, while maintaining dimensional consistency with the respective real spaces, A movement control unit that moves the mobile object model in the virtual three-dimensional space based on user operations, A determination unit that determines the mobility of the mobile model, including whether the mobile model can pass through and / or come into contact with a region within the environment model, based on the positional relationship between the mobile model and the environment model, A presentation processing unit that performs processing to present information indicating the determination result by the determination unit, The system includes a result management unit that generates simulation result data including the result of the determination by the determination unit or the movement history by the movement control unit, and stores the simulation result data in a server in association with the environment model. The second acquisition unit acquires the simulation result data from other users stored on the server, and the presentation processing unit presents the simulation result data from other users as reference information. Information processing system.

15. The system further includes a viewpoint control unit that controls the viewpoint when displaying the virtual three-dimensional space, The viewpoint control unit switches between a third-person viewpoint that provides an overview of the space including the mobile model, and a first-person viewpoint that reproduces the view of the user riding in the mobile model, based on user input. The information processing system according to claim 1.

16. The synthesis processing unit detects the floor surface in the environment model and automatically adjusts the height position of the mobile model so that the contact surface of the mobile model is in contact with the floor surface. The information processing system according to claim 1.

17. To operate at least one computer as an information processing system according to any one of claims 1 to 16. program.

Images