Simulation device, simulation system, simulation method, and simulation program

Abstract

A simulation device, a simulation system, a simulation method, and a simulation program are provided that enable easy verification of construction gauges. [Solution] The system is configured to be able to execute a real-time video display process that displays real-time video of the real space, a still image acquisition process that acquires still images of the real space, a rail identification process that identifies the positions of a pair of rails in the real-time video based on the still images, and an overlay display process that overlays a construction gauge model on the rails in the real-time video.

Description

[Technical Field]

[0001] The present invention relates to a simulation device, a simulation system, a simulation method, and a simulation program. [Background technology]

[0002] Conventionally, a construction gauge measurement system has been known that includes a point cloud image generation unit capable of generating a three-dimensional measurement image from a viewpoint that can recognize all or part of a building, etc. based on three-dimensional point cloud data; a track setting unit that determines data on tracks, etc. based on three-dimensional point cloud data obtained by three-dimensionally measuring tracks, etc. and buildings, etc. located along the tracks, etc.; and a three-dimensional model placement unit that places a three-dimensional construction gauge model of a vehicle having a length corresponding to the length of all or part of a building, etc. on the data on tracks, etc. (for example, Patent Document 1, etc.).

[0003] The construction gauge measurement system described in Patent Document 1 performs 3D measurements using a 3D measurement device and acquires 3D point cloud data of railway tracks, buildings, etc. in advance before processing the construction gauge measurement. The construction gauge measurement system described in Patent Document 1 then sets track data based on point cloud information corresponding to the railway tracks contained in the acquired 3D point cloud data. Specifically, by matching with a railway model stored in a database, the system extracts track information (point cloud data) contained in the 3D point cloud data and determines the track data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2024-098457 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional construction gauge measurement systems have the problem that the processing is complicated and time-consuming because they set track data based on 3D measurement images acquired from 3D point cloud data obtained by 3D measurement of tracks, etc. and buildings, etc. installed along the tracks, etc. Furthermore, as mentioned above, in order to set track data, it is necessary to create a track model in advance and store it in a database.

[0006] Furthermore, the structure gauge measurement system described in Patent Document 1 can only verify the structure gauge within the range where 3D measurements have been performed in advance, and because it displays a 3D structure gauge model on a 3D measurement image obtained from the 3D measurement, it is not possible to overlay the 3D structure gauge model on a video or image of the real space in real time. Therefore, the structure gauge measurement system described in Patent Document 1 leaves room for improvement in terms of operational convenience when verifying the structure gauge.

[0007] The present invention has been made in view of the above-mentioned problems, and has as its object to provide a simulation device, a simulation system, a simulation method, and a simulation program that enable easy verification of structure gauges. [Means for solving the problem]

[0008] The simulation device according to the present invention is characterized in that it is configured to be able to execute a real-time image display process for displaying a real-time image of real space, a still image acquisition process for acquiring a still image of the real space, a rail identification process for identifying the positions of a pair of rails in the real-time image based on the still image, and a superimposed display process for superimposing a construction gauge model on the rails in the real-time image.

[0009] The simulation device of the present invention may be configured to be able to execute, in the rail identification process, a rail extraction process that classifies the still image into rail areas where the rail exists and other areas, a rail coordinate acquisition process that thins out and acquires position coordinates of corresponding areas in the real-time video that correspond to the rail areas, and a rail position identification process that identifies the position of the rail in the real-time video based on the acquired position coordinates.

[0010] The simulation device according to the present invention may be configured to be able to execute, in the rail position identification process, a coordinate classification process that classifies the position coordinates of the corresponding area into a first group of position coordinates that constitute a first row and a second group of position coordinates that constitute a second row; a rail equally spaced point search process that searches for a plurality of equally spaced rail points that are positioned at predetermined intervals along the longitudinal direction of each row in each classified position coordinate group; and a rail approximate curve identification process that identifies an approximate curve of the identified plurality of equally spaced rail points.

[0011] The simulation device according to the present invention may, in the rail equally spaced point search process, search for a second position coordinate at a position that is a first search distance from a first position coordinate included in each position coordinate group, and if the second position coordinate is not found at the first search distance, gradually shorten the search distance from the first search distance to search for the second position coordinate, and search for a third position coordinate at a position that is the first search distance from the second position coordinate.

[0012] The simulation device according to the present invention is configured to be capable of performing a filtering process in the rail position identification process to exclude, from the plurality of identified equally spaced rail points, those whose coordinate values ​​in a direction along the height direction of the rail in the real-time video are less than a predetermined threshold, and may perform the rail approximate curve identification process after the filtering process.

[0013] The simulation device according to the present invention is configured to be able to execute, in the superimposed display process, a center line identification process that identifies a center line located midway between the approximation curve of equally spaced rail points of the first group of position coordinates and the approximation curve of equally spaced rail points of the second group of position coordinates, and a model display process that displays a plurality of the structure gauge models along the center line, and in the model display process, the structure gauge models may be displayed so that the height direction of the structure gauge models is along a direction perpendicular to both the extension direction of the center line and the opposing direction of the approximation curve of equally spaced rail points of the first group of position coordinates and the approximation curve of equally spaced rail points of the second group of position coordinates.

[0014] The simulation device according to the present invention may be configured to be able to execute a position coordinate group selection reception process that receives a selection of at least one of the first position coordinate group and the second position coordinate group, a position coordinate group division process that divides the selected position coordinate group into two columns, and a guardrail discrimination process that determines one of the two divided columns as a position coordinate group of a main rail and the other as a position coordinate group of a guardrail.

[0015] The simulation device according to the present invention may be configured to be capable of executing a model movement process for moving the structure gauge model superimposed on the real-time video along the opposing direction of the pair of rails.

[0016] The simulation system of the present invention comprises a camera and a simulation device, and is characterized in that the simulation device is configured to be able to execute a real-time image display process for displaying real-time images of real space captured by the camera, a still image acquisition process for acquiring still images of the real space captured by the camera, a rail identification process for identifying the positions of a pair of rails in the real-time image based on the still images, and a superimposed display process for superimposing a construction gauge model on the rails in the real-time image.

[0017] The simulation method according to the present invention is characterized in that a simulation device executes a real-time image display step of displaying a real-time image of real space, a still image acquisition step of acquiring a still image of the real space, a rail identification step of identifying the positions of a pair of rails in the real-time image based on the still image, and a superimposition display step of superimposing a structure gauge model on the rails in the real-time image.

[0018] The simulation program of the present invention is characterized in that it causes a simulation device to execute a real-time image display process for displaying a real-time image of real space, a still image acquisition process for acquiring a still image of the real space, a rail identification process for identifying the positions of a pair of rails in the real-time image based on the still image, and a superimposed display process for superimposing a structure gauge model on the rails in the real-time image. [Effects of the Invention]

[0019] According to the present invention, it is possible to provide a simulation device, a simulation system, a simulation method, and a simulation program that can easily verify structure gauges. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a block diagram illustrating a simulation system according to an embodiment of the present invention. [Figure 2] 1 is a schematic diagram showing a simulation device according to an embodiment of the present invention; [Figure 3] 1 is a schematic diagram illustrating a coordinate classification process and a rail equally-spaced point search process according to the present embodiment; [Figure 4] FIG. 10 is a diagram showing an example of a process for searching for equally spaced rail points according to the present embodiment; [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a filtering process according to the present embodiment. [Figure 6] FIG. 10 is a schematic diagram illustrating a rail approximate curve identification process according to the present embodiment; [Figure 7]FIG. 10 is a schematic diagram illustrating a midpoint identification process according to the present embodiment; [Figure 8] 8A and 8B are schematic diagrams illustrating a midpoint approximation curve specification process and a central equally spaced point setting process according to the present embodiment; [Figure 9] FIG. 10 is a schematic diagram illustrating a process for setting equally spaced points in the sky according to the present embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a simulation screen when using a rail separation function according to the present embodiment. [Figure 11] 10A and 10B are schematic diagrams illustrating a position coordinate group division process and a card rail discrimination process according to the present embodiment; [Figure 12] A flowchart showing an example of a simulation method according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] The best mode for carrying out the present invention will be described below with reference to the drawings. Note that the following embodiments do not limit the inventions according to the claims, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. Furthermore, the drawings are schematic diagrams in which emphasis, omission, and proportion adjustments have been made as appropriate to illustrate the present invention, and may differ from the actual shapes, positional relationships, and proportions.

[0022] [Configuration of the Simulation System According to the Present Embodiment] First, a simulation system 1 according to an embodiment of the present invention will be outlined with reference to Fig. 1. The simulation system 1 according to this embodiment is generally used to verify construction gauges at a site where a railway line including a pair of rails R (rail tracks) is laid. As shown in Fig. 1, the simulation system 1 according to this embodiment includes a simulation device 10 including a camera unit 40 that functions as a camera capable of photographing real space, and a control unit 50 that functions as a control device. That is, in this embodiment, the simulation system 1 can be composed of only the simulation device 10.

[0023] However, the present invention is not limited to this. The simulation system 1 may include a camera and a simulation device 10 that does not include a camera unit 40. The simulation system 1 may further include a storage device that can be connected to the simulation device 10 offline or online, such as an auxiliary storage device or a server.

[0024] In this embodiment, the simulation device 10 includes an input unit 20, a display unit 30, and a storage unit 60 in addition to a camera unit 40 and a control unit 50. In this embodiment, the simulation device 10 is a tablet terminal. However, this is not limited to this. The simulation device 10 may also be a smartphone, a laptop computer, or a desktop computer. Note that, when the simulation system 1 is composed of only the simulation device 10, or the simulation device 10 and a storage device, from the viewpoint of acquiring still images SI using the camera unit 40 of the simulation device 10, it is preferable that the simulation device 10 be a portable terminal such as a tablet terminal or a smartphone that can be easily carried around by the user on-site.

[0025] The display unit 30 has a display as a display device, and in addition to the screen display function normally required in the simulation device 10, displays, for example, a simulation screen 32 as shown in Fig. 2. In this embodiment, the display unit 30 is configured with a touch panel that functions as the input unit 20, and by operating the display unit 30, for example, the user can input operations such as the operation of a shooting button 32c (described later) and the selection of a position coordinate group (described later) to the simulation device 10, in addition to the information input function normally required in the simulation device 10.

[0026] 2, the simulation screen 32 has an AR display area 32a, a rail identification result display area 32b, a shooting button 32c, a setting list display area 32d, and a tool display area 32e. The AR display area 32a is configured to be able to display a real-time image LV of the real space captured by the camera unit 40. Furthermore, the AR display area 32a is configured to be able to superimpose a construction gauge model M on the real-time image LV, as will be described later.

[0027] The rail identification result display area 32b is configured to be able to display a still image SI captured of real space by the camera unit 40. Specifically, the rail identification result display area 32b displays an image that visualizes the rail area ra in the still image SI, as will be described later.

[0028] The capture button 32c is configured so that when the user selects or presses the capture button 32c, the camera unit 40 captures a still image SI. The setting list display area 32d is configured so that it can display various settings of the simulation device 10. The tool display area 32e displays various function tools used in the simulation.

[0029] Furthermore, when the simulation device 10 includes an input unit 20 in addition to the display unit 30, the input unit 20 may be configured with input devices such as a keyboard, a mouse, a touchpad, a joystick, a microphone, etc. By operating the input unit 20, the user can perform operations such as operating the photograph button 32c and selecting a position coordinate group.

[0030] The configurations of the input unit 20 and the display unit 30 are not limited to those described above, and any configuration having equivalent functions (for example, a display means or input means that can be used remotely) can be used instead of the input unit 20 and the display unit 30, and is not limited to this.

[0031] The camera unit 40 has a lens and an image sensor, and is configured to be able to capture images of the real space. The camera unit 40 is configured to be able to capture real-time images LV of the real space and still images SI of the real space. In this embodiment, the camera unit 40 always captures the real-time images LV while the simulation screen 32 is displayed on the display unit 30, and captures still images SI when the user operates the capture button 32c on the simulation screen 32. The still images SI captured by the camera unit 40 are recorded in the storage unit 60.

[0032] On the other hand, the real-time video LV captured by the camera unit 40 is not recorded in the storage unit 60. In other words, the real-time video LV is simply a video image from the image sensor of the camera unit 40 displayed in the AR display area 32a of the simulation screen 32 of the display unit 30, which is a so-called live view function. However, this is not limited to this. The camera unit 40 may capture and record the real-time video LV as video data.

[0033] The camera unit 40 also has a distance sensor (so-called LiDAR) that can acquire information about three-dimensional space as three-dimensional point cloud data by emitting laser light. The three-dimensional point cloud data is composed of a plurality of three-dimensional point data. Each three-dimensional point data has position coordinates (coordinate values) in a three-dimensional coordinate system that is composed of plane coordinates and height coordinates orthogonal to the plane coordinates. In this embodiment, the distance sensor acquires three-dimensional space data of a corresponding area in a still image SI in the real-time video LV that corresponds to a rail area ra (described later) as a point cloud. In other words, the distance sensor measures the distance to the rail R in real space, which is the object of measurement, and the shape of the rail R.

[0034] The control unit 50 is configured, for example, by an integrated processing unit having a CPU and a GPU. The control unit 50 may also have an NPU (neural processing unit) in order to speed up AI processing using a machine learning model 61 (described later). In this embodiment, the control unit 50 includes an image acquisition unit 52, a rail identification unit 54, a display control unit 56, and an obstacle determination unit 58.

[0035] The image acquisition unit 52 is configured to be able to execute a real-time video display process (S1 in FIG. 12) for displaying a real-time video LV of the real space. In this embodiment, the image acquisition unit 52 is configured to acquire a real-time video LV captured by the camera unit 40 of the real space, and display it on the display unit 30. Specifically, the image acquisition unit 52 displays the acquired real-time video LV in the AR display area 32a of the simulation screen 32 of the display unit 30, as shown in FIG.

[0036] Furthermore, the image acquisition unit 52 is configured to be able to execute a still image acquisition process (S2 in FIG. 12) for acquiring a still image SI obtained by capturing a real space. In this embodiment, the image acquisition unit 52 is configured so that the camera unit 40 acquires a still image SI obtained by capturing a real space when the user performs a capturing operation on the simulation screen 32. Note that the image acquisition unit 52 may acquire the still image SI by cutting out real-time video LV captured by the camera unit 40, or may acquire a screenshot of the real-time video LV as the still image SI. Furthermore, when the camera unit 40 captures a moving image, the still image SI may be acquired by cutting out the moving image, or may acquire a screenshot of the moving image as the still image SI.

[0037] The rail identification unit 54 is configured to be able to execute a rail identification process (S3 in FIG. 12) for identifying the positions of the pair of rails R in the real-time video LV based on the still image SI acquired by the image acquisition unit 52.

[0038] The rail identification unit 54 according to this embodiment is configured to be able to execute a rail extraction process (S4 in FIG. 12) in the rail identification process, which classifies the still image SI into a rail area ra where the rail R exists and other areas. Specifically, in the rail extraction process, the rail identification unit 54 identifies the rail R in the still image SI based on the machine learning model 61 in the storage unit 60. Then, the rail identification unit 54 classifies the still image SI into the rail area ra where the identified rail R exists and other areas.

[0039] More specifically, in the rail extraction process, the rail identification unit 54 divides the still image SI into pixel units through a segmentation process, and labels each pixel as either "rail R" or "non-rail R," thereby classifying the still image SI into rail areas ra and other areas.

[0040] 2, the rail identification unit 54 displays the still image SI acquired by the image acquisition unit 52 in the rail identification result display area 32b on the simulation screen 32 of the display unit 30. Furthermore, the rail identification unit 54 is configured to superimpose a specific color or pattern on the rail area ra in the still image SI so that the user can identify that area in the still image SI as the rail area ra.

[0041] Furthermore, the rail identification unit 54 is configured to be able to execute a rail coordinate acquisition process (S5 in FIG. 12) in the rail identification process to acquire position coordinates of a corresponding area corresponding to the rail area ra in the real-time video LV. Specifically, the rail identification unit 54 causes the distance sensor of the camera unit 40 to acquire three-dimensional point cloud data of the corresponding area corresponding to the rail area ra in the real-time video LV. Then, the rail identification unit 54 acquires the position coordinates of the corresponding area corresponding to the rail area ra in the real-time video LV from the three-dimensional point cloud data acquired by the distance sensor.

[0042] In this embodiment, the rail identification unit 54 is configured to thin out and acquire position coordinates of a corresponding area corresponding to the rail area ra in the real-time video LV during the rail coordinate identification process. That is, during the rail coordinate identification process, the rail identification unit 54 is configured to acquire position coordinates for a predetermined number of pixels out of all pixels, rather than acquiring position coordinates of all pixels in the corresponding area corresponding to the rail area ra in the real-time video LV.

[0043] For example, the rail identification unit 54 acquires the position coordinates of one pixel out of every six pixels in a corresponding area corresponding to the rail area ra in the real-time video LV. However, this is not limited to this. The simulation device 10 according to this embodiment is configured so that the user can change the predetermined number of pixels in the setting list display area 32d of the simulation screen 32. Furthermore, by changing the predetermined number of pixels, the rail identification unit 54 can also acquire the position coordinates of all pixels in the corresponding area in the rail coordinate identification process.

[0044] In addition, as shown in Figures 2 and 10, the rail identification unit 54 is configured to be able to superimpose a point cloud representing the three-dimensional point cloud data acquired by the distance sensor of the camera unit 40 onto the real-time video LV displayed in the AR display area 32a of the simulation screen 32 of the display unit 30.

[0045] Furthermore, the rail identification unit 54 is configured to be able to execute a rail position identification process (S6 in FIG. 12) that identifies the position of the rail R in the real-time video LV based on the acquired position coordinates in the rail identification process. As shown in FIG. 3, the rail identification unit 54 is configured to be able to execute a coordinate classification process (S61 in FIG. 12) that classifies (clusters) the position coordinates of the corresponding area into a first position coordinate group G1 that constitutes a first column and a second position coordinate group G2 that constitutes a second column in the rail position identification process. The coordinate classification process is executed based on the distance between each position coordinate and a threshold value for that distance.

[0046] In addition, in the rail position identification process, the rail identification unit 54 is configured to be able to execute a rail equally spaced point search process (S62 in Figure 12) that searches for multiple equally spaced rail points Pr located at predetermined intervals along the longitudinal direction of each row in each position coordinate group classified in the coordinate classification process, as shown in Figure 3.

[0047] In this embodiment, the equally spaced rail points Pr are point data included in each position coordinate group, i.e., point cloud data. Therefore, in the equally spaced rail points search process, the rail identification unit 54 searches for multiple equally spaced rail points Pr based on the position coordinates of each point data included in each classified position coordinate group (each point cloud data). In this embodiment, the rail identification unit 54 does not search for all point data included in each position coordinate group as the equally spaced rail points Pr, but instead searches only point data located above the rail R in the real-time video LV, i.e., near the top surface of the rail R. The rail identification unit 54 determines whether each point data corresponds to point data located near the top surface of the rail R based on the coordinate value of the height direction of the rail R among the coordinate values ​​of each point data. For example, the rail identification unit 54 determines that point data with a larger coordinate value in the height direction of the rail R corresponds to point data located near the top surface of the rail R from multiple point data with the same or similar coordinate values ​​in the extension direction (longitudinal direction) of the rail R.

[0048] Furthermore, since the longitudinal direction of each row of position coordinate groups is along the longitudinal direction of the rail R in the real-time video LV, it can be said that the multiple equally spaced rail points Pr, which are located at a predetermined interval, are located along the longitudinal direction of the rail R.

[0049] In the process of searching for equally spaced rail points, the rail identification unit 54 searches for a second position coordinate P2 at a position that is a first search distance d1 from a first position coordinate P1 included in each position coordinate group. The first position coordinate P1 is the position coordinate of point data that is located at one end of the string of position coordinate groups. In order to absorb measurement errors, the first position coordinate P1 is not the position coordinate of the point data that is located at the extreme end of one end of the string, but the position coordinate of point data that is located a predetermined distance away from the position coordinate of the point data that is located at the extreme end of one end of the string.

[0050] The search direction for the plurality of equally spaced rail points Pr is the direction from one end of the row of position coordinate groups to the other end, along the longitudinal direction of the row, and is determined based on the direction in which the point data included in the position coordinate groups continue.

[0051] In this embodiment, the "search distance position" is not a single point, but rather spreads out in a fan shape over a predetermined range in the forward and backward directions from a position that is a search distance d away from a specific position coordinate, as shown in FIG.

[0052] Furthermore, if the second position coordinate P2 is not found at the first search distance d1, the rail identification unit 54 searches for the second position coordinate P2 by gradually shortening the search distance d from the first search distance d1. Specifically, if the second position coordinate P2 is not found at the first search distance d1 from the first position coordinate P1, the rail identification unit 54 resets the search distance d to a second search distance d2 that is shorter than the first search distance d1, and searches again to see if the second position coordinate P2 is found at the second search distance d2 from the first position coordinate P1. Furthermore, if the second position coordinate P2 is not found at the second search distance d2 from the first position coordinate P1, the rail identification unit 54 resets the search distance d to a third search distance d3 that is shorter than the second search distance d2, and searches again to see if the second position coordinate P2 is found at the third search distance d3 from the first position coordinate P1.

[0053] In this embodiment, the user resets the search distance d by operating the setting list display area 32d of the simulation screen 32. However, this is not limitative. The rail identification unit 54 may automatically reset the search distance d and search for the position coordinates again.

[0054] After finding the second position coordinate P2, the rail identification unit 54 searches for a third position coordinate P3 at a position that is a first search distance d1 from the second position coordinate P2. Specifically, if the second position coordinate P2 is found at a position that is the first search distance d1 from the first position coordinate P1, the rail identification unit 54 searches for the third position coordinate P3 from the second position coordinate P2 while keeping the search distance d at the first search distance d1. On the other hand, if the second position coordinate P2 is not found at a position that is the first search distance d1 from the first position coordinate P1, that is, if the search distance d is set shorter than the first search distance d1 and the second position coordinate P2 is found, the rail identification unit 54 resets the shortened search distance d to the first search distance d1 and searches for the third position coordinate P3 from the second position coordinate P2.

[0055] For example, as shown in FIG. 4(a), the rail identification unit 54 searches for a second position coordinate P2 at a position that is a first search distance d1 from the first position coordinate P1. In the example shown in FIG. 4, since no position coordinate is found at a position that is the first search distance d1 from the first position coordinate P1, the rail identification unit 54 resets the search distance d to a second search distance d2 and searches again for the second position coordinate P2. In the example shown in FIG. 4, since no position coordinate is found at a position that is the second search distance d2 from the first position coordinate P1, the rail identification unit 54 further resets the search distance d to a third search distance d3 and searches again for the second position coordinate P2. Thereafter, when the second position coordinate P2 is found at a position that is the third search distance d3 from the first position coordinate P1, the rail identification unit 54 searches for a third position coordinate P3 at a position that is the first search distance d1 from the found second position coordinate P2, as shown in FIG. 4(b).

[0056] In the rail equally spaced point search process, the rail identification unit 54 repeats the above-mentioned process to search for position coordinates from one end of the row of position coordinate groups to the other end, until the nth position coordinate (n is a positive integer) is no longer found.

[0057] In addition, in the rail position identification process, the rail identification unit 54 further performs a filtering process (S63 in Figure 12) to exclude, from the multiple equally spaced rail points Pr identified in the equally spaced rail point search process, those equally spaced rail points Pr whose coordinate values ​​in the direction along the height direction of the rail R in the real-time video LV are less than a predetermined threshold value.

[0058] In this embodiment, the predetermined threshold for the filtering process is determined based on the proportion of valid equally-spaced rail points Pr to be retained. For example, when 70% of the plurality of equally-spaced rail points Pr identified in the equally-spaced rail point search process are to be retained, the rail identification unit 54 sorts the plurality of equally-spaced rail points Pr in descending order of coordinate value along the height direction of the rail R in the real-time video LV and excludes the equally-spaced rail points Pr with the lowest coordinate values. In this embodiment, the rail identification unit 54 performs the filtering process for each column on which the equally-spaced rail point search process has been performed. Therefore, the number of equally-spaced rail points Pr excluded from the plurality of equally-spaced rail points Pr identified from the columns of the first position coordinate group G1 is the same as the number of equally-spaced rail points Pr excluded from the plurality of equally-spaced rail points Pr identified from the columns of the second position coordinate group G2. However, this is not a limitation. The predetermined threshold does not have to be determined based on the proportion of valid equally-spaced rail points Pr to be retained. Furthermore, the number of equally-spaced rail points Pr excluded from the plurality of equally-spaced rail points Pr identified from each column may differ for each column.

[0059] The proportion of valid equally spaced rail points Pr to be left, and thus the predetermined threshold value, can be set appropriately based on the shooting position of the camera unit 40 (position of the camera unit 40) when capturing the real-time video LV and still image SI.

[0060] For example, in the filtering process, in the example shown in Figure 5, the rail identification unit 54 excludes, from among the multiple equally spaced rail points Pr, the equally spaced rail points Pr1 to Pr6 whose coordinate values ​​in the direction along the height direction of the rail R in the real-time video LV are less than a predetermined threshold value.

[0061] Furthermore, as shown in Fig. 6(a), the rail identification unit 54 is configured to be able to execute a rail approximation curve identification process (S63 in Fig. 12) in the rail position identification process to identify an approximation curve (rail approximation curve RAC in this embodiment) of the plurality of equally spaced rail points Pr identified in the equally spaced rail point search process. The rail identification unit 54 executes the rail approximation curve identification process after the filtering process. In this embodiment, the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 is defined as a first rail approximation curve RAC1, and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2 is defined as a second rail approximation curve RAC2.

[0062] Furthermore, the rail identification unit 54 sets equally spaced virtual rail points Pi at predetermined intervals on the identified rail approximate curve RAC, as shown in Fig. 6(b). In this embodiment, the predetermined intervals at which the equally spaced virtual rail points Pi are set are the model display intervals dm when displaying multiple construction gauge models M, as will be described later.

[0063] Furthermore, the rail identification unit 54 is configured to be able to execute a position coordinate group selection reception process that receives the selection of at least one of the first position coordinate group G1 and the second position coordinate group G2, a position coordinate group division process that divides the selected position coordinate group into two columns, and a guardrail discrimination process that determines one of the two columns as the position coordinate group Gmr of the main line rail and the other as the position coordinate group Ggr of the guardrail (derailment prevention guard, guard rail).

[0064] The simulation screen 32 in Fig. 10 shows an example of a track in which a guardrail GR is laid inside a main rail R, captured by the camera unit 40. The user can use the rail separation function of the simulation device 10 by operating the tool display area 32e on the simulation screen 32. When the user wishes to separate at least one of a first position coordinate group G1 and a second position coordinate group G2, which include the main rail and the guardrail GR, into a main rail position coordinate group Gmr and a guardrail position coordinate group Ggr, the user selects a point cloud that represents 3D point cloud data corresponding to at least one of the first position coordinate group G1 and the second position coordinate group G2 that the user wishes to separate, which is displayed in the AR display area 32a.

[0065] If the position coordinate group corresponding to the rail R laid adjacent to the guardrail GR is the second position coordinate group G2, the user selects the second position coordinate group G2. Next, the rail identification unit 54 accepts the selection of the second position coordinate group G2 in a position coordinate group selection receiving process. Then, in a position coordinate group division process, the rail identification unit 54 identifies a center line along the longitudinal direction of the row of the second position coordinate group G2, as shown in FIG. 11(a), and divides it into two rows (position coordinate groups) as shown in FIG. 11(b).

[0066] Then, in the guardrail discrimination process, the rail identification unit 54 discriminates one of the two divided rows as the group of position coordinates Gmr of the main rail and the other as the group of position coordinates Ggr of the guardrail, based on information about the gauge of the main rail (for example, stored in the storage unit 60). Specifically, the rail identification unit 54 identifies the center line of each divided row and the information about the gauge of the main rail, and discriminates based on the distance between the center line of the divided row and the center line of the first group of position coordinates G1. Note that the above-described series of processes can also be performed when an image of a track in which a guardrail GR is laid outside a rail R of the main rail is captured.

[0067] Similarly, the rail identification unit 54 is configured to be able to execute a position coordinate group selection reception process that receives the selection of at least one of the first position coordinate group G1 and the second position coordinate group G2, a position coordinate group division process that divides the selected position coordinate group into two columns, and a rail discrimination process that determines one of the two columns as a standard gauge (medium gauge) position coordinate group and the other as a narrow gauge position coordinate group.

[0068] In the rail discrimination process, the rail identification unit 54 discriminates one of the two divided trains as a group of standard gauge position coordinates and the other as a group of narrow gauge position coordinates, based on information about the standard gauge and narrow gauge track gauges (for example, stored in the storage unit 60). Specifically, the rail identification unit 54 makes the discrimination based on information about the standard gauge and narrow gauge track gauges and the distance between the center lines of each train.

[0069] In addition, the rail identification unit 54 again executes the rail equally spaced point search process and the rail approximate curve identification process on the two position coordinate groups (the main line rail position coordinate group Gmr and the guard rail position coordinate group Ggr, or the standard gauge position coordinate group and the narrow gauge position coordinate group) divided in the position coordinate division process.

[0070] The display control unit 56 is configured to be able to execute a superimposed display process (S7 in FIG. 12) for superimposing and displaying the construction gauge model M on the rails R in the real-time video LV. Specifically, the display control unit 56 superimposes and displays the construction gauge model M on the rails R in the real-time video LV displayed in the AR display area 32a in the simulation screen 32 displayed on the display unit 30.

[0071] The display control unit 56 according to this embodiment is configured to be able to execute a center line identification process (S8 in FIG. 12) in the superimposed display process to identify a center line lc located midway between the approximation curve (first rail approximation curve RAC1) of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve (second rail approximation curve RAC2) of the equally spaced rail points Pr of the second position coordinate group G2.

[0072] 7 and 8(a), the display control unit 56 first executes an intermediate point identification process to identify intermediate points Pm between each of the equally spaced virtual rail points Pi on the first rail approximation curve RAC1 and each of the equally spaced virtual rail points Pi on the second rail approximation curve RAC2 that correspond to each of the equally spaced virtual rail points Pi on the first rail approximation curve RAC1. In this embodiment, the "equally spaced virtual rail points on the second rail approximation curve that correspond to each of the equally spaced virtual rail points of the first position coordinate group" refers to each of the equally spaced virtual rail points Pi on the second rail approximation curve RAC2 that is closest to each of the equally spaced virtual rail points Pi on the first rail approximation curve RAC1.

[0073] Then, the display control unit 56 executes a midpoint approximation curve specification process to specify an approximation curve (in this embodiment, a center line lc) of the specified midpoints Pm, as shown in Fig. 8(b). Next, the display control unit 56 executes a central equally spaced point setting process to set central equally spaced points Pc at predetermined intervals on the center line lc, as shown in Fig. 8(c). In this embodiment, the predetermined intervals at which the central equally spaced points Pc are set are the model display interval dm.

[0074] The display control unit 56 then executes a process for setting equally spaced overhead points Pa at positions above each of the set equally spaced central points Pc, as shown in Fig. 9. The positions above the equally spaced central points Pc are positions spaced a predetermined distance from the equally spaced overhead points Pc in a direction perpendicular to both the extension direction of the center line lc and the opposing direction of the first rail approximation curve RAC1 and the second rail approximation curve RAC2. In this embodiment, the equally spaced overhead points Pa function as positioning points when positioning the center of the structure gauge model M, as will be described later, and therefore the predetermined distance is determined based on the distance from the bottom end of the structure gauge model M to the center of the structure gauge model M in the height direction. Note that the intermediate points Pm, the equally spaced central points Pc, and the equally spaced overhead points Pa are imaginary points.

[0075] Furthermore, the display control unit 56 is configured to be able to execute a model display process (S9 in Fig. 12) in the superimposed display process to display multiple structure gauge models M along the center line lc. Specifically, in the model display process, the display control unit 56 displays the structure gauge model M so that the height direction of the structure gauge model M is along a direction perpendicular to both the extension direction of the center line lc and the opposing direction of the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 (first rail approximation curve RAC1) and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2 (second rail approximation curve RAC2).

[0076] The direction can be specified by the vector direction from the central equally spaced points Pc toward the aerial equally spaced points Pa located above the central equally spaced points Pc. The display control unit 56 also displays multiple construction gauge models M along the center line lc so that the center of the depth direction and the center of the height direction of each construction gauge model M coincides with the aerial equally spaced points Pa. In the example of the simulation screen 32 shown in Fig. 2, 17 construction gauge models M are displayed on the rail R in the real-time image LV of the AR display area 32a.

[0077] In other words, the display control unit 56 displays the construction gauge model M along a direction perpendicular to both the extension direction of the pair of rails R and the opposing direction of the pair of rails R in the real-time video LV. That is, the construction gauge model M is displayed vertically and superimposed on the pair of rails R in the real-time video LV.

[0078] The display control unit 56 reads out the determination point cloud data 63 from the storage unit 60, and superimposes and displays the structure gauge model M based on the determination point cloud data 63. Specifically, in this embodiment, the structure gauge model M is a frame-shaped 3D model indicating the structure gauge. The display control unit 56 sorts the multiple point data included in the read determination point cloud data 63 so that they form the frame shape of the structure gauge model M, and displays the structure gauge model M by connecting the sorted point data with lines. With this configuration, the simulation device 10 can superimpose and display the structure gauge model M on the real-time video LV at the same scale as the rail R in the real-time video LV.

[0079] The frame of the 3D model indicates the construction gauge. In addition, obstacle ranges are set and displayed inside and outside the frame of the 3D model to determine whether obstacles along the tracks are located inside the construction gauge.

[0080] The display control unit 56 is configured to be able to enlarge the obstruction range of the structure gauge model M superimposed on the real-time video LV to the inside and outside of the structure gauge model M. Specifically, when the user inputs or changes information on the curve radius of the rail R and information on the slack (expansion dimension of the gauge) of the rail R in the tool display area 32e of the simulation screen 32, the display control unit 56 enlarges and displays the obstruction range of the structure gauge model M based on the information. Furthermore, when the rail R is canted at the position of the rail R in the real-time video LV identified by the rail identification unit 54, the display control unit 56 enlarges and displays the obstruction range of the structure gauge model M based on the amount of displacement due to the cant.

[0081] The display control unit 56 is configured to be able to execute a model movement process for moving the construction gauge model M superimposed on the real-time video LV along the opposing direction of the pair of rails R. Specifically, when the user inputs an operation to move the construction gauge model M along the opposing direction of the pair of rails R in the tool display area 32e of the simulation screen 32, the display control unit 56 moves and displays the construction gauge model M superimposed on the real-time video LV based on the movement amount input in the operation.

[0082] The obstacle determination unit 58 is configured to be able to execute an obstacle determination process that determines whether or not an obstacle around the structure gauge model M (along the tracks) is located inside the structure gauge, in other words, whether or not there is a possibility of contact between the obstacle and the railway vehicle. In this embodiment, the obstacle determination process employs an intersection number determination method as an obstacle determination algorithm. Specifically, the system identifies the position coordinates of the frame line and obstacle range of the structure gauge model M superimposed on the real-time video LV, as well as the position coordinates of the obstacle, and determines whether or not the obstacle is located inside the frame line or obstacle range of the structure gauge model M.

[0083] When the display control unit 56 executes the model movement process, the obstacle determination unit 58 executes the obstacle determination process based on the position after the movement of the structure gauge model M. When the display control unit 56 expands the obstacle range of the structure gauge model M, the obstacle determination unit 58 executes the obstacle determination process based on the expanded structure gauge model M.

[0084] The storage unit 60 has a storage medium such as an eMMC, HDD, or SSD, and stores various data in a readable and writable manner. As shown in Fig. 1, the storage unit 60 stores a machine learning model 61, determination point cloud data 63, and a simulation program 65. The storage unit 60 also stores programs required for controlling each part of the simulation device 10.

[0085] The machine learning model 61 is a machine learning model trained using still images SI including rails R as training data. The determination point cloud data 63 is point cloud data for displaying the construction gauge model M, and includes multiple point data. Note that the machine learning model 61 and the determination point cloud data 63 may be stored in a server or another device, instead of the storage unit 60.

[0086] The simulation program 65 causes the simulation device 10 to execute a real-time image display process for displaying a real-time image LV of the real space, a still image acquisition process for acquiring a still image SI photographed of the real space, a rail identification process for identifying the positions of a pair of rails R in the real-time image LV based on the still image SI, and a superimposed display process for superimposing a construction gauge model M on the rails R in the real-time image LV.

[0087] [Description of Simulation Method According to This Embodiment] A simulation method using the simulation device 10 according to this embodiment will be described with reference to Fig. 12. In the simulation method according to this embodiment, the simulation device 10 executes a real-time video display step (S1 in Fig. 12) of displaying a real-time video LV of real space, a still image acquisition step (S2 in Fig. 12) of acquiring a still image SI of real space, a rail identification step (S3 in Fig. 12) of identifying the positions of a pair of rails R in the real-time video LV based on the still image SI, and a superimposition display step (S7 in Fig. 12) of superimposing and displaying a construction gauge model M on the rails R in the real-time video LV. Note that the specific processing executed by the simulation device 10 in each step has been described in detail in the description of each unit included in the control unit 50 of the simulation device 10, and therefore description thereof will be omitted.

[0088] [Advantages of the simulation device, simulation system, simulation method, and simulation program according to the present embodiment] As described above, the simulation device 10 according to this embodiment is configured to be able to execute a real-time video display process for displaying a real-time video LV of the real space, a still image acquisition process for acquiring a still image SI photographed of the real space, a rail identification process for identifying the positions of a pair of rails R in the real-time video LV based on the still image SI, and a superimposed display process for superimposing a construction gauge model M on the rails R in the real-time video LV.

[0089] The simulation device 10 according to the present embodiment, having such a configuration, can identify the positions of a pair of rails R in the real-time video LV based on the still image SI and superimpose the structure gauge model M on the rails R in the real-time video LV, thereby eliminating the need to perform three-dimensional measurements of the rails, etc. and buildings, etc., installed along the rails, etc., in advance, as in the structure gauge measurement system described in Patent Document 1. This allows the structure gauge to be verified immediately on the spot, thereby providing the advantage of easily verifying the structure gauge. Furthermore, the structure gauge measurement system described in Patent Document 1 requires three-dimensional measurements to be performed in advance, so if a measurement omission is discovered during verification of the structure gauge, the three-dimensional measurements must be performed again, which takes time before verification can begin. However, the simulation device 10 according to the present embodiment can capture a still image SI of the real space including the rails R at the location where the structure gauge is to be verified, and superimpose the structure gauge model M on the real-time video LV for verification, thereby providing the advantage of shortening the time before verification can be started compared to conventional methods.

[0090] Furthermore, the simulation device 10 according to this embodiment is configured to be able to execute the following rail identification processing steps: a rail extraction processing step for classifying the still image SI into rail areas ra where the rail R exists and other areas; a rail coordinate acquisition processing step for thinning out and acquiring position coordinates of corresponding areas in the real-time video LV that correspond to the rail areas ra; and a rail position identification processing step for identifying the position of the rail R in the real-time video LV based on the acquired position coordinates. This configuration allows the position of the rail R in the real-time video LV to be identified by partially acquiring only the position coordinates of corresponding areas in the real-time video LV that correspond to the rail areas ra, which further provides the advantage that the total amount of position coordinate data to be processed is smaller than in the past, thereby shortening the time required for data processing.

[0091] In addition, by thinning out and acquiring the position coordinates of the corresponding areas corresponding to the rail area ra in the real-time video LV, the total amount of position coordinate data to be processed can be reduced compared to acquiring all of the position coordinates of the corresponding areas corresponding to the rail area ra in the real-time video LV, and the time required for data processing can be further shortened. Furthermore, since the total amount of data to be processed is reduced, processing is possible even with a control unit 50 having low processing power, making it possible to adopt an inexpensive control unit 50, which has the advantage of reducing the cost of the simulation device 10.

[0092] Furthermore, the simulation device 10 according to this embodiment is configured to be able to execute the following rail position identification process: a coordinate classification process that classifies the position coordinates of the corresponding area into a first position coordinate group G1 constituting a first row and a second position coordinate group G2 constituting a second row; a rail equally spaced point search process that searches for a plurality of equally spaced rail points Pr located at predetermined intervals along the longitudinal direction of each classified position coordinate group; and a rail approximation curve identification process that identifies an approximation curve (rail approximation curve RAC in this embodiment) of the identified plurality of equally spaced rail points Pr. This configuration has the advantage of improving the accuracy of identifying the position of the rail R in the real-time video LV. Furthermore, identifying the approximation curve of the plurality of equally spaced rail points Pr has the further advantage of accurately identifying the curve, cant, etc. of the rail R. Furthermore, improving the accuracy of identifying the position of the rail R in the real-time video LV allows the construction gauge model M to be accurately superimposed on the rail R in the real-time video LV, thereby improving the accuracy of verifying the construction gauge.

[0093] Furthermore, in the process of searching for equally spaced rail points, the simulation device 10 according to this embodiment searches for a second position coordinate P2 at a position that is a first search distance d1 from a first position coordinate P1 included in each position coordinate group. If the second position coordinate P2 is not found at the first search distance d1, the search distance d is gradually reduced from the first search distance d1 to search for the second position coordinate P2, and then searches for a third position coordinate P3 at a position that is the first search distance d1 from the second position coordinate P2. This configuration allows the simulation device 10 to search for equally spaced rail points Pr as a single sequence, even if the sequence of position coordinate groups is interrupted due to weeds or the like beside the rail R, and thus the sequence of position coordinate groups is interrupted. This further advantageously improves the accuracy of searching for the equally spaced rail points Pr and, ultimately, the accuracy of identifying the rail approximate curve RAC. Furthermore, by improving the accuracy of identifying the rail approximation curve RAC, it becomes possible to accurately identify the position of the rail R in the real-time video LV, and the construction gauge model M can be accurately superimposed on the rail R in the real-time video LV, thereby improving the accuracy of verifying the construction gauge.

[0094] Furthermore, the simulation device 10 according to this embodiment is configured to be able to perform a filtering process in the rail position identification process to exclude, from among the identified plurality of equally spaced rail points Pr, those whose coordinate values ​​in the direction along the height direction of the rail R in the real-time video LV are less than a predetermined threshold, and then executes a rail approximation curve identification process after the filtering process. This configuration has the additional advantage of preventing a decrease in the identification accuracy of the rail approximation curve RAC due to the influence of the shooting angle and shooting position of the camera unit 40.

[0095] Furthermore, the simulation device 10 according to this embodiment is configured to be able to execute a center line identification process for identifying a center line lc located midway between the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2 in the superimposed display process, and a model display process for displaying multiple structure gauge models M along the center line lc. In the model display process, the structure gauge model M is displayed so that its height direction is aligned with both the extension direction of the center line lc and the opposing direction of the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2. With this configuration, multiple structure gauge models M are displayed along the extension direction of the rail R and are superimposed and displayed so as to be perpendicular to the rail R, which has the further advantage that the structure gauge model M is accurately displayed on the rail R in the real-time video LV even when the rail R is curved, has a cant, or is laid on an uphill or downhill slope. Furthermore, the construction gauge model M is accurately displayed on the rail R in the real-time video LV according to the rail R installation status, which can improve the accuracy of construction gauge verification.

[0096] Furthermore, the simulation device 10 according to this embodiment is configured to be able to execute a position coordinate group selection receiving process that receives the selection of at least one of the first position coordinate group G1 and the second position coordinate group G2, a position coordinate group division process that divides the selected position coordinate group into two columns, and a guardrail discrimination process that identifies one of the two columns as the main rail position coordinate group Gmr and the other as the guardrail position coordinate group Ggr. This configuration offers the additional advantage that, even if a guardrail GR is provided on at least one of the outer and inner sides of a pair of rails R, the pair of rails R, i.e., the main rails, can be discriminated and the construction gauge model M can be accurately displayed on the main rail in the real-time video LV. Furthermore, accurate display of the construction gauge model M on the main rail in the real-time video LV, even if a guardrail GR is provided, can improve the accuracy of verification of the construction gauge.

[0097] The simulation device 10 according to this embodiment is also configured to perform a rail discrimination process that discriminates one of the two divided tracks as a standard-gauge position coordinate group and the other as a narrow-gauge position coordinate group. This configuration allows for support for special rails such as triple-rail and quadruple-rail systems, and has the added advantage of being able to discriminate between a pair of standard-gauge rails R and accurately display a construction gauge model M on the standard-gauge rail R in the real-time video LV. Accurately displaying the construction gauge model M on the standard-gauge rail R in the real-time video LV, even for triple-rail and quadruple-rail systems, can also improve the accuracy of construction gauge verification. Furthermore, by preparing a construction gauge model M corresponding to the standard gauge and a construction gauge model M corresponding to the narrow gauge, when verifying the construction gauge for the standard gauge, the construction gauge model M corresponding to the standard gauge can be accurately displayed on the standard-gauge rail R in the real-time video LV, and when verifying the construction gauge for the narrow gauge, the construction gauge model M corresponding to the narrow gauge can be accurately displayed on the narrow-gauge rail R in the real-time video LV.

[0098] Furthermore, the simulation device 10 according to this embodiment is configured to be able to execute a model movement process for moving the construction gauge model M superimposed on the real-time video LV along the opposing direction of the pair of rails R. This configuration has the additional advantage that, when the position of the construction gauge model M superimposed on the real-time video LV is slightly deviated from the rails R, the display position of the construction gauge model M can be adjusted.

[0099] [Variations] Although the preferred embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope described in the above-described embodiments. Various modifications and improvements can be made to the above-described embodiments.

[0100] For example, in the above-described embodiment, the simulation device 10 has been described as being configured to execute a rail extraction process that classifies the still image SI into rail areas ra where the rail R exists and other areas in the rail identification process, a rail coordinate acquisition process that thins out and acquires position coordinates of corresponding areas in the real-time video LV that correspond to the rail areas ra, and a rail position identification process that identifies the position of the rail R in the real-time video LV based on the acquired position coordinates, but this is not limiting. In the rail coordinate acquisition process, the simulation device 10 does not need to thin out and acquire position coordinates of corresponding areas in the real-time video LV that correspond to the rail areas ra. In other words, in the rail coordinate acquisition process, the simulation device 10 may acquire all position coordinates of corresponding areas in the real-time video LV that correspond to the rail areas ra.

[0101] In the above-described embodiment, the simulation device 10 has been described as being configured to execute, in the rail position identification process, a coordinate classification process that classifies the position coordinates of the corresponding area into a first position coordinate group G1 that constitutes a first row and a second position coordinate group G2 that constitutes a second row, a rail equally spaced point search process that searches for a plurality of equally spaced rail points Pr that are positioned at predetermined intervals along the longitudinal direction of each row in each classified position coordinate group, and a rail approximate curve identification process that identifies an approximate curve of the identified plurality of equally spaced rail points Pr. However, the simulation device 10 does not necessarily have to execute the coordinate classification process in the rail position identification process. For example, in the rail extraction process, simulation device 10 may classify still image SI into a first rail region ra1 where a first rail R1 constituting a pair of rails R exists, a second rail region ra2 where a second rail R2 exists, and other regions, and based on this classification, in the rail coordinate acquisition process, acquire the position coordinates of the corresponding region in the real-time video LV that corresponds to the first rail region ra1 and the position coordinates of the corresponding region in the real-time video LV that corresponds to the second rail region ra2. Furthermore, simulation device 10 does not necessarily have to be able to execute rail approximate curve identification processing.

[0102] In the above-described embodiment, the simulation device 10 searches for a second position coordinate P2 at a position that is a first search distance d1 from a first position coordinate P1 included in each position coordinate group in the process of searching for equally spaced rail points. If the second position coordinate P2 is not found at the first search distance d1, the simulation device 10 searches for a third position coordinate P3 at a position that is a first search distance d1 from the second position coordinate P2 by gradually decreasing the search distance d from the first search distance d1. However, this is not limited to this. If the second position coordinate P2 is not found in the process of searching for equally spaced rail points, the simulation device 10 may not search for the second position coordinate P2 by gradually decreasing the search distance d from the first search distance d1. Alternatively, if the second position coordinate P2 is found by gradually decreasing the search distance d from the first search distance d1, the simulation device 10 may search for the third position coordinate P3 starting from a position that is a first search distance d1 from the first position coordinate P1, rather than the second position coordinate P2.

[0103] In the above-described embodiment, the simulation device 10 is configured to be able to perform a filtering process in which, from among the multiple equally spaced rail points Pr identified in the rail position identification process, those equally spaced rail points Pr whose coordinate values ​​in a direction along the height direction of the rail R in the real-time video LV are less than a predetermined threshold, and the rail approximation curve identification process is executed after the filtering process. However, the present invention is not limited to this. The simulation device 10 may not be able to perform the filtering process, and may instead identify approximation curves for all of the identified equally spaced rail points Pr in the rail approximation curve identification process.

[0104] In the above-described embodiment, the simulation device 10 is configured to be able to execute, in the superimposed display process, a center line identification process that identifies a center line lc located midway between the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2, and a model display process that displays multiple structure gauge models M along the center line lc. In the model display process, the structure gauge models M are displayed so that their height directions are perpendicular to both the extension direction of the center line lc and the opposing directions of the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2. However, this is not limiting. The simulation device 10 does not necessarily have to be able to execute the center line identification process and the model display process. In this case, for example, the simulation device 10 may be configured to be able to display the construction limit model M so that the approximate curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximate curve of the equally spaced rail points Pr of the second position coordinate group G2 coincide with the part of the construction limit model M corresponding to the position of the vehicle wheels.

[0105] Furthermore, the simulation device 10 does not have to be able to display multiple structure gauge models M. Furthermore, in the model display processing, the simulation device 10 does not have to display the structure gauge model M so that the height direction of the structure gauge model M is along a direction perpendicular to both the extension direction of the center line lc and the opposing direction of the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2. For example, in the model display processing, the simulation device 10 may display the structure gauge model M so that the height direction of the structure gauge model M is along the vertical direction.

[0106] In the above-described embodiment, the simulation device 10 has been described as being configured to execute a position coordinate group selection receiving process that receives a selection of at least one of the first position coordinate group G1 and the second position coordinate group G2, a position coordinate group division process that divides the selected position coordinate group into two columns, and a guardrail discrimination process that identifies one of the two divided columns as a main rail position coordinate group Gmr and the other as a guardrail position coordinate group Ggr. However, this is not a limitation. The simulation device 10 does not necessarily have to be able to execute the position coordinate group selection receiving process, the position coordinate group division process, and the guardrail discrimination process. Furthermore, in the above-described embodiment, the simulation device 10 has been described as being configured to execute a track discrimination process that identifies one of the two divided columns as a standard gauge position coordinate group and the other as a narrow gauge position coordinate group. However, this is not a limitation. The simulation device 10 does not necessarily have to be able to execute the track discrimination process.

[0107] In the above-described embodiment, the simulation device 10 has been described as being configured to be able to execute a model movement process for moving the construction gauge model M superimposed on the real-time video LV along the opposing direction of the pair of rails R, but this is not limiting. The simulation device 10 does not have to be able to execute the model movement process. Furthermore, the simulation device 10 may be configured to be able to move the construction gauge model M superimposed on the real-time video LV along the extending direction of the pair of rails R.

[0108] In the above-described embodiment, the rail identification unit 54 of the control unit 50 of the simulation device 10 is described as identifying the rail R in the still image SI based on the machine learning model 61 in the storage unit 60 in the rail extraction determination process, but this is not limited to this. In the rail extraction determination process, the rail identification unit 54 does not have to identify the rail R in the still image SI based on the machine learning model 61. The rail R in the still image SI can be identified by any of various identification methods.

[0109] In the above-described embodiment, the display control unit 56 of the control unit 50 of the simulation device 10 is described as specifying the center line lc by executing the midpoint specification process and the midpoint approximation curve specification process, but this is not limited to this. For example, the simulation device 10 may specify the center line lc by drawing virtual line segments that are perpendicular to both the approximation curve of the equally spaced rail points Pr of the first position coordinate group G1 and the approximation curve of the equally spaced rail points Pr of the second position coordinate group G2 at least in two or more locations along the extension direction of the approximation curves, and connecting the centers of the virtual line segments.

[0110] In the above-described embodiment, the display control unit 56 of the control unit 50 of the simulation device 10 is described as being configured to be able to execute the processing for setting equally spaced central points and the processing for setting equally spaced aerial points, but this is not limiting. The display control unit 56 does not necessarily have to be able to execute the processing for setting equally spaced central points and the processing for setting equally spaced aerial points.

[0111] In the above-described embodiment, the display control unit 56 of the control unit 50 of the simulation device 10 has been described as being configured to be able to expand the obstruction range of the structure gauge model M superimposed on the real-time video LV to both the inside and outside of the structure gauge model M, but this is not limited to this. The display control unit 56 does not have to be able to expand the obstruction range of the structure gauge model M.

[0112] In the above-described embodiment, the control unit 50 of the simulation device 10 is described as including the obstacle determination unit 58, but this is not limited thereto, and the control unit 50 does not necessarily have to include the obstacle determination unit 58. [Explanation of symbols]

[0113] 1 Simulation System 10 Simulation equipment 20 Input section 30 Display section 32 Simulation screen 32a AR display area 32b Rail identification result display area 32c Shooting button 32d Settings list display area 32e Tool display area 40 Camera Department 50 control section 52 Image acquisition unit 54 Rail Identification Section 56 Display control unit 58 Obstacle Determination Department 60 Storage section 61 Machine Learning Models 63 Point cloud data for judgment 65 Simulation Program G1 First position coordinate group G2 Second position coordinate group GR guardrail Ggr Guardrail position coordinates Gmr Main line rail position coordinate group LV real-time video M Construction Gauge Model P1 First position coordinate P2 Second position coordinate P3 Third position coordinate Pa Equally spaced points in the sky Pc center equidistant points Pi Virtual Rail Equally Spaced Points Pm midpoint Pr Rail Equally Spaced Points R rail RAC Rail Approximation Curve RAC1 First rail approximation curve RAC2 Second rail approximation curve SI still image d search distance d1 First search distance d2 Second search distance d3 Third search distance dm Model display interval lc Chuo Line ra Rail area

Claims

1. a real-time image display process for displaying a real-time image of the real space; a still image acquisition process for acquiring a still image obtained by capturing the real space; a rail identification process for identifying the positions of a pair of rails in the real-time video based on the still image; a superimposition display process for superimposing and displaying a construction gauge model on the rail in the real-time image; It is configured to be able to execute In the rail identification process, a rail extraction process for classifying the still image into a rail area where the rail exists and an area other than the rail; a rail coordinate acquisition process for acquiring position coordinates of a corresponding area corresponding to the rail area in the real-time video by thinning out the position coordinates; a rail position identification process for identifying the position of the rail in the real-time video based on the acquired position coordinates; is configured to run A simulation device characterized by:

2. In the rail position identification process, a coordinate classification process for classifying the position coordinates of the corresponding area into a first group of position coordinates constituting a first column and a second group of position coordinates constituting a second column; a rail equally-spaced point searching process for searching for a plurality of rail equally-spaced points located at predetermined intervals along the longitudinal direction of each row in each classified position coordinate group; a rail approximation curve identification process for identifying an approximation curve of the identified plurality of equally spaced rail points; is configured to run 2. The simulation device according to claim 1.

3. In the rail equally spaced point search process, Searching for second position coordinates at a first search distance from the first position coordinates included in each group of position coordinates; If the second position coordinates are not found at the position of the first search distance, the search distance is gradually shortened from the first search distance to search for the second position coordinates; Searching for third position coordinates at a position that is the first search distance from the second position coordinates.

3. The simulation device according to claim 2.

4. In the rail position identification process, a filtering process can be executed to exclude, from the identified plurality of equally spaced rail points, those equally spaced rail points whose coordinate values ​​in a direction along the height direction of the rail in the real-time image are less than a predetermined threshold value, and the rail approximate curve identification process is executed after the filtering process.

4. The simulation device according to claim 2 or 3.

5. In the superimposed display processing, a center line identification process for identifying a center line located midway between an approximation curve of the equally spaced rail points of the first position coordinate group and an approximation curve of the equally spaced rail points of the second position coordinate group; a model display process for displaying a plurality of the construction gauge models along the center line; It is configured to be able to execute In the model display processing, the construction gauge model is displayed so that the height direction of the construction gauge model is along a direction perpendicular to both the extending direction of the center line and the opposing direction of the approximation curve of the equally spaced rail points of the first position coordinate group and the approximation curve of the equally spaced rail points of the second position coordinate group.

4. The simulation device according to claim 2 or 3.

6. a position coordinate group selection receiving process for receiving a selection of at least one of the first position coordinate group and the second position coordinate group; a position coordinate group division process for dividing the selected position coordinate group into two columns; A guardrail discrimination process is performed to discriminate one of the two divided rows as a group of position coordinates of the main rail and the other as a group of position coordinates of the guardrail. is configured to run 4. The simulation device according to claim 2 or 3.

7. A real-time image display process for displaying a real-time image of a real space; a still image acquisition process for acquiring a still image obtained by capturing the real space; a rail identification process for identifying the positions of a pair of rails in the real-time video based on the still image; a superimposition display process for superimposing a construction gauge model on the rail in the real-time image; a movement operation receiving process for receiving an input of an operation to move the structure gauge model along the opposing directions of the pair of rails; a model movement process for moving the construction gauge model superimposed on the real-time image along the opposing direction of the pair of rails based on the input movement amount; is configured to run A simulation device characterized by:

8. equipped with a camera and a simulation device, The simulation device includes: a real-time image display process for displaying a real-time image of the real space captured by the camera; a still image acquisition process for acquiring a still image of the real space captured by the camera; a rail identification process for identifying the positions of a pair of rails in the real-time video based on the still image; a superimposition display process for superimposing and displaying a construction gauge model on the rail in the real-time image; It is configured to be able to execute In the rail identification process, a rail extraction process for classifying the still image into a rail area where the rail exists and an area other than the rail; a rail coordinate acquisition process for acquiring position coordinates of a corresponding area corresponding to the rail area in the real-time video by thinning out the position coordinates; a rail position identification process for identifying the position of the rail in the real-time video based on the acquired position coordinates; is configured to run A simulation system comprising:

9. a real-time image display step of displaying a real-time image of the real space; a still image acquisition step of acquiring a still image obtained by capturing the real space; a rail identification step of identifying the positions of a pair of rails in the real-time video based on the still image; a superimposition display step of superimposing and displaying a construction gauge model on the rail in the real-time image; The simulation device executes the In the rail identification step, a rail extraction step of classifying the still image into a rail area where the rail exists and an area other than the rail; a rail coordinate acquisition step of thinning out and acquiring position coordinates of a corresponding area corresponding to the rail area in the real-time video; a rail position identifying step of identifying the position of the rail in the real-time video based on the acquired position coordinates; The simulation device executes A simulation method comprising:

10. a real-time image display process for displaying a real-time image of the real space; a still image acquisition process for acquiring a still image obtained by capturing the real space; a rail identification process for identifying the positions of a pair of rails in the real-time video based on the still image; a superimposition display process for superimposing and displaying a construction gauge model on the rail in the real-time image; is executed by a simulation device, In the rail identification process, a rail extraction process for classifying the still image into a rail area where the rail exists and an area other than the rail; a rail coordinate acquisition process for acquiring position coordinates of a corresponding area corresponding to the rail area in the real-time video by thinning out the position coordinates; a rail position identification process for identifying the position of the rail in the real-time video based on the acquired position coordinates; The simulation device executes A simulation program characterized by:

Images