Immersive train scene rendering method and device, storage medium and electronic equipment

Abstract

The invention discloses an immersive train scene rendering method and device, a storage medium and electronic equipment. The method comprises the steps that virtual reality tracking equipment, a virtual camera and a virtual reality locator are arranged; the device position of the virtual reality tracking device is a user position, and the camera view angle of the virtual camera is a user view angle; determining a current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator; determining real-time simulation train scene data corresponding to the current user view angle based on the current relative position; and rendering the scene display screen based on the real-time simulation train scene data. According to the method and the device provided by the invention, the display picture of each scene display screen is adjusted in real time along with the change of the user position and the user visual angle, so that the immersion feeling of the train operation scene is increased, and the immersive experience is provided for the user.

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and in particular to an immersive train scene rendering method, device, storage medium and electronic device. Background Art

[0002] When the train virtual reality content is started separately on a computer, the viewport area in the presented virtual train scene is fixed, and the user's perspective is generally fixed at the driver's perspective, which only meets the display requirements of the train driver's perspective. When the user's perspective is switched to outside the train or the train operation principle is displayed from a bird's-eye view, the open display space gives the user a poor visual experience.

[0003] Therefore, how to render the train scene in real time according to the changing user perspective to provide users with an immersive experience has become a technical problem that needs to be solved urgently in the industry. Summary of the invention

[0004] The present application provides an immersive train scene rendering method, device, storage medium and electronic device to solve the technical problem in the prior art of how to render the train scene in real time according to the changing user perspective, thereby providing the user with an immersive experience.

[0005] In a first aspect, the present application provides an immersive train scene rendering method, comprising:

[0006] Setting a virtual reality tracking device, a virtual camera and a virtual reality locator; the device position of the virtual reality tracking device is the user position, and the camera perspective of the virtual camera is the user perspective;

[0007] Determine the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator;

[0008] Determine the real-time simulation train scene data corresponding to the current user perspective based on the current relative position;

[0009] The scene display screen is rendered based on the real-time simulated train scene data.

[0010] In some embodiments, determining the real-time simulated train scene data corresponding to the current user perspective based on the current relative position includes:

[0011] Setting simulation train scene parameters in the simulation environment, wherein the simulation train scene parameters include train formation parameters and track equipment parameters;

[0012] Controlling the simulated train to run in the simulation environment and generating real-time train running data;

[0013] Determine the real-time simulation train scene data corresponding to the current camera view of the virtual camera based on the current relative position, the current train operation data, and the simulation environment;

[0014] Wherein, the train operation data includes the current train speed, the current train position, and the train obstacle data.

[0015] In some embodiments, the position of the virtual reality locator is fixed, and the virtual reality tracking device is a handheld device;

[0016] Wherein, the virtual position and virtual direction of the virtual camera change with the change of the position of the virtual reality tracking device.

[0017] In some embodiments, the rendering of the scene display screen based on the real-time simulation train scene data includes:

[0018] Generate the vertex information of the grid on the scene display screen based on the real-time simulation train scene data;

[0019] Generate a new grid based on the vertex information, and render each scene display screen based on the new grid.

[0020] In some embodiments, the generating of the vertex information of the grid on the scene display screen based on the real-time simulation train scene data includes:

[0021] Define the coordinate system of the scene display screen; each scene display screen is a regular quadrilateral;

[0022] Determine the corner coordinates of each scene display screen based on the coordinate system, and obtain the width and height of each scene display screen based on the corner coordinates;

[0023] In the case where the width and height of any scene display screen are different, fill in the display picture of the any scene display screen based on the maximum value of the width and height; the display picture is generated based on the real-time simulation train scene data;

[0024] Determine the texture coordinates of each vertex of the grid on the scene display screen based on the filled display picture, and obtain the vertex information.

[0025] In some embodiments, the generating of a new grid based on the vertex information and the rendering of each scene display screen based on the new grid includes:

[0026] Perform an initialization process on the grid of the current scene display screen;

[0027] Assign values to the initialized mesh based on each vertex and the vertex information to obtain the new mesh;

[0028] Render each scene display screen based on the new mesh.

[0029] In some embodiments, after rendering the scene display screen based on the real-time simulation train scene data, a train scene picture is obtained;

[0030] Wherein, the train scene picture includes at least one of a picture of the simulated train running forward, a picture of the simulated train meeting, a picture of the train platform door opening and closing, a picture of the train signal display, or a picture of the display of information about obstacles ahead.

[0031] In a second aspect, the present application provides an immersive train scene rendering device, including:

[0032] A setting module, configured to set a virtual reality tracking device, a virtual camera, and a virtual reality locator; the device position of the virtual reality tracking device is the user position, and the camera view of the virtual camera is the user view;

[0033] A position module, configured to determine the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator;

[0034] A generation module, configured to determine real-time simulation train scene data corresponding to the current user view based on the current relative position;

[0035] A rendering module, configured to render the scene display screen based on the real-time simulation train scene data.

[0036] In a third aspect, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0037] In a fourth aspect, the present application provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to implement the above method when executing the program through the computer program.

[0038] The immersive train scene rendering method, device, storage medium and electronic device provided by this application can collect the user's position in real time through the VRtracker device and the VR locator. The relative position between the virtual camera and the scene display screen can be determined based on the relative position between the VR tracker device and the VR locator, so as to obtain the real-time simulation train scene data corresponding to the current user's perspective. The display images of each scene display screen in the multi-screen immersive display space are generated according to the real-time simulation train scene data, so that the display images of each scene display screen are adjusted in real time as the user's position and perspective change, increasing the immersion of the train operation visual scene and thus giving the user an immersive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application and used together with the specification to explain the principles of this application.

[0040] In order to more clearly illustrate the technical solutions in this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 It is one of the flow diagrams of the immersive train scene rendering method provided by the embodiments of this application;

[0042] Figure 2 It is the structural diagram of the train visual scene display system provided by the embodiments of this application;

[0043] Figure 3 It is the structural diagram of the multi-screen immersive display space provided by the embodiments of this application;

[0044] Figure 4 It is the structural diagram of the terminal device provided by the embodiments of this application;

[0045] Figure 5 It is the second flow diagram of the immersive train scene rendering method provided by the embodiments of this application;

[0046] Figure 6 It is the third flow diagram of the immersive train scene rendering method provided by the embodiments of this application;

[0047] Figure 7 It is the fourth flow diagram of the immersive train scene rendering method provided by the embodiments of this application;

[0048] Figure 8Schematic structural diagram of a triangle provided by an embodiment of the present application;

[0049] Figure 9 Schematic structural diagram of an immersive train scene rendering device provided by an embodiment of the present application;

[0050] Figure 10 Schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0052] It should be noted that the terms "first", "second", etc. in the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices.

[0053] The immersive train scene rendering method provided by the embodiments of the present application is applicable to a terminal device. The terminal device can be various electronic devices having a display screen and supporting web browsing, including but not limited to servers, smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0054] Figure 1 One of the flow diagrams of the immersive train scene rendering method provided by the embodiments of the present application is as follows Figure 1 As shown, the method includes step 110, step 120, step 130, and step 140. The flow steps of this method are only a possible implementation manner of the present application.

[0055] Step 110: Set up a virtual reality tracking device, a virtual camera, and a virtual reality locator; the device position of the virtual reality tracking device is the user position, and the camera view of the virtual camera is the user view.

[0056] Specifically, the execution subject of the immersive train scene rendering method provided by the embodiments of the present application is an immersive train scene rendering device, which can be a hardware device independently set in a terminal device or a software program running in the terminal device. For example, when the terminal device is a desktop computer, the immersive train scene rendering device can be embodied as an application program such as virtual reality (VR) software in a mobile phone.

[0057] The virtual reality tracking (VRtracker) device can be a handheld device. The user can move by holding the VR tracker device, and the real-time position of the user can be determined by real-time positioning of the VR tracker device.

[0058] The virtual camera is used to simulate the user's eyes, and the camera view of the virtual camera is the user's view. The virtual position and virtual direction of the virtual camera change with the change of the position of the VR tracker device. The virtual direction of the virtual camera is the camera view of the virtual camera. Therefore, the user position and user view can be obtained according to the position information of the VR tracker device. The position information of the VR tracker device includes the position coordinates and position orientation of the VR tracker device, etc.

[0059] The VR locator is used to locate the VR tracker device. The VR locator can be fixedly installed around the multi-screen immersive display space formed by the scene display screen. Because the communication between the VR tracker device and the VR locator will be affected when the distance between them is too large, the VR tracker device needs to be used within the target area.

[0060] Step 120: Determine the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator.

[0061] Specifically, the positions of the scene display screen and the VR locator are fixed, and the positions of the scene display screen and the VR locator can be obtained in advance. The position of the VR tracker device is the same as the position of the virtual camera. Therefore, the current relative position between the virtual camera and the scene display screen can be obtained according to the current relative position between the VR tracker device and the VR locator.

[0062] Step 130: Determine the real-time simulation train scene data corresponding to the current user view based on the current relative position.

[0063] Specifically, the current relative position between the virtual camera and the scene display screen can determine the current relative position between the current user view and the scene display screen, so as to determine the VR scene that the scene display screen should present under the current user view.

[0064] The real-time simulation train scene data are the relevant data of the scene and environment of the real-time simulated train operation. When the user's perspective is the in-vehicle perspective, the real-time simulation train scene data may include the dynamic picture data in front of the train; when the user's perspective is the in-vehicle perspective, the real-time simulation train scene data may include the station, track, and train operation data that can be observed from outside the train, etc.

[0065] Step 140, render the scene display screen based on the real-time simulation train scene data.

[0066] Specifically, the real-time simulation train scene data can be processed by a modeling software and a texture baking software, and the scene display screen can be rendered to obtain a train scene picture; the train scene picture includes at least one of a picture of a simulated train running forward, a picture of a simulated train meeting, a picture of the train platform door opening and closing, a picture of the train signal display, or a picture of the display of the front obstacle information.

[0067] According to the current user's perspective, it can be determined which specific pictures are included in the current train scene picture.

[0068] The terminal device may include a train visual display system, and the immersive train scene rendering device can be applied to the train visual display system. Figure 2 The structural schematic diagram of the train visual display system provided by the embodiment of the present application is as Figure 2 shown. The train visual display system includes a main program module, a VR control module, a communication module, a screen display module, a model module, a user interface (UI) module, and a special effect module.

[0069] The main program module is used to form the overall program of the train visual display system, connect each module in the train visual display system, and manage the operation of each module, etc.

[0070] The VR control module is used to control the VR tracker device to move the virtual camera software. Then, the corresponding relationship between the virtual camera and the scene display screen is calculated through the relative position conversion of the space between the VR tracker device and the VR locator.

[0071] The communication module is used to provide a communication channel for the information interaction between the train visual display system and the signal system and the trackside simulation system. The communication method can be wireless communication.

[0072] The screen display module is used to calculate the camera view of the virtual camera, so as to display the naked-eye 3D picture under the virtual camera view. The screen display module may include multiple scene display screens, and each scene display screen may be rectangular or square. Three scene display screens can form three faces of a cube or a cuboid, so as to form a multi-screen immersive display space. Figure 3 It is a schematic structural diagram of the multi-screen immersive display space provided by the embodiment of the present application. Figure 3 The solid lines in it are the edges of each scene display screen, and the dotted lines are added to show more clearly that the multi-screen immersive display space is a concave space structure. There is no dotted line part in the actual scene. It should be noted that Figure 3 It is only one structure of the multi-screen immersive display space.

[0073] The model module is used to simulate the subway tunnel scene through modeling software and texture baking software, including model building simulation of tunnels, stations, trains, etc.

[0074] The UI module is used to display train operation information, such as train speed, train position, obstacle information, etc.

[0075] The special effect module is used to simulate the detection wave emitted when the train detects an obstacle, and can display the detection wave in a graphical way on the scene display screen.

[0076] The immersive train scene rendering method provided by the embodiment of the present application can collect the user's position in real time through the VR tracker device and the VR locator. The relative position between the virtual camera and the scene display screen can be determined through the relative position between the VR tracker device and the VR locator, so as to obtain the real-time simulation train scene data corresponding to the current user's perspective. According to the real-time simulation train scene data, the display pictures of each scene display screen in the multi-screen immersive display space are generated, so that the display pictures of each scene display screen are adjusted in real time as the user's position and user's perspective change, increasing the immersion of the train operation visual scene, and thus giving the user an immersive experience.

[0077] It should be noted that each embodiment of the present application can be freely combined, the order can be swapped, or each can be executed alone, and does not need to rely on or depend on a fixed execution order.

[0078] In some embodiments, step 130 includes:

[0079] Set the simulation train scene parameters in the simulation environment, and the simulation train scene parameters include train formation parameters and track equipment parameters;

[0080] Control the simulation train to run in the simulation environment and generate real-time train operation data;

[0081] Determine the real-time simulation train scene data corresponding to the current camera view of the virtual camera based on the current relative position, the current train operation data, and the simulation environment;

[0082] Among them, the train operation data includes the current train speed, the current train position, and the train obstacle data.

[0083] Specifically, Figure 4 It is a schematic structural diagram of the terminal device provided by the embodiment of the present application; as Figure 4 shown, the terminal device includes a train vision display system, a signal system, and a simulation system.

[0084] The train vision display system receives the train operation and train front obstacle messages of the signal system and the simulation system through Ethernet.

[0085] The signal system includes an Intelligent Train Supervision (ITS), an Intelligent Train Protection (ITP), an Object Controller (OC), a Data Communication System (DCS), an Intelligent Train Operation (ITO), and an Intelligent Eye System (ITE).

[0086] The ITP is used to send the calculated train operation data (such as speed and position, etc.) to the train vision display system, and the ITE is used to send obstacle detection data, etc. to the train vision display system. The signal system realizes the formation operation of the simulation train through complex operations of each internal system. The signal system controls the simulation train to run in the simulation environment simulated by the simulation system, generates real-time train operation data, and sends this data to the train vision display system.

[0087] The simulation system includes a trackside simulation system and a train simulation system.

[0088] The trackside simulation system is used to simulate the states of trackside devices such as signal lights and switches, etc., and send the state data of each trackside device to the train vision display system. The train simulation system is used to simulate the train state and send the train state data to the train vision display system. The simulation system sets simulation train scene parameters in the simulation environment, including train formation parameters and track equipment parameters, so as to provide a simulation environment for the operation of the simulation train controlled by the signal system indoors.

[0089] The train vision display system obtains the real-time simulation train scene data corresponding to the current camera view of the virtual camera based on the simulation environment data received from the simulation system and the train operation data received from the signal system, and twins and displays the train operation scene equipped with the virtual formation signal system in the virtual environment according to the real-time simulation train scene data, and combines the special effect module in the virtual formation to display the operation principle of the simulation train in real time.

[0090] The train vision display system demonstrates the principle of the virtual formation train according to the train operation data sent by the ITP, including information such as the simulation train speed, formation status, and countdown to stop, and demonstrates the operation of other simulation trains, the switching of trackside equipment status, and train information according to the data sent by the trackside simulation system.

[0091] The train vision display system can output the model and texture to the development engine through modeling software and texture baking software for virtual scene development, and display the forward running picture of the train, the chasing and meeting pictures with other trains, the real-time operation data, the opening and closing of platform screen doors, the signal display, and the information of obstacles ahead, etc. on the scene display screen.

[0092] The size of the scene display screen can be set according to the sizes of the real train and real trackside equipment, so that the real track of the train in the displayed simulation environment can be modeled and restored at a ratio of 1:1.

[0093] The immersive train scene rendering method provided by the embodiments of the present application increases the realism and immersion of the demonstration of the operation principle of the virtual formation train by simulating the train operation environment and train operation status, and improves the user experience.

[0094] In some embodiments, step 140 includes:

[0095] Generating vertex information of the grid on the scene display screen based on the real-time simulation train scene data;

[0096] Generating a new grid based on the vertex information and rendering each scene display screen based on the new grid.

[0097] Generating vertex information of the grid on the scene display screen based on the real-time simulation train scene data includes:

[0098] Defining the coordinate system of the scene display screen; each scene display screen is a regular quadrilateral;

[0099] Determining the corner point coordinates of each scene display screen based on the coordinate system and obtaining the width and height of each scene display screen based on the corner point coordinates;

[0100] In the case where the width and height of any scene display screen are different, the display screen of any scene display screen is filled based on the maximum value of the width and height; the display screen is generated based on the real-time simulation train scene data;

[0101] Based on the completed display image, the texture coordinates of each vertex of the mesh on the scene display screen are determined to obtain vertex information.

[0102] Generate a new mesh based on the vertex information, and render each scene display screen based on the new mesh, including:

[0103] Initialize the grid of the current scene display screen;

[0104] Based on each vertex and vertex information, the initialized mesh is assigned a value to obtain a new mesh;

[0105] Each scene display screen is rendered based on the new grid.

[0106] Specifically, the steamVR can be adjusted to the headless display mode, the VR tracker device can be connected to the virtual train workstation, and the VR base station can be set in the target area. The virtual train workstation refers to a workstation that simulates and presents train operation and control in a virtual reality environment. The VR base station includes multiple VR locators.

[0107] Use one of the VR locators as the initial position of the virtual camera. After placing the locator, place a scene display screen model that is one-to-one with the actual train at the position corresponding to the actual train. Then place the virtual camera at the set initial position. Then use the VR tracker device to move in the real scene and adjust the position of the initial position in the virtual scene relative to the screen model to ensure that the position of the VR tracker device in reality corresponds to the position of the virtual camera in the virtual scene.

[0108] Figure 5 The second flowchart of the immersive train scene rendering method provided in the embodiment of the present application; Figure 5 As shown, before the user uses the VR tracker device or is about to use the VR tracker device, the initial position of the virtual camera can be set first. Each scene display screen generates vertex information of the mesh on the scene display screen according to the initial position. The vertex information can include the vertex position after stretching or compression, so that the content displayed on the scene display screen from the user's perspective is distortion-free. Then, the texture coordinates (UV) of each vertex in the Mesh are converted according to the initial position. The vertex information can also include UV information. A new Mesh is generated based on the generated vertices and UVs, and the new Mesh is assigned to each scene display screen, and the train scene is displayed through the scene display screen.

[0109] Figure 6 The third flowchart of the immersive train scene rendering method provided in the embodiment of the present application is as follows: Figure 7 A fourth flow chart of the immersive train scene rendering method provided in an embodiment of the present application is as follows: Figure 6 and Figure 7 As shown, when the user moves with the VR tracker device, the position and orientation of the VR tracker device and the virtual camera may change in real time. According to the real-time relative position of the VR tracker device and the VR locator, the real-time relative position of the virtual camera and the scene display screen can be adjusted accordingly so that the virtual camera moves accordingly.

[0110] A coordinate system may be defined in a multi-screen immersive display space formed by scene display screens, and coordinate axes of the coordinate system may be determined.

[0111] Get the points on the four corners of each scene display screen, and determine the corner point coordinates of each scene display screen according to the coordinate system. According to the obtained corner point coordinates, filter out the maximum and minimum values of x and y in each scene display screen, so as to obtain the width and height of each scene display screen. According to the converted width and height, fill the smaller width and height value with half of the difference, and obtain the upper, lower, left and right sides of the converted UV. The purpose of filling the display screen is to prevent the image from being deformed.

[0112] The Mesh of the scene display screen is constructed by triangles. The number of triangles in the Mesh = (number of x segments + 1) * (number of y segments + 1) * 6. The number of x segments and y segments are the number of segments of the mesh in the horizontal and vertical directions.

[0113] Because two triangles need to be drawn for each vertex, involving a total of six vertices, the initial size of the triangle array is calculated first to obtain the order of each vertex in the triangle, and the triangle starting subscript of each vertex is recorded. Figure 8 A schematic diagram of a triangle structure provided in an embodiment of the present application, such as Figure 8 As shown, in the first triangle, the first vertex 0 is the initial point, the second point 1 is the point directly above the initial point, and the third point is the point adjacent to the right of 2; in the second triangle, the first vertex 3 is the point directly above the initial point, the second vertex 4 is the point to the upper right of the initial point, and the third vertex 5 is the point adjacent to the right of the initial point.

[0114] Generate new vertices based on the position and orientation of the real-time virtual camera: number of points = (number of x segments + 1) * (number of y segments + 1)

[0115] All points are generated based on the axes, segmentation numbers, and initial points at the lower left corner of each scene display screen after the virtual camera moves. The new vertex positions, new triangles, and new UVs can be calculated, thereby generating a new Mesh.

[0116] The new UVs in the new Mesh are generated in the following way:

[0117] The number of UVs = the number of points. Set the UVs for each vertex. Based on the position of each vertex, the coordinates converted on its corresponding scene display screen and the boundary coordinates of the scene display screen, the UVs of this vertex can be obtained. The formula is as follows:

[0118] New_UV.x = (Screen_Pos.x - Screen_Left) / (Screen_Right - Screen_Left) New_UV.y = (Screen_Pos.y - Screen_Down) / (Screen_Up - Screen_Down)

[0119] Among them, Screen_Left and Screen_Right respectively represent the coordinates on the left and right sides of the scene display screen; Screen_Down and Screen_Up respectively represent the coordinates on the upper and lower sides of the scene display screen; New_UV.x represents the relative distance of the vertex on the scene display screen from the left side of the scene display screen in the horizontal direction; New_UV.y represents the relative distance of the point on the scene display screen from the bottom of the scene display screen in the vertical direction.

[0120] A new Mesh can be created first and initialized. The built-in methods in Unity can be used: RecalculateBounds, RecalculateNormals, and RecalculateTangents to initialize the Mesh.

[0121] Then assign the newly created triangles, vertices, and UVs to the new Mesh.

[0122] Assign the newly generated Mesh to each scene display screen to display the train scene.

[0123] The immersive train scene rendering method provided by the embodiments of this application calculates the surrounding picture in the virtual world as a UV map and finally renders it onto the screen in the real world. By dynamically updating the UVs, the rendering of the train running view in the multi-screen semi-surround display space is realized. The rendering algorithm can be adjusted according to the actual screen configuration, increasing the realism and immersion of the virtual formation train operation principle display.

[0124] The immersive train scene rendering device provided by the embodiments of the present application will be described below. The immersive train scene rendering device described below can be correspondingly referred to the immersive train scene rendering method described above.

[0125] Figure 9 is a schematic structural diagram of the immersive train scene rendering device provided by the embodiments of the present application. As Figure 9 shown, the device includes a setting module 910, a position module 920, a generation module 930, and a rendering module 940.

[0126] The setting module is used to set a virtual reality tracking device, a virtual camera, and a virtual reality locator; the device position of the virtual reality tracking device is the user position, and the camera view of the virtual camera is the user view.

[0127] The position module is used to determine the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator.

[0128] The generation module is used to determine the real-time simulation train scene data corresponding to the current user view based on the current relative position.

[0129] The rendering module is used to render the scene display screen based on the real-time simulation train scene data.

[0130] Specifically, according to the embodiments of the present application, any multiple of the setting module, the position module, the generation module, and the rendering module can be combined and implemented in one module, or any one of them can be split into multiple modules.

[0131] Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0132] According to the embodiments of the present application, at least one of the setting module, the position module, the generation module, and the rendering module can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable way of integrating or packaging circuits and other hardware or firmware, or can be implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them.

[0133] Alternatively, at least one of the setting module, the position module, the generation module, and the rendering module can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0134] The immersive train scene rendering device provided by the embodiment of the present application can collect the user's position in real time through the VR tracker device and the VR locator. The relative position between the virtual camera and the scene display screen can be determined based on the relative position between the VR tracker device and the VR locator, so as to obtain the real-time simulation train scene data corresponding to the current user's perspective. The display images of each scene display screen in the multi-screen immersive display space are generated according to the real-time simulation train scene data, so that the display images of each scene display screen are adjusted in real time as the user's position and perspective change, increasing the immersion of the train operation visual scene, and thus giving the user an immersive experience.

[0135] In some embodiments, the generation module is specifically configured to:

[0136] Set the simulation train scene parameters in the simulation environment, where the simulation train scene parameters include train formation parameters and track equipment parameters;

[0137] Control the simulation train to run in the simulation environment and generate real-time train operation data;

[0138] Determine the real-time simulation train scene data corresponding to the current camera view of the virtual camera based on the current relative position, the current train operation data, and the simulation environment;

[0139] Among them, the train operation data includes the current train speed, the current train position, and the train obstacle data.

[0140] In some embodiments, the position of the virtual reality locator is fixed, and the virtual reality tracking device is a handheld device;

[0141] Among them, the virtual position and virtual direction of the virtual camera change as the position of the virtual reality tracking device changes.

[0142] In some embodiments, the rendering module is specifically configured to:

[0143] Generate the vertex information of the grid on the scene display screen based on the real-time simulation train scene data;

[0144] Generate a new grid based on the vertex information and render each scene display screen based on the new grid.

[0145] In some embodiments, the rendering module includes a filling sub-module, and the filling sub-module is specifically configured to:

[0146] Define the coordinate system of the scene display screen; each scene display screen is a regular quadrilateral;

[0147] Determine the corner coordinates of each scene display screen based on the coordinate system, and obtain the width and height of each scene display screen based on the corner coordinates;

[0148] When the width and height of any scene display screen are different, complete the display image of any scene display screen based on the maximum value of the width and height; the display image is generated based on real-time simulation train scene data;

[0149] Determine the texture coordinates of each vertex of the grid on the scene display screen based on the completed display image, and obtain vertex information.

[0150] In some embodiments, the rendering module further includes an initialization sub-module, and the initialization sub-module is specifically configured to:

[0151] Perform initialization processing on the grid of the current scene display screen;

[0152] Assign values to the initialized grid based on each vertex and the vertex information to obtain a new grid;

[0153] Render each scene display screen based on the new grid.

[0154] In some embodiments, the immersive train scene rendering device further includes a display module, and the display module is specifically configured to:

[0155] After rendering the scene display screen based on the real-time simulation train scene data, obtain a train scene image;

[0156] Wherein, the train scene image includes at least one of a simulation train moving forward image, a simulation train meeting image, a train platform door opening and closing image, a train signal display image, or a front obstacle information display image.

[0157] It should be noted here that the immersive train scene rendering device provided in the embodiments of the present application can implement all the method steps implemented in the embodiments of the above-mentioned immersive train scene rendering method, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0158] Figure 10 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application, as Figure 10As shown, the electronic device may include: a processor 1010, a communications interface 1020, a memory 1030, and a communications bus 1040. Among them, the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other through the communications bus 1040. The processor 1010 may call the logical commands in the memory 1030 to execute the above method, which includes:

[0159] Set up a virtual reality tracking device, a virtual camera, and a virtual reality locator; the device position of the virtual reality tracking device is the user position, and the camera view of the virtual camera is the user view;

[0160] Determine the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator;

[0161] Determine the real-time simulation train scene data corresponding to the current user view based on the current relative position;

[0162] Render the scene display screen based on the real-time simulation train scene data.

[0163] In addition, when the logical commands in the above memory are implemented in the form of software function modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0164] The processor in the electronic device provided in the embodiments of the present application may call the logical instructions in the memory to implement the above method. Its specific implementation manner is consistent with the foregoing method implementation manner and can achieve the same beneficial effects, which will not be elaborated here.

[0165] The embodiments of the present application also provide a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the methods provided in the above various embodiments.

[0166] The specific implementation manner is consistent with the foregoing method implementation manner and can achieve the same beneficial effects, which will not be elaborated here.

[0167] The embodiment of the present application provides a computer program product, including a computer program, which when executed by a processor implements the method as described above.

[0168] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0169] Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. An immersive train scene rendering method, characterized in that, Including: A virtual reality tracking device, a virtual camera, and a virtual reality locator are provided; the device position of the virtual reality tracking device is the user position, and the camera view of the virtual camera is the user view; Determine the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator; Determine the real-time simulation train scene data corresponding to the current user view based on the current relative position; Render the scene display screen based on the real-time simulation train scene data.

2. The immersive train scene rendering method according to claim 1, wherein The determining the real-time simulation train scene data corresponding to the current user view based on the current relative position includes: Set simulation train scene parameters in the simulation environment, where the simulation train scene parameters include train formation parameters and track equipment parameters; Control the simulation train to run in the simulation environment and generate real-time train operation data; Determine the real-time simulation train scene data corresponding to the current camera view of the virtual camera based on the current relative position, the current train operation data, and the simulation environment; Wherein, the train operation data includes the current train speed, the current train position, and train obstacle data.

3. The immersive train scene rendering method according to claim 1, wherein The position of the virtual reality locator is fixed, and the virtual reality tracking device is a handheld device; Wherein, the virtual position and virtual direction of the virtual camera change as the position of the virtual reality tracking device changes.

4. The immersive train scene rendering method according to claim 1, wherein, The rendering the scene display screen based on the real-time simulation train scene data includes: Generate vertex information of the grid on the scene display screen based on the real-time simulation train scene data; Generate a new grid based on the vertex information and render each scene display screen based on the new grid.

5. The immersive train scene rendering method according to claim 4, wherein The generating the vertex information of the grid on the scene display screen based on the real-time simulation train scene data includes: Define the coordinate system of the scene display screen; each scene display screen is a regular quadrilateral; Determine the corner coordinates of each scene display screen based on the coordinate system, and obtain the width and height of each scene display screen based on the corner coordinates; When the width and height of any scene display screen are different, fill in the display image of the any scene display screen based on the maximum value of the width and height; the display image is generated based on the real-time simulation train scene data; Determine the texture coordinates of each vertex of the grid on the scene display screen based on the filled display image to obtain the vertex information.

6. The immersive train scene rendering method according to claim 5, wherein, The generating a new grid based on the vertex information and rendering each scene display screen based on the new grid includes: Perform initialization processing on the grid of the current scene display screen; Assign values to the initialized grid based on each vertex and the vertex information to obtain the new grid; Render each scene display screen based on the new grid.

7. The immersive train scene rendering method according to claim 1, characterized in that After rendering the scene display screen based on the real-time simulation train scene data, a train scene image is obtained; Among them, the train scene picture includes at least one of a simulated train moving forward picture, a simulated train meeting picture, a train platform door opening and closing picture, a train signal display picture, or a front obstacle information display picture.

8. An immersive train scene rendering device, characterized in that, Including: A setting module for setting a virtual reality tracking device, a virtual camera, and a virtual reality locator; The device position of the virtual reality tracking device is the user position, and the camera view of the virtual camera is the user view; A position module for determining the current relative position between the virtual camera and the scene display screen based on the relative position between the virtual reality tracking device and the virtual reality locator; A generation module for determining real-time simulation train scene data corresponding to the current user view based on the current relative position; A rendering module for rendering the scene display screen based on the real-time simulation train scene data.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the immersive train scene rendering method according to any one of claims 1 to 7.

10. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the immersive train scene rendering method according to any one of claims 1 to 7 through the computer program.

Images