Display fusion method, system and storage medium for urban rail transit trains

Abstract

The present application relates to the field of data processing technology, and in particular to a display fusion method, system and storage medium for urban rail transit. The method includes: after the train is powered on, after receiving the on-board information sent by the on-board signal system and the fusion screen is started, the on-board information and the train information are sent to the fusion screen for display in a first display mode; the display mode of the fusion screen is obtained by analyzing the on-board information and the train information; the user's predicted viewpoint trajectory is predicted by the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and viewpoint prediction model, and the predicted viewpoint position in the predicted viewpoint trajectory is determined by analyzing the train information. When the predicted viewpoint position falls on the fusion screen, the user's target viewpoint is determined based on the display mode and predicted viewpoint position of the fusion screen, and the display content of the fusion screen is viewed. The present invention solves the problem of display fusion and improves viewing efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a display fusion method, system and storage medium for urban rail transit. Background Art

[0002] At present, the on-board signal system and vehicle of urban rail transit adopt independent human-computer interaction scheme. The two systems are equipped with on-board display screens in the train cab respectively, and the driver realizes the human-computer interaction function through two independent human-computer interfaces. During the operation of the train, the driver needs to monitor the two display screens in real time and operate according to the relevant prompt information. In this way, some information is repeatedly displayed on the signal and vehicle display screens, which is not convenient for the driver to use. Monitoring two human-computer interfaces at the same time makes the driver easy to fatigue and has high labor intensity. In addition, arranging multiple human-computer interfaces in the limited cab space causes the cab space to be cramped and increases the construction and operation and maintenance costs. Similar prior art includes a Chinese patent with publication number CN112486448B, which proposes a method, device and storage medium for digitizing information on the driving platform. The method includes: combining the first human-computer interface corresponding to the train control and management system with the second human-computer interface corresponding to the on-board signal system, and digitizing the target entity switch and target indicator light on the driving platform as the display items after combining the screens, which can reduce the overall cost and equipment complexity of the vehicle and improve the automation level and maintainability of the vehicle. In addition, similar prior art includes a U.S. patent with publication number US10618406B2, which provides a user interface device, including: a first user interface configured to display host information; a second user interface for displaying visitor information, and if a link signal of visitor information is received, the visitor information is sent to the first user interface, wherein the first user interface displays the visitor information by changing the host information to the visitor. Therefore, by integrating the cluster as the first user interface with the central dashboard as the second user interface to realize a large-area screen, and linking information between the cluster and the central dashboard, distraction during driving can be reduced to improve the driver's recognition rate and improve the driver's convenience. Both of the above two patent documents solve the problem of display screen fusion, but the display mode cannot be adjusted according to the content status of the display screen, and the user's access efficiency and driving safety are not high enough. Summary of the invention

[0003] The present application provides a display fusion method for urban rail transit trains, the method comprising:

[0004] When the train is powered on, a start signal is sent to the on-board signal system through the control unit, and after the train receives the on-board information sent by the on-board signal system and the fusion screen is started, the on-board information and the train information of the train are sent to the fusion screen, and the fusion screen displays the on-board information and the train information in a first display mode;

[0005] Acquire the display mode of the fusion screen by analyzing the vehicle-mounted information and the train information;

[0006] Predicting the user's predicted viewpoint trajectory by using the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and viewpoint prediction model, and determining whether the predicted viewpoint position in the predicted viewpoint trajectory falls on the fusion screen by analyzing the train information;

[0007] When the predicted viewpoint position falls on the fused screen, the target viewpoint of the user is determined based on the display mode of the fused screen and the predicted viewpoint position, and the display content of the fused screen is viewed.

[0008] As a preferred technical solution of the present invention, the fusion screen displays the on-board information and the train information in a first display mode, including:

[0009] When the train is powered on, the start signal is sent to the on-board signal system through the control unit in the train. After receiving the start signal, the on-board signal system sends the on-board information corresponding to the train according to the identification information of the train in the start signal. After the control unit receives the on-board information and the start signal of the fusion screen, the on-board information and the train information are sent to the fusion screen and displayed in the first display mode, wherein the first display mode is displayed in two areas on the left and right, and the start signal includes the train identification information and the start instruction.

[0010] As a preferred technical solution of the present invention, selecting a target display page through user viewpoint detection includes:

[0011] The control unit periodically acquires the train information of the train and receives the onboard information in real time, and when the train information is normal, the fusion screen displays the onboard information and the train information in the first display mode;

[0012] When an abnormality occurs in one of the vehicle-borne information and the train information, the abnormal vehicle-borne information or the train information is used as the first content, and the normal vehicle-borne information and the train information are used as the second content, and the fusion screen displays the first content and the second content in a second display mode, wherein the second display mode is that the fusion screen displays the first content in full screen, and displays the abnormal information in the first content in the form of a warning icon, and displays the second content in the form of a thumbnail at the top corner of the fusion screen, wherein the top corner positions corresponding to the vehicle-borne information and the train information are different when they are the second content;

[0013] When abnormalities occur simultaneously in the on-board information and the train information, the fused screen displays the on-board information and the train information in a third display mode, and the third display mode is that the on-board information and the train information are respectively displayed in the left and right areas of the fused screen, wherein, when the priority of the first abnormal information in the on-board information is high, the warning icon corresponding to the first abnormal information is displayed on the top layer of the fused screen, and the warning icon corresponding to the second abnormal information in the train information is second, and the display position of the train information is away from the user, and the on-board information and the train information are displayed at the bottom layer of the fused screen, and vice versa.

[0014] As a preferred technical solution of the present invention, judging whether the predicted viewpoint position falls on the fusion screen by analyzing the running state of the train includes:

[0015] Taking a first image of the user by a camera unit, and obtaining, based on the first image, a viewpoint distance between the user's eyes and the camera unit and a viewpoint angle between a line connecting the user's eyes and the camera unit and a horizontal line when the user is facing forward, wherein the first image is a depth image;

[0016] Periodically collecting the viewpoint information of the user, the viewpoint information including the viewpoint position of the user and the corresponding collection time, and calculating the viewpoint trajectory of the user and the viewpoint change speed based on the viewpoint information of the user every first time period, and inputting the viewpoint distance of the user, the viewpoint angle, the viewpoint trajectory of the user in the first time period, and the viewpoint change speed of the user in the first time period into the viewpoint prediction model, obtaining the viewpoint prediction trajectory in a second time period after the first time period, and when the viewpoint prediction trajectory intersects the fused screen, the intersection point of the viewpoint prediction trajectory with the fused screen is used as the predicted viewpoint position;

[0017] The train information is used to determine whether the driving state of the train has changed. When the driving state of the train has changed, the predicted viewpoint position is on the fusion screen; when the driving state of the train has not changed, the predicted viewpoint position is not on the fusion screen, wherein the driving state includes traction, braking and coasting.

[0018] As a preferred technical solution of the present invention, determining the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position, and viewing the display content of the fused screen, includes:

[0019] When the fusion screen is displayed in the first display mode, the display area in the vehicle information or the train information that is closest to the predicted viewpoint position is obtained as the target viewpoint according to the predicted viewpoint position, and when the viewpoint position of the user is less than a preset distance from the target viewpoint, the content details of the display area corresponding to the target viewpoint are displayed;

[0020] When the fusion screen is displayed in the second display mode, the warning icon on the fusion screen is used as the target viewpoint of the user, and when the viewpoint position of the user is less than the preset distance from the target viewpoint, the abnormal detailed information and the coping strategy corresponding to the warning icon are displayed, and when the viewpoint position of the user falls at the top corner thumbnail position of the fusion screen and the duration exceeds the first preset time, the display mode is switched to the first display mode and the vehicle information and the train information are displayed simultaneously;

[0021] When the fusion screen is displayed in the third display mode, when the user's viewpoint position is less than a preset distance from the display position of the warning icon with the highest priority, the abnormal detailed information and response strategies corresponding to the warning icon are displayed, and after continuing for a second preset time, the warning icon with the next highest priority is displayed. The method of this step is repeated until all the warning icons and the corresponding abnormal information and response strategies are displayed, and then the display mode is switched to the first display mode.

[0022] As a preferred technical solution of the present invention, the user's viewpoint position stays on the fusion screen for a time less than or equal to a third preset time each time, and the first preset time is greater than the second preset time, and the third preset time is greater than the second preset time.

[0023] As a preferred technical solution of the present invention, the on-board information includes operating level, driving mode, target distance, target speed, arrival information and information related to train control, and the train information includes the train's operating mode, operating speed, traction braking status, door status and train-related status information.

[0024] As a preferred technical solution of the present invention, the on-board signal system periodically receives and forwards the on-board information to the fusion screen through the Ethernet port of the train. When the fusion screen fails to receive the on-board information sent by the on-board signal system for more than a preset time, the communication between the fusion screen and the on-board signal system fails, the on-board information display range is displayed in black, and the fusion screen switches to the dual-channel RS422 serial port to communicate with the on-board signal system.

[0025] The present invention also provides a display fusion system for urban rail transit, the system is used to implement the above method, the system comprises:

[0026] a control unit, configured to send a start signal to the on-board signal system when the train is powered on, and after the train receives the on-board information sent by the on-board signal system and the fusion screen is started, send the on-board information and the train information of the train to the fusion screen, and the fusion screen displays the on-board information and the train information in a first display mode;

[0027] An analyzing unit, configured to acquire a display mode of the fusion screen by analyzing the vehicle-mounted information and the train information;

[0028] A judgment unit, configured to predict a predicted viewpoint trajectory of the user according to the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and a viewpoint prediction model, and to judge whether a predicted viewpoint position in the predicted viewpoint trajectory falls on the fusion screen by analyzing the train information;

[0029] A viewing unit is configured to determine the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position when the predicted viewpoint position falls on the fused screen, and to view the display content of the fused screen.

[0030] The present invention also provides a computer-readable storage medium, on which instructions are stored, and the above method is implemented when the instructions are executed by a processor.

[0031] The present invention integrates vehicle-mounted information and train information on a fusion screen to achieve an integrated display of information, reduce the number of information sources that the driver needs to pay attention to during driving, reduce the complexity of information processing, and improve the efficiency of information acquisition. The system can predict the user's predicted viewpoint trajectory based on the user's viewpoint change trajectory, viewpoint change speed and viewpoint prediction model, determine the target viewpoint based on the predicted viewpoint trajectory, and dynamically adjust the display content to ensure that the driver can quickly obtain key information when needed, reduce distractions caused by viewing multiple display screens, and thus improve driving safety. According to different vehicle-mounted information and train information states, the system can automatically switch display modes to make information display more intuitive. It is convenient for the driver to quickly identify and respond, and improves the efficiency of information processing. By reducing the number of human-machine interfaces in the cab, it not only saves space, but also reduces construction and operation and maintenance costs, and achieves cost-effectiveness optimization. The system can dynamically adjust the display mode according to the driving status of the train and the user's predicted viewpoint position, so that the system can adapt to different operating environments and user needs, improving the adaptability and flexibility of the system. When the communication between the on-board signal system and the fusion screen fails, the system can automatically switch to the backup communication mode to ensure the continuous display of information and enhance the robustness of the system. The mutual cooperation of the above technical solutions can not only improve the user's viewing efficiency, but also improve the user's driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0033] Figure 1 This is a flow chart of a display fusion method for urban rail transit trains in an embodiment of the present application;

[0034] Figure 2 This is a structural diagram of a display fusion system for urban rail transit trains in an embodiment of the present application. DETAILED DESCRIPTION

[0035] Embodiments of the present application provide a display fusion method, system, and storage medium for urban rail transit trains. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0036] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 As shown, an embodiment of the display fusion method for urban rail transit trains in the embodiment of the present application includes:

[0037] Step S1: when the train is powered on, a start signal is sent to the on-board signal system through a control unit, and after the train receives the on-board information sent by the on-board signal system and the fusion screen is started, the on-board information and the train information of the train are sent to the fusion screen, and the fusion screen displays the on-board information and the train information in a first display mode;

[0038] Specifically, when the train is powered on, the start signal is sent to the on-board signal system through the control unit. After receiving the start signal, the on-board signal system sends the corresponding on-board information to the train based on the identification information of the train. After receiving the start signal of the fusion screen, the on-board information and the train information of the train are sent to the fusion screen to prevent data loss. The on-board information and the train information are also displayed on the fusion screen in the first display mode. Through the technical solution, not only can the on-board information and the train information be accurately initialized and displayed, but also the on-board information and the train information can be displayed simultaneously through the first display mode, so that users can check at any time.

[0039] Step S2: acquiring the display mode of the fusion screen by analyzing the vehicle-mounted information and the train information;

[0040] Specifically, the control unit periodically obtains the train information of the train and receives the on-board information in real time. When the on-board information and the train information are both normal, the fusion screen displays the on-board information and the train information in the first display mode. When any one of the on-board information and the train information is abnormal, in order to facilitate the user to easily and prominently view the abnormality, when the on-board information is abnormal, it is displayed in the second display mode, wherein the second display mode is that the fusion screen displays the abnormal on-board information or train information in full screen, and displays the abnormal information in the form of a warning icon on the abnormal on-board information or train information. When the on-board information and the train information are abnormal at the same time, the fusion screen displays the on-board information and the train information in the third display mode. The third display mode is similar to the first display mode, except that there is no warning icon. Through the technical solution, when both the on-board information and the train information are abnormal, it is convenient for the user to view the abnormal information according to the priority of the abnormal information and take timely measures to improve driving safety and efficiency.

[0041] Step S3: predicting the user's predicted viewpoint trajectory according to the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and viewpoint prediction model, and determining whether the predicted viewpoint position in the predicted viewpoint trajectory falls on the fusion screen by analyzing the train information;

[0042] Specifically, when driving the train, users of different heights and body shapes have different corresponding viewpoint trajectories when viewing the fusion screen. In order to obtain a more accurate viewpoint prediction trajectory through the viewpoint prediction model, the driving image of the user is captured by the camera unit while the state data of the train remains unchanged for a preset time. Since the driving image is a depth image, the viewpoint distance between the user's eye and the camera unit is obtained through the pixel depth value corresponding to the user's eye center in the driving image, and the viewpoint angle between the line between the user's eye and the camera unit and the horizontal direction is obtained. The viewpoint distance and the viewpoint angle determine Different viewpoint trajectories of different users and the same user at different seat heights when viewing the above-mentioned fused screen, and at the same time, the viewpoint information of the above-mentioned users is periodically collected by the above-mentioned camera unit, and the viewpoint trajectory and viewpoint change speed of the above-mentioned users in the above-mentioned first time period are calculated based on the viewpoint information of the above-mentioned users at intervals of the above-mentioned first time period, and the viewpoint distance, viewpoint angle, and the viewpoint trajectory and viewpoint change speed of the user in the first time period are input into the viewpoint prediction model, so as to obtain the viewpoint prediction trajectory in the second time period after the above-mentioned first time period, and when the viewpoint prediction trajectory intersects with the above-mentioned fused screen, the intersection point of the viewpoint prediction trajectory with the above-mentioned fused screen is used as the above-mentioned predicted viewpoint position. Since the above-mentioned fusion screen is generally arranged in front of the train operating console, when the user needs to check the track condition in front of the train or operate the console controller, the viewpoint trajectory may pass through the above-mentioned fusion screen or overlap with the above-mentioned fusion screen. In order to accurately determine whether the above-mentioned viewpoint predicted viewpoint position is on the above-mentioned fusion screen, therefore, it is judged whether the driving state of the above-mentioned train has changed through the train state information in the above-mentioned train information. When the driving state of the train has changed, it is considered that the above-mentioned viewpoint predicted position is on the above-mentioned fusion screen. When the driving state of the above-mentioned train has not changed, it is considered that the above-mentioned viewpoint predicted position is not on the above-mentioned fusion screen. Through the above-mentioned technical scheme, it is possible to accurately obtain whether the above-mentioned user's viewpoint predicted position is on the above-mentioned fusion screen, so as to facilitate the display of the content that the user wants to view before the user's viewpoint falls on the target viewpoint position of the above-mentioned fusion screen, thereby improving the user's viewing efficiency and improving driving safety.

[0043] Step S4: when the predicted viewpoint position falls on the fused screen, the target viewpoint of the user is determined based on the display mode of the fused screen and the predicted viewpoint position, and the display content of the fused screen is checked.

[0044] Specifically, the first display mode is a display mode when both the on-board information and the train information are normal, and the display area on the fusion screen that is closest to the predicted viewpoint position is used as the target viewpoint, and when the user's viewpoint position is at the preset distance from the target viewpoint, the content details of the display area corresponding to the target viewpoint are displayed. The second display mode is a display mode when any one of the on-board information and the train information is abnormal, and the display area corresponding to the warning icon is used as the target viewpoint, and when the user's viewpoint position is at the preset distance from the warning icon, the abnormal detailed information and coping strategies corresponding to the warning icon are displayed. The display of the thumbnails can also be controlled by the user's viewpoint. The content in the fusion screen is converted into the above-mentioned first display mode. When the above-mentioned display mode is the above-mentioned third display mode, the viewing method of the above-mentioned warning icon with the highest priority is the same as the viewing method of the warning icon in the above-mentioned second mode, wherein the priority of the warning icon is consistent with the priority of its corresponding abnormal information, and it is converted into the above-mentioned first display mode after all the above-mentioned warning icons and corresponding contents are displayed. The time for the user to view the above-mentioned fusion screen each time is less than or equal to the above-mentioned third preset time. Through the above-mentioned technical scheme, corresponding viewing methods can be provided according to different display modes, which improves the convenience of users viewing the fusion screen, and can fully grasp the train information and on-board information without operation, and also improves the driving safety of users.

[0045] Further, the fusion screen is displayed in a first display mode based on the vehicle-mounted information and the train information, including:

[0046] When the train is powered on, the start signal is sent to the on-board signal system through the control unit in the train. After receiving the start signal, the on-board signal system sends the on-board information corresponding to the train according to the identification information of the train in the start signal. After the control unit receives the on-board information and the start signal of the fusion screen, the on-board information and the train information are sent to the fusion screen and displayed in the first display mode, wherein the first display mode is displayed in two areas on the left and right, and the start signal includes the train identification information and the start instruction.

[0047] Specifically, when the train is powered on, that is, when the train starts, it is necessary to send the above-mentioned start signal to the above-mentioned on-board signal system through the above-mentioned control unit, that is, the identification information and start-up instructions of the above-mentioned train. After the above-mentioned on-board signal system receives the above-mentioned start signal, it sends the corresponding above-mentioned on-board information to the above-mentioned train based on the identification information of the above-mentioned train. Since it takes time to start the fusion screen, the data sent before the fusion screen starts will be lost. Therefore, after the above-mentioned control unit receives the above-mentioned on-board information, it is also necessary to wait for receiving the start signal of the above-mentioned fusion screen. After receiving the start signal of the above-mentioned fusion screen, the above-mentioned on-board information and the train information of the above-mentioned train are sent to the above-mentioned fusion screen to prevent data loss, and the above-mentioned on-board information and the above-mentioned train information are displayed on the above-mentioned fusion screen in the above-mentioned first display mode, wherein the above-mentioned first display mode is displayed in two left and right areas, that is, the above-mentioned on-board information and the above-mentioned train information are respectively displayed in two different left and right areas of the above-mentioned fusion screen. Through the above-mentioned technical solution, not only can the above-mentioned on-board information and the above-mentioned train information be accurately initialized and displayed, but also the above-mentioned on-board information and the above-mentioned train information can be displayed simultaneously through the above-mentioned first display mode, so that users can view at any time.

[0048] Furthermore, by analyzing the vehicle-mounted information and the train information, the display mode of the fusion screen is obtained, including:

[0049] The control unit periodically acquires the train information of the train and receives the onboard information in real time, and when the train information is normal, the fusion screen displays the onboard information and the train information in the first display mode;

[0050] When an abnormality occurs in one of the vehicle-borne information and the train information, the abnormal vehicle-borne information or the train information is used as the first content, and the normal vehicle-borne information and the train information are used as the second content, and the fusion screen displays the first content and the second content in a second display mode, wherein the second display mode is that the fusion screen displays the first content in full screen, and displays the abnormal information in the first content in the form of a warning icon on top of the first content, and displays the second content in the form of a thumbnail at the top corner of the fusion screen, wherein the top corner positions corresponding to the vehicle-borne information and the train information are different when they are the second content;

[0051] Specifically, the control unit periodically obtains the train information of the train and receives the on-board information in real time, wherein the on-board information includes the operation level, driving mode, target distance, target speed, arrival information and information related to train control, and the train information includes the train's operation mode, operation speed, traction braking status, door status and train-related status information. When the on-board information and the train information are normal, that is, the on-board information is executed according to the normal operation plan, and the train's operation parameters are not abnormal, that is, the operation mode, operation speed, traction braking status, door status and train-related status information are all within the normal range, then the fusion screen displays the on-board information and the train information in the first display mode. When any one of the on-board information and the train information is abnormal, in order to facilitate the user to easily and prominently view the abnormality, when the on-board information is abnormal, The above-mentioned on-board information is taken as the above-mentioned first content. Similarly, when the above-mentioned train information is abnormal, the above-mentioned train information is taken as the above-mentioned first content, and the normal above-mentioned on-board information or normal train information is taken as the above-mentioned second content, and the above-mentioned first content and the above-mentioned second content are displayed in the above-mentioned second display mode through the above-mentioned fusion screen, that is, the above-mentioned first content is displayed in full screen on the above-mentioned fusion screen, and the abnormal information in the above-mentioned first content is displayed in the form of a warning icon on the upper layer of the first content, and the above-mentioned second content is displayed in the form of a thumbnail at the top corner of the above-mentioned fusion screen. At the same time, according to the difference of the above-mentioned second content, the display position of the thumbnail of the second content is also different. Through the above-mentioned technical solution, the contents of the two display screens are displayed through the fusion screen, which not only saves costs, but also the page content display mode and approximate content can be determined through the user's peripheral vision, thereby improving the user's driving safety and the convenience of obtaining on-board information and train information.

[0052] When abnormalities occur simultaneously in the on-board information and the train information, the fused screen displays the on-board information and the train information in a third display mode, and the third display mode is that the on-board information and the train information are respectively displayed in the left and right areas of the fused screen, wherein, when the priority of the first abnormal information in the on-board information is high, the warning icon corresponding to the first abnormal information is displayed on the top layer of the fused screen, and the warning icon corresponding to the second abnormal information in the train information is second, and the display position of the train information is away from the user, and the on-board information and the train information are displayed at the bottom layer of the fused screen, and vice versa.

[0053] Specifically, when abnormalities occur in both the on-board information and the train information at the same time, the fusion screen displays the on-board information and the train information in the third display mode. The third display mode is similar to the first display mode, except that there is no warning icon. In order to facilitate users to obtain more urgent abnormal information in time, the priorities of the first abnormal information and the second abnormal information are compared, and the first abnormal information or the second abnormal information with higher priority is displayed on the top layer of the fusion screen, and the first abnormal information or the second abnormal information with lower priority is displayed on the layer below the top layer, and on the layer above the on-board information and the train information. Through the above technical solution, when abnormalities occur in both the on-board information and the train information, it is convenient for users to view the abnormal information according to the priority of the abnormal information, and take timely measures to improve driving safety and efficiency.

[0054] Further, judging whether the predicted viewpoint position falls on the fusion screen by analyzing the running state of the train includes:

[0055] Taking a first image of the user by a camera unit, and obtaining, based on the first image, a viewpoint distance between the user's eyes and the camera unit and a viewpoint angle between a line connecting the user's eyes and the camera unit and a horizontal line when the user is facing forward, wherein the first image is a depth image;

[0056] Periodically collecting the viewpoint information of the user, the viewpoint information including the viewpoint position of the user and the corresponding collection time, and calculating the viewpoint trajectory of the user and the viewpoint change speed based on the viewpoint information of the user every first time period, and inputting the viewpoint distance of the user, the viewpoint angle, the viewpoint trajectory of the user in the first time period, and the viewpoint change speed of the user in the first time period into the viewpoint prediction model, obtaining the viewpoint prediction trajectory in a second time period after the first time period, and when the viewpoint prediction trajectory intersects the fused screen, the intersection point of the viewpoint prediction trajectory with the fused screen is used as the predicted viewpoint position;

[0057] The train information is used to determine whether the driving state of the train has changed. When the driving state of the train has changed, the predicted viewpoint position is on the fusion screen; when the driving state of the train has not changed, the predicted viewpoint position is not on the fusion screen, wherein the driving state includes traction, braking and coasting.

[0058] Specifically, when driving the train, users of different heights and body shapes have different corresponding viewpoint trajectories when viewing the fusion screen. In order to obtain a more accurate viewpoint prediction trajectory through the viewpoint prediction model, the driving image of the user is captured by the camera unit while the state data of the train remains unchanged for a preset time. Since the driving image is a depth image, the viewpoint distance between the user's eye and the camera unit is obtained through the pixel depth value corresponding to the user's eye center in the driving image, and the viewpoint angle between the line between the user's eye and the camera unit and the horizontal direction is obtained. The viewpoint distance and the viewpoint angle determine the viewpoint distance of different users and the same user in different seats. The different viewpoint trajectories of the above-mentioned fusion screen are viewed at different heights, and the viewpoint information of the above-mentioned user is periodically collected by the above-mentioned camera unit, and the viewpoint trajectory and viewpoint change speed of the above-mentioned user in the above-mentioned first time period are calculated based on the viewpoint information of the above-mentioned user at intervals of the above-mentioned first time period, and the viewpoint distance, viewpoint angle, and the viewpoint trajectory and viewpoint change speed of the user in the first time period are input into a viewpoint prediction model, wherein the above-mentioned viewpoint prediction model is a model trained with historical data, so as to obtain the viewpoint prediction trajectory in a second time period after the above-mentioned first time period, and when the above-mentioned viewpoint prediction trajectory intersects with the above-mentioned fusion screen, the intersection point of the above-mentioned viewpoint prediction trajectory with the above-mentioned fusion screen is used as the above-mentioned predicted viewpoint position. Since the above-mentioned fusion screen is generally arranged in front of the train operating console, when the user needs to check the track condition in front of the train or operate the control console controller, the viewpoint trajectory may pass through the above-mentioned fusion screen or overlap with the above-mentioned fusion screen. In order to accurately determine whether the above-mentioned viewpoint predicted viewpoint position is on the above-mentioned fusion screen, the train state information in the above-mentioned train information is used to judge whether the driving state of the above-mentioned train has changed, for example: traction or braking. When the driving state of the above-mentioned train has changed, it is considered that the above-mentioned viewpoint predicted position is on the above-mentioned fusion screen. When the driving state of the above-mentioned train has not changed, it is considered that the above-mentioned viewpoint predicted position is not on the above-mentioned fusion screen. Through the above-mentioned technical scheme, it is possible to accurately obtain whether the above-mentioned user's viewpoint predicted position is on the above-mentioned fusion screen, so as to facilitate displaying the content that the user wants to view before the user's viewpoint falls on the target viewpoint position of the above-mentioned fusion screen, thereby improving the user's viewing efficiency and improving driving safety.

[0059] Further, determining the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position, and viewing the display content of the fused screen, includes:

[0060] When the fusion screen is displayed in the first display mode, the display area in the vehicle information or the train information that is closest to the predicted viewpoint position is obtained as the target viewpoint according to the predicted viewpoint position, and when the viewpoint position of the user is less than a preset distance from the target viewpoint, the content details of the display area corresponding to the target viewpoint are displayed;

[0061] When the fusion screen is displayed in the second display mode, the warning icon on the fusion screen is used as the target viewpoint of the user, and when the viewpoint position of the user is less than the preset distance from the target viewpoint, the abnormal detailed information and the coping strategy corresponding to the warning icon are displayed, and when the viewpoint position of the user falls at the top corner thumbnail position of the fusion screen and the duration exceeds the first preset time, the display mode is switched to the first display mode and the vehicle information and the train information are displayed simultaneously;

[0062] When the fusion screen is displayed in the third display mode, when the user's viewpoint position is less than a preset distance from the display position of the warning icon with the highest priority, the abnormal detailed information and response strategies corresponding to the warning icon are displayed, and after continuing for a second preset time, the warning icon with the next highest priority is displayed. The method of this step is repeated until all the warning icons and the corresponding abnormal information and response strategies are displayed, and then the display mode is switched to the first display mode.

[0063] Furthermore, the user's viewpoint stays on the fused screen for a period of time that is less than or equal to a third preset time each time, and the first preset time is greater than the second preset time, and the third preset time is greater than the second preset time.

[0064] Specifically, the first display mode is a display mode when both the on-board information and the train information are normal. Therefore, it is only necessary to obtain the display area on the fusion screen that is closest to the predicted viewpoint position as the target viewpoint according to the predicted viewpoint position, and display the content details of the display area corresponding to the target viewpoint when the user's viewpoint position is at the preset distance from the target viewpoint, thereby reducing the user's waiting time and improving driving safety. The second display mode is a display mode when any one of the on-board information and the train information is abnormal. Since the warning icon corresponding to the abnormal information is the information that the user needs to view most urgently, the display area corresponding to the warning icon is used as the target viewpoint, and the abnormal detailed information and response strategies corresponding to the warning icon are displayed when the user's viewpoint position is at the preset distance from the warning icon. The display of the thumbnail can also be controlled by the user's viewpoint. , that is, converted into the above-mentioned first display mode. When the above-mentioned display mode is the above-mentioned third display mode, the viewing method of the above-mentioned warning icon with the highest priority is the same as the viewing method of the warning icon in the above-mentioned second mode, wherein the priority of the warning icon is consistent with the priority of its corresponding abnormal information, wherein the viewing time of each of the above-mentioned warning icons and the corresponding content continues for the above-mentioned second preset time, and then the above-mentioned warning icon with the next highest priority and the corresponding abnormal information and response strategy are automatically displayed, and after all the above-mentioned warning icons and the corresponding content are displayed, it is converted into the above-mentioned first display mode. For the driving safety of the user, the time for the above-mentioned user to view the above-mentioned fusion screen each time is less than or equal to the above-mentioned third preset time. Through the above-mentioned technical scheme, corresponding viewing methods can be provided according to different display modes, which improves the convenience of users viewing the fusion screen, and can fully grasp the train information and on-board information without operation, and also improves the driving safety of users.

[0065] Furthermore, the on-board information includes operating level, driving mode, target distance, target speed, arrival information and information related to train control, and the train information includes the train's operating mode, operating speed, traction braking status, door status and train-related status information.

[0066] Furthermore, the on-board signal system periodically receives and forwards the on-board information to the fusion screen through the Ethernet port of the train. When the fusion screen fails to receive the on-board information sent by the on-board signal system for more than a preset time, the communication between the fusion screen and the on-board signal system fails, the on-board information display range is displayed in black, and the fusion screen switches to the dual-channel RS422 serial port to communicate with the on-board signal system.

[0067] Specifically, the on-board signal system receives the on-board information through the Ethernet port of the train, and forwards the on-board information to the fusion screen through the communication unit of the train. When the fusion screen fails to receive the on-board information for the preset time, wherein the preset time is 1s, it is considered that a communication failure has occurred between the on-board signal system and the train. At this time, the display range of the on-board information displayed on the fusion screen is displayed in black, and a fault signal is sent to the on-board signal system through the dual-channel RS422 of the fusion screen. After the on-board signal system receives the fault signal, it communicates with the fusion screen through the dual-channel RS422 serial port. Through the technical solution, even if a communication failure occurs between the on-board signal system and the train, the communication between the on-board signal system and the fusion screen can be guaranteed.

[0068] The present invention also provides a display fusion system for urban rail transit, the system is used to implement the above method, such as Figure 2 As shown, the system comprises:

[0069] a control unit, configured to send a start signal to the on-board signal system when the train is powered on, and after the train receives the on-board information sent by the on-board signal system and the fusion screen is started, send the on-board information and the train information of the train to the fusion screen, and the fusion screen displays the on-board information and the train information in a first display mode;

[0070] An analyzing unit, configured to acquire a display mode of the fusion screen by analyzing the vehicle-mounted information and the train information;

[0071] A judgment unit, configured to predict a predicted viewpoint trajectory of the user according to the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and a viewpoint prediction model, and to judge whether a predicted viewpoint position in the predicted viewpoint trajectory falls on the fusion screen by analyzing the train information;

[0072] A viewing unit is configured to determine the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position when the predicted viewpoint position falls on the fused screen, and to view the display content of the fused screen.

[0073] The present invention also provides a computer-readable storage medium, on which instructions are stored, and the above method is implemented when the instructions are executed by a processor.

[0074] In summary, the present invention realizes the integrated display of information by integrating the vehicle-mounted information and the train information on a fusion screen, reduces the number of information sources that the driver needs to pay attention to during driving, reduces the complexity of information processing, and improves the efficiency of information acquisition. The system can predict the user's predicted viewpoint trajectory according to the user's viewpoint change trajectory, viewpoint change speed and viewpoint prediction model, determine the target viewpoint based on the predicted viewpoint trajectory, and dynamically adjust the display content to ensure that the driver can quickly obtain key information when needed, reduce the distraction caused by viewing multiple display screens, thereby improving driving safety. According to different vehicle-mounted information and train information states, the system can automatically switch the display mode to make the information display more It is intuitive, easy for the driver to quickly identify and respond, and improves the efficiency of information processing. By reducing the number of human-machine interfaces in the cab, it not only saves space, but also reduces construction and operation and maintenance costs, and achieves cost-effectiveness optimization. The system can dynamically adjust the display mode according to the driving status of the train and the user's predicted viewpoint position, so that the system can adapt to different operating environments and user needs, improving the adaptability and flexibility of the system. When the communication between the on-board signal system and the fusion screen fails, the system can automatically switch to the backup communication mode to ensure the continuous display of information and enhance the robustness of the system. The mutual cooperation of the above technical solutions can not only improve the user's viewing efficiency, but also improve the user's driving safety.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0076] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0077] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A display fusion method for urban rail transit trains, characterized in that: The method comprises: When the train is powered on, a start signal is sent to the on-board signal system through the control unit, and after the train receives the on-board information sent by the on-board signal system and the fusion screen is started, the on-board information and the train information of the train are sent to the fusion screen, and the fusion screen displays the on-board information and the train information in a first display mode; Acquire the display mode of the fusion screen by analyzing the vehicle-mounted information and the train information; Predicting the user's predicted viewpoint trajectory by using the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and viewpoint prediction model, and determining whether the predicted viewpoint position in the predicted viewpoint trajectory falls on the fusion screen by analyzing the train information; When the predicted viewpoint position falls on the fused screen, determining the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position, and viewing the display content of the fused screen; Wherein, judging whether the predicted viewpoint position falls on the fusion screen by analyzing the running state of the train includes: Taking a first image of the user by a camera unit, and obtaining, based on the first image, a viewpoint distance between the user's eyes and the camera unit and a viewpoint angle between a line connecting the user's eyes and the camera unit and a horizontal line when the user is facing forward, wherein the first image is a depth image; Periodically collecting the viewpoint information of the user, the viewpoint information including the viewpoint position of the user and the corresponding collection time, and calculating the viewpoint trajectory of the user and the viewpoint change speed based on the viewpoint information of the user every first time period, and inputting the viewpoint distance of the user, the viewpoint angle, the viewpoint trajectory of the user in the first time period, and the viewpoint change speed of the user in the first time period into the viewpoint prediction model, obtaining the viewpoint prediction trajectory in a second time period after the first time period, and when the viewpoint prediction trajectory intersects the fused screen, the intersection point of the viewpoint prediction trajectory with the fused screen is used as the predicted viewpoint position; The train information is used to determine whether the driving state of the train has changed. When the driving state of the train has changed, the predicted viewpoint position is on the fusion screen; when the driving state of the train has not changed, the predicted viewpoint position is not on the fusion screen, wherein the driving state includes traction, braking and coasting.

2. The method according to claim 1, characterized in that , the fusion screen displays in a first display mode based on the vehicle-mounted information and the train information, including: When the train is powered on, the start signal is sent to the on-board signal system through the control unit in the train. After receiving the start signal, the on-board signal system sends the on-board information corresponding to the train according to the identification information of the train in the start signal. After the control unit receives the on-board information and the start signal of the fusion screen, the on-board information and the train information are sent to the fusion screen and displayed in the first display mode, wherein the first display mode is displayed in two areas on the left and right, and the start signal includes the train identification information and the start instruction.

3. The method according to claim 1, characterized in that ,The target display page is selected through user viewpoint detection, including: The control unit periodically acquires the train information of the train and receives the onboard information in real time, and when the train information is normal, the fusion screen displays the onboard information and the train information in the first display mode; When an abnormality occurs in one of the vehicle-borne information and the train information, the abnormal vehicle-borne information or the train information is used as the first content, and the normal vehicle-borne information and the train information are used as the second content, and the fusion screen displays the first content and the second content in a second display mode, wherein the second display mode is that the fusion screen displays the first content in full screen, and displays the abnormal information in the first content in the form of a warning icon, and displays the second content in the form of a thumbnail at the top corner of the fusion screen, wherein the top corner positions corresponding to the vehicle-borne information and the train information are different when they are the second content; When abnormalities occur simultaneously in the on-board information and the train information, the fused screen displays the on-board information and the train information in a third display mode, and the third display mode is that the on-board information and the train information are respectively displayed in the left and right areas of the fused screen, wherein, when the priority of the first abnormal information in the on-board information is high, the warning icon corresponding to the first abnormal information is displayed on the top layer of the fused screen, and the warning icon corresponding to the second abnormal information in the train information is second, and the display position of the train information is away from the user, and the on-board information and the train information are displayed at the bottom layer of the fused screen, and vice versa.

4. The method according to claim 3, characterized in that , determining the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position, and viewing the display content of the fused screen, including: When the fusion screen is displayed in the first display mode, the display area in the vehicle information or the train information that is closest to the predicted viewpoint position is obtained as the target viewpoint according to the predicted viewpoint position, and when the viewpoint position of the user is less than a preset distance from the target viewpoint, the content details of the display area corresponding to the target viewpoint are displayed; When the fusion screen is displayed in the second display mode, the warning icon on the fusion screen is used as the target viewpoint of the user, and when the viewpoint position of the user is less than the preset distance from the target viewpoint, the abnormal detailed information and the coping strategy corresponding to the warning icon are displayed, and when the viewpoint position of the user falls at the top corner thumbnail position of the fusion screen and the duration exceeds the first preset time, the display mode is switched to the first display mode and the vehicle information and the train information are displayed simultaneously; When the fusion screen is displayed in the third display mode, when the user's viewpoint position is less than a preset distance from the display position of the warning icon with the highest priority, the abnormal detailed information and response strategies corresponding to the warning icon are displayed, and after continuing for a second preset time, the warning icon with the next highest priority is displayed. The method of this step is repeated until all the warning icons and the corresponding abnormal information and response strategies are displayed, and then the display mode is switched to the first display mode.

5. The method according to claim 4, characterized in that , the user's viewpoint stays on the fusion screen for a period of time that is less than or equal to a third preset time, and the first preset time is greater than the second preset time, and the third preset time is greater than the second preset time.

6. The method according to claim 1, characterized in that , the on-board information includes the operating level, driving mode, target distance, target speed, arrival information and information related to train control, and the train information includes the train's operating mode, operating speed, traction braking status, door status and train-related status information.

7. The method according to claim 1, characterized in that The on-board signal system periodically receives and forwards the on-board information to the fusion screen through the Ethernet port of the train. When the fusion screen fails to receive the on-board information sent by the on-board signal system for more than a preset time, the communication between the fusion screen and the on-board signal system fails, the on-board information display range is displayed in black, and the fusion screen switches to the dual-channel RS422 serial port to communicate with the on-board signal system.

8. A display fusion system for urban rail transit, the system being used to implement the method according to any one of claims 1 to 7, characterized in that: The system comprises: a control unit, configured to send a start signal to the on-board signal system when the train is powered on, and after the train receives the on-board information sent by the on-board signal system and the fusion screen is started, send the on-board information and the train information of the train to the fusion screen, and the fusion screen displays the on-board information and the train information in a first display mode; An analyzing unit, configured to acquire a display mode of the fusion screen by analyzing the vehicle-mounted information and the train information; A judgment unit, configured to predict a predicted viewpoint trajectory of the user according to the user's viewpoint distance, viewpoint angle, viewpoint change trajectory, viewpoint change speed and a viewpoint prediction model, and to judge whether a predicted viewpoint position in the predicted viewpoint trajectory falls on the fusion screen by analyzing the train information; A viewing unit is configured to determine the target viewpoint of the user based on the display mode of the fused screen and the predicted viewpoint position when the predicted viewpoint position falls on the fused screen, and to view the display content of the fused screen.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images