Cross-model fusion remote control man-machine interaction interface system and switching method

Abstract

The invention relates to the technical field of remote control, and provides a cross-model fusion remote control man-machine interaction interface system and a switching method. The system comprises a remote console programmable controller and a touch screen controller which are in communication connection. The remote console programmable controller is used for acquiring and sending an instruction containing crane type, number and running state data; shore crane and field crane operation pages are arranged in the touch screen controller, and the corresponding operation pages can be automatically switched based on received instructions. The system can also be additionally provided with a data display terminal and a video data processing module to realize the functions of instruction interaction, login verification, video data acquisition and display and the like. Cross-type compatibility of the quay crane and the field crane is achieved through the integrated human-computer interaction interface system, multiple sets of human-computer interaction interface systems do not need to be independently deployed and maintained for different types, the equipment deployment and maintenance cost is reduced, the operation process is simplified, the learning cost of operators and the misoperation risk are reduced, and the remote control operation efficiency and safety are improved.

Description

Technical Field

[0001] This application relates to the field of remote control technology, and in particular to a cross-model integrated remote control human-machine interface system and switching method. Background Technology

[0002] In existing remote operating system human-machine interfaces (ROS-HMIs), each ROS-HMI is typically adapted to only one type of machine. For terminal users with multiple types of equipment (such as quay cranes and yard cranes), the core disadvantage is that it greatly limits operational flexibility, increases overall operational complexity and cost, and hinders the optimal allocation of resources. This forces users to deploy and maintain a separate, isolated ROS-HMI for each type of equipment. This not only leads to redundant investment and high maintenance burden, but also requires operators to constantly switch and adapt to different operating interfaces, reducing operational efficiency and increasing the risk of misoperation. Summary of the Invention

[0003] In view of this, this application provides a cross-model integrated remote control human-machine interface system and switching method, which can integrate the ROS-HMI systems of different equipment into the same control device, and control the operation of different types of equipment (such as yard cranes and quay cranes) through the same control device.

[0004] This application provides a cross-model integrated remote control human-machine interface system and switching method. The following describes this application from multiple aspects, and the embodiments and beneficial effects described below can be referenced interchangeably.

[0005] In a first aspect, this application provides a cross-model integrated remote control human-machine interface system, including:

[0006] A touchscreen controller is used to receive user instructions, wherein the user instructions include the selection of a target crane, including a quay crane or a yard crane;

[0007] The remote control programmable controller is configured to communicate with a touch screen controller and multiple cranes. The remote control programmable controller determines the target crane from the multiple cranes based on user instructions to obtain monitoring information corresponding to the target crane. The monitoring information includes the type, number, operating status data, current task information of the target crane, and remote control status information.

[0008] The remote control programmable controller controls the touch screen controller to display the operation page corresponding to the target crane based on the monitoring information. The operation page includes the monitoring information.

[0009] According to the embodiments of this application, the above-mentioned technical solution of this application has at least one of the following beneficial effects: by cooperating with the remote control programmable controller and the touch screen controller with dual-model operation page, the compatibility between the two models of quay crane and yard crane is realized, eliminating the need to deploy and maintain multiple isolated HMI systems for different models, completely avoiding redundant investment, and significantly reducing equipment deployment costs and subsequent maintenance burden.

[0010] The touchscreen controller can automatically switch between the quay crane and yard crane operation pages based on instructions. Operators do not need to frequently adapt to the operation interface of different models, which reduces the learning cost and cumbersome operation of interface switching. This not only improves the operator's proficiency and work efficiency, but also reduces the risk of misoperation caused by interface differences.

[0011] In one possible implementation of the first aspect described above, the touchscreen controller is further configured to receive numbered data and communicate with a remote control programmable controller corresponding to the numbered data.

[0012] According to the embodiments of this application, a touch screen controller can be connected to any remote control programmable controller on a remote control console, avoiding the limitation that a touch screen controller can only interact with one remote control programmable controller. Through this method, the touch screen controller can remotely operate a remote control console, thereby controlling the operation of the lifting equipment connected to that remote control console, effectively avoiding the need to assign a dedicated operator to each remote control console, and reducing the waste of manpower and resources.

[0013] In one possible implementation of the first aspect described above, a video processing module is also included, which is communicatively connected to the remote control programmable controller and the camera mounted on each crane, respectively, wherein the video processing module is used to acquire the video stream captured by the camera.

[0014] The remote control programmable controller is also used to acquire the video stream corresponding to the target crane from the video processing module based on user instructions, and control the touch screen controller to display the video stream.

[0015] According to the embodiments of this application, the above-mentioned technical solution of this application has at least the following beneficial effects: The remote control programmable controller, based on user instructions, acquires the video stream corresponding to the target crane from the video module and controls the touchscreen to display the video stream. This allows the operator to combine monitoring information and video data to understand the working status of the target crane in real time, adjust the working mode of the target crane in real time, reduce the probability of malfunctions, and promptly handle unexpected situations involving the target crane.

[0016] In one possible implementation of the first aspect above, the remote control programmable controller is further configured to acquire video switching data corresponding to the target crane, and control the video processing module to perform screen switching and zooming on the video stream corresponding to the target crane based on the video switching data, wherein the video switching data includes the type, number, position and screen switching mode of the target controller.

[0017] According to the embodiments of this application, the above-mentioned technical solution of this application has at least the following beneficial effects: the remote control programmable controller can automatically control the video processing module to display the video stream data corresponding to the target crane based on the video switching data, without the need for manual searching of the video stream running with the target crane from the video processing module, so that the operator can understand the working condition of the target crane in a timely manner.

[0018] In one possible implementation of the first aspect mentioned above, it further includes: a data display terminal, which is communicatively connected to a remote control programmable controller for storing and displaying monitoring information.

[0019] The remote control programmable controller switches the display page corresponding to the target crane based on the type of the target crane. The display page includes monitoring information.

[0020] According to the embodiments of this application, the above-mentioned technical solution of this application has at least the following beneficial effects: the data display terminal displays monitoring information in an intuitive page format, transforming abstract equipment data into visual charts and dynamic interfaces, enabling remote operators to quickly grasp the equipment status without professional programming knowledge, and significantly reducing the learning cost and risk of misoperation in remote operation.

[0021] As an information integration hub, the data display terminal can integrate multi-dimensional monitoring information from remote control programmable controllers, crane sensors, and other sources to form a unified view of equipment status. This solves the pain points of traditional systems where information is scattered and operators need to query across platforms, thereby improving operational efficiency.

[0022] The remote control programmable controller actively switches the corresponding display page based on the target crane type (quay crane / yard crane) to ensure that the monitoring information on the display page is fully matched with the characteristics of the currently operating equipment, avoiding information redundancy or missing key parameters caused by general pages, and ensuring adaptability to cross-type remote control scenarios.

[0023] Without the need for operators to manually search for or switch pages, the remote control programmable controller can automatically trigger page updates based on the machine type switching command, reducing manual intervention steps and shortening the preparation time for machine type switching and operation start-up. It is especially suitable for high-frequency switching scenarios of multi-machine collaborative operation in ports, improving the overall operation flow efficiency.

[0024] In one possible implementation of the first aspect above, the touchscreen controller is also used to obtain a login request, which is used to verify the operator's identity;

[0025] The remote-controlled programmable controller, based on monitoring information, controls the touchscreen controller to display the operation page corresponding to the target crane, including:

[0026] The remote control programmable controller controls the data processing terminal to verify login requests, and after successful verification, it retrieves monitoring information from the data processing terminal to control the touch screen controller to display the operation page corresponding to the target crane.

[0027] According to the implementation of this application, the above-mentioned technical solution of this application has at least the following beneficial effects: by obtaining login requests through the touch screen controller and verifying them through the data processing terminal, an access mechanism for identity verification and operation authorization is established to ensure that only legally authorized operators can access the system, thereby avoiding the safety risks of unauthorized personnel operating cranes from the source, ensuring the safety of personnel, goods and equipment in port heavy equipment operations, and meeting the safety and compliance requirements of industrial remote control systems.

[0028] After successful authentication, the remote control programmable controller acquires the monitoring information of the target crane and controls the touch screen to display the corresponding operation page, allowing the operator to view the monitoring information of the crane connected to the remote control programmable controller remotely, and promptly send control signals to the remote control programmable controller through the touch screen controller to control the target crane.

[0029] Secondly, this application provides a method for switching cross-model integrated remote control human-machine interface, including: receiving a user instruction, wherein the user instruction includes a selected target crane, and the target crane includes a quay crane or a yard crane.

[0030] The target crane is determined from multiple cranes based on user instructions, and monitoring information corresponding to the target crane is obtained. The monitoring information includes the type, number, operating status data, current task information, and remote control console status information of the target crane.

[0031] The touchscreen controller displays the operation page corresponding to the target crane based on the monitoring information, and the operation page includes the monitoring information.

[0032] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a cross-model integrated remote control human-machine interaction interface switching method disclosed in the second aspect and any possible implementation of the second aspect above.

[0033] Fourthly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement a cross-model integrated remote control human-machine interface switching method disclosed in the second aspect and any possible implementation thereof.

[0034] Fifthly, this application provides a computer program product, which includes: computer instructions that, when executed on an electronic device, cause the electronic device to perform a cross-model integrated remote control human-machine interface switching method disclosed in the second aspect and any possible implementation thereof.

[0035] The beneficial effects of the second to fifth aspects can be found in the first aspect and the beneficial effects of any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0036] Figure 1 This is a scene diagram illustrating the connection between the crane equipment and the remote control console in an embodiment of this application;

[0037] Figure 2 This is a schematic diagram of data transmission in a cross-model converged remote control HMI system as described in this application embodiment;

[0038] Figure 3 This is a diagram of the cross-model converged remote control HMI system architecture in the embodiments of this application;

[0039] Figure 4 This is an example diagram of the automatic control page of the quay crane in the embodiments of this application;

[0040] Figure 5 This is an example diagram of the spreader extension control page of the quay crane in the embodiments of this application;

[0041] Figure 6 This is an example image of a video page for a quay crane in an embodiment of this application;

[0042] Figure 7 This is a flowchart illustrating the connection between the touchscreen controller and the remote control programmable controller according to an embodiment of this application.

[0043] Figure 8 This is an initialization interface diagram of an embodiment of this application;

[0044] Figure 9 This is a data transmission diagram of an embodiment of this application;

[0045] Figure 10 This is a flowchart of the cross-model converged remote control HMI switching method according to an embodiment of this application;

[0046] Figure 11 This is a block diagram of the electronic device in the embodiments of this application;

[0047] Figure 12 This is a block diagram of a system-on-chip (SoC) in the embodiments of this application.

[0048] Figure label:

[0049] 100. Crane equipment; 200. Remote control console; 300. Editable remote control console controller; 400. Touch screen controller; 10. TLS Home button; 11. Hatch Cover button; 12. Ship sweeping status display bar; 13. Bay position display bar; 14. Truck operation area; 15. Container loading and unloading control area; 16. Lane display area; 17. Parking space display area; 18. Location display area; 19. Custom information area; 20. Work sequence display area; 21. Right menu bar; 31. TLS operation area; 32. Spreader mode switching area; 33. Container size selection area; 34. Spreader display area; 41. Top title bar; 42. Status display area; 43. TLS selection button; 411. Status indicator light; 412. Settings button; 413. Settings bar; 4131. Quantity definition bar; 50. Initial interface diagram; 51. Remote control console editor number input box. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] The following is an explanation of the terms used in the embodiments of this application.

[0052] Quay crane: A ship-to-shore crane (STS), commonly known as a quay crane or quay gantry crane. It resembles a large steel gantry that moves along quay tracks, with a cantilevered arm extending from the top, allowing it to cross over the ship's side and reach directly above the container ship's hatch. A lifting trolley travels back and forth on the cantilever to lift containers directly from the ship onto quay trucks, automated guided vehicles (AGVs), or straddle carriers, or vice versa. It is the most crucial loading and unloading equipment at the front line of a container terminal, determining the terminal's throughput efficiency.

[0053] Yard cranes: including rail-mounted gantry cranes (RMG) and rubber-tyred gantry cranes (RTG), are specifically used in container yards to complete the loading, unloading, handling, and stacking of containers between the yard and trucks.

[0054] Human-Machine Interface (HMI): An HMI is a dialogue window that connects operators with machines or processes. Its core task is to convert the internal operating data of the equipment into human-understandable graphics, text, alarms, and other information, and to transmit human commands back to the control system.

[0055] The technical problems to be solved by the embodiments of this application will be described below.

[0056] As described in the background technology section, the single-model adaptation mode means that a single system can only be adapted to one type of machine, making it incompatible with various equipment such as quay cranes and yard cranes at the terminal. This severely limits operational and scheduling flexibility and makes it difficult to meet the needs of multi-equipment collaborative operations. Consequently, the terminal needs to deploy and maintain independent systems for each type of equipment, resulting in redundant investments in hardware procurement and software development, as well as high long-term maintenance costs. At the same time, operators need to frequently switch between different systems and adapt to different operating interfaces, which reduces operational efficiency and significantly increases the risk of misoperation.

[0057] Therefore, to address the aforementioned issues, this application provides a cross-model integrated remote control human-machine interface system. This system includes a remote programmable controller and a touchscreen controller, with the remote programmable controller and touchscreen controller communicatively connected. The remote programmable controller acquires instructions regarding crane type, crane number, and crane operation data, and transmits these instructions to the touchscreen controller. The touchscreen controller then switches between the quay crane operation page and the yard crane operation page based on these instructions. By cooperating with the remote programmable controller and the touchscreen controller with dual-model operation pages, both quay crane and yard crane compatibility are achieved, eliminating the need to deploy and maintain multiple HMI systems separately, avoiding redundant investment and reducing deployment and maintenance costs. Furthermore, the touchscreen controller automatically switches operation pages according to instructions, reducing operator interface adaptation costs and operational complexity, thereby improving operational efficiency and reducing the risk of misoperation.

[0058] To better understand the inventive concept of this application, the remote control structure of the present invention will be described in detail below.

[0059] In some embodiments, each remote console is equipped with a remote console programmable controller, a data display terminal, a video processing module, and a touch screen controller.

[0060] The aforementioned remote control programmable controller is communicatively connected to the crane, the data display terminal, the video processing module, and the touch screen controller, so that the remote control programmable controller can not only interact with the crane programmable controller, but also interact with the data display terminal, the video processing module, and the touch screen controller.

[0061] When the remote control programmable controller receives monitoring data transmitted from the crane connected to it, the control data display terminal stores and displays the data. This allows the operator near the remote control console to intuitively understand the operating status data of the crane connected to it.

[0062] When the remote control programmable controller receives the video switching data transmitted from the target crane, it controls the video processing module to perform screen switching and zoom operations on the corresponding video stream data of the target crane, so that the operator near the remote control can intuitively understand the monitoring data of the crane connected to the remote control.

[0063] The touchscreen controller is deployed on a remote control console and can communicate with both local remote control programmable controllers and remote remote control programmable controllers.

[0064] Once the touchscreen controller is successfully connected to the remote control programmable controller, the remote control programmable controller will transmit the monitoring data and video data of the crane connected to it to the touchscreen controller, so that the operator using the touchscreen controller can understand the monitoring information and video stream of the crane and make adjustments to the crane quickly.

[0065] To better understand the inventive concept of this application, the following description, in conjunction with the accompanying drawings, details the application scenarios of the cross-model integrated remote control human-machine interface system and switching method of this invention.

[0066] refer to Figure 1 , Figure 1 A scene diagram illustrating the connection between a crane device and a remote control console according to an embodiment of the present invention is shown. Figure 1 As shown, the crane equipment 100 is communicatively connected to the remote control console 200, and transmits the corresponding crane number, crane type, and crane operating status data to the remote control console 200 via the connection. The remote control console 200 also transmits the collected control signals to the crane equipment 100 via the connection to regulate the operating status of the crane equipment 100. The crane equipment 100 can be a quay crane or a yard crane, and the remote control console 200 is equipped with a remote control programmable controller.

[0067] In actual operation, crane equipment 100 can actively or passively connect to remote control console 200. In the active connection mode, the remote control programmable controller of remote control console 200 sends remote control data to each crane device via User Datagram Protocol (UDP) communication. This remote control data includes console number, connection status, and whether jump-over is allowed. After receiving the remote control data, crane equipment 100 selects remote control console 200 from multiple remote control consoles based on the operation control data and its own work tasks, and sends a binding request to remote control console 200. Upon receiving the binding request, remote control console 200 establishes a connection with the equipment via Transmission Control Protocol (TCP) communication to confirm the binding relationship. Then, the crane equipment activates the ROS PLC slave station, establishing real-time bus communication. The work tasks include monitoring tasks, fault repair tasks, and semi-automatic tasks.

[0068] It should be noted that the jump platform indicates that the current crane equipment is connected to the remote control console.

[0069] The remote control console needs to send remote control data to all crane equipment in batches. The core requirement at this stage is rapid and wide-ranging notification. Using UDP, data can be sent to all cranes at once, eliminating the need to establish a separate connection for each device and significantly reducing communication overhead. This remote control data is status notification information; even if individual devices occasionally lose packets, they can be resent via subsequent UDP packets without affecting the overall process. Its priority is lower than rapid delivery.

[0070] After a remote control console is selected for the crane, a series of steps need to be completed, such as binding request, connection confirmation, activation of ROS PLC slave, and real-time bus communication. The core requirements at this stage are reliable data and stable connection. Among them, the binding request and connection confirmation are key one-to-one interactions. TCP's connection-oriented nature ensures that binding information is not lost or disordered, avoiding binding failure between the crane and the remote control console. Activating the ROS PLC slave and subsequent real-time bus communication involves the crane's work tasks. This data must be ordered, complete, and error-free. TCP's retransmission mechanism and flow control can ensure the reliable transmission of critical instructions and prevent operational accidents caused by data loss or out-of-order delivery.

[0071] Once the touchscreen controller is connected to the remote control, if the remote control is not connected to any crane, a passive connection can be established between the remote control and the crane.

[0072] In the passive connection mode, the aforementioned touchscreen controller can display the overall crane view of the remote control console. This overall crane view includes the number and connection status of each crane. The overall crane view further includes views of quay cranes and yard cranes. The operator can select the crane to connect on the touchscreen controller as needed. After capturing the point values ​​on the screen, the touchscreen controller obtains the crane number to be connected based on these values ​​and transmits the crane number to the remote control console's editable controller. Upon receiving the crane number corresponding to that crane, the remote control console's editable controller sends a connection signal to the crane's editable controller corresponding to that crane number, thus establishing a connection with the crane.

[0073] It should be noted that, in addition to the above methods, the crane number to be connected can also be obtained through voice input, etc. These methods are all within the scope of protection of this application, and will not be elaborated here.

[0074] Whether it's an active or passive connection, once the remote control console establishes a connection with the crane equipment, the crane equipment will transmit its corresponding crane type, crane number, and crane operating status data to the remote control console. The remote control console's programmable controller will then transmit this data to the data display terminal.

[0075] The following is combined with Figures 2 to 6 First, the cross-model integrated remote control human-machine interface system of this application will be described in detail.

[0076] refer to Figure 2 , Figure 2 This is a schematic diagram of data transmission in a cross-model converged remote control HMI system as described in this application embodiment. Figure 2 As shown, the cross-model integrated remote control HMI system includes a remote control programmable controller 300 and a touch screen controller 400, which are communicatively connected. The remote control programmable controller 300 is mounted on the remote control console. The touch screen controller 400 acquires user commands and transmits them to the remote control programmable controller 300. Based on the user commands, the remote control programmable controller 300 identifies the target crane from multiple cranes and obtains monitoring information corresponding to the target crane. This monitoring information includes the target crane's type, number, operating status data, current task information, and remote control console status information.

[0077] The remote control programmable controller 300 controls the touch screen controller 400 to display the operation page corresponding to the target crane based on the monitoring information, wherein the operation page includes the aforementioned monitoring information.

[0078] It should be noted that, due to the different types of cranes, the remote control programmable controller 300 will control the touch screen controller 400 to display different operation pages according to the type of crane it is connected to, including the quay crane operation page and the yard crane operation page.

[0079] refer to Figure 3 , Figure 3 This is a diagram of the cross-model converged remote control HMI system architecture in an embodiment of this application. Figure 3 As shown, the HMI system includes a target remote control programmable controller (ROS PLC) and a touchscreen controller. The touchscreen controller displays the overall view of the quay cranes and the overall view of the yard cranes. The quay crane view shows which quay cranes the remote control programmable controller can connect to, and the yard crane view shows which yard cranes the remote control programmable controller can connect to. The operator can select the crane to be connected on the touchscreen controller. After the remote control programmable controller establishes a connection with the crane, the operator can access the corresponding operation page for that crane.

[0080] Each operation page contains multiple sub-pages. The quay crane operation page includes the main page (Home), automatic control page (Auto), spreader extension control page (Spreader), video page (CCTV), and auxiliary interface (Auxiliary). The yard crane operation page includes the video page (CCTV) and the main interface (Home).

[0081] The design of each page for both quay cranes and yard cranes is consistent, including a top indicator bar, a bottom status bar (displaying time, connection status, number of runs, crane ID, and remote connection status), a right-side menu bar (for switching interfaces), and a core function operation area. To gain a deeper understanding of the technical solution of this invention, the various pages of both quay cranes and yard cranes will be described below.

[0082] The main page of the quay crane focuses on core operations such as anchoring, unanchoring, lifting adaptation, and motion control connection, while the main page of the yard crane focuses on load anti-sway and anti-torsion, wheel clamp control, and mode switching.

[0083] refer to Figure 4 , Figure 4 This is an example diagram of the automatic control page for the quay crane in this application embodiment. Figure 4As shown, the Auto interface of the quay crane revolves around automated operation processes. The Auto interface integrates a TLSHome button (10) and a Hatch Cover button (11). The central area of ​​the Auto interface includes a ship sweeping status display (12), a bay position display (13), a truck operation area (14), a container loading / unloading control area (15), a lane display area (16), a parking space display area (17), a location display area (18), a custom information area (19), a work sequence display area (20), and a right-side menu bar (21). The TLS Home button (10) controls the spreader tilting. The Hatch Cover button (11) selects the hatch cover mode. The ship sweeping status display (12) displays the ship's height and mode. The bay position display (13) enables target and current bay position settings and alignment / verification. The truck operation area (14) controls truck movement. The container loading / unloading control area (15) controls container loading and unloading. The lane display area (16) displays available truck lanes. The parking space display area (17) displays available truck parking locations. The location display area (18) displays the target / current location ID. The custom information area 19 allows users to customize content such as notes and precautions. The work sequence display area 20 displays the order in which tasks are performed. The right-hand menu 21 is used to switch between different pages.

[0084] refer to Figure 5 , Figure 5 This is an example diagram of the spreader extension control page of the quay crane in this application embodiment. Figure 5 As shown, the spreader extension control page of the quay crane is dedicated to spreader management, including the TLS operation area 31, the spreader mode switching area 32, the container size selection area 33, the spreader display area 34, and the right-side menu bar 21. The TLS operation area 31 is used to control the spreader tilt and rotation; the spreader mode switching area 32 is used to switch spreader modes; the container size selection area 33 is used to select the size of the container to be lifted; and the spreader display area 34 displays a schematic diagram of the spreader structure, reflecting its dimensions, extension status, weight, opening and closing, and key status such as container / guide plate engagement, enabling centralized management of spreader operation and monitoring. The right-side menu bar 21 is similar to the Auto interface description of the quay crane mentioned above and will not be repeated here.

[0085] It should be noted that the spreader extension control page has added a function to ensure that the spreader size remains consistent with the actual size. Simply inputting the spreader extension time will enable the spreader extension animation to be displayed, maintaining consistency with the actual spreader size and achieving visual synchronization. This function parses the current spreader size status word MtSpr_Size_FB and the target size command MtSpr_SizeWord_FB, combined with pre-configured spreader extension time parameters (including six sets of millisecond-level time bases such as Duration20_45 and Duration20_40), to drive precise displacement calculation for the spreader extension animation. The system first initializes the spreader display position SprDis based on the SprSize enumeration values ​​(0-6), establishing a size-position mapping relationship: 20ft corresponds to 16 pixels, 30ft to 119 pixels, 40ft / 2020ft to 214 pixels, and 45ft to 286 pixels. The system determines the displacement start point SprPosStart and end point SprPosFinal through 32 combinations of SprSize and SprSizeCmd states, and matches the corresponding Duration parameter.

[0086] The displacement engine calculates the total displacement SprDistance = SprPosFinal - SprPosStart, deriving the displacement velocity number = SprDistance / (Duration / 1000), in pixels per second. A SetInterval timer (1000ms cycle) updates the position value every second: value += number, writing it to the SprDis tag in real time to drive the animation. A SetTimeout controller is synchronously started, forcibly terminating the animation process after the Duration expires, precisely locating to the target value SprPosFinal, clearing the timer, and completing the displacement process. The diagnostic module outputs the displacement, velocity value, and real-time position coordinates via HMIRuntime.Trace, achieving a displacement accuracy of ±1 pixel and a time synchronization accuracy of ≤10ms. This solution decouples hard-coded position mapping from parameterized time control, supporting standard sizes of 20 / 30 / 40 / 45ft and dual 20ft conditions, with system response latency controlled within 50ms, ensuring strict consistency between visual display and actual physical dimensions.

[0087] By decoupling the hard-coded position mapping table from the kinematic parameters, millimeter-level visual synchronization of spreader size switching is achieved. It is applicable to all standard container sizes (20 / 30 / 40 / 45ft) and special working conditions with double 20ft, with a system response latency of <50ms.

[0088] refer to Figure 6 , Figure 6 This is an example image of a video page for a quay crane in an embodiment of this application. Figure 6 As shown, the video page of the quay crane is used to mark the location of monitoring points on the quay crane via crane structure icons for component status monitoring. It includes a top title bar 41, a status display area 42, a TLS selection button 43, and a right-side menu bar 21. The top title bar 41 has status indicator lights 411 whose quantity and content can be defined. Operators can configure the status indicator lights 411 by clicking the settings button 412 on the right. Specific settings are shown in the settings bar 413. Operators can set the number of status indicator lights 411 in the quantity definition bar 4131 and define the name of the currently configured status indicator lights 411 in the name bar 414. In addition to the above settings, background color, language settings, and HMI ID page settings can also be configured in the settings bar 413. For more details, please refer to [reference needed]. Figure 6 This will not be elaborated further here. The status display area 42 is used to display the working status of various parts of the quay crane. The TLS selection button 43 is used to switch the small handle that originally controlled the TLS to control the CCTV camera.

[0089] It should be noted that the video page for the yard crane can be found in the detailed introduction of the video page for the quay crane mentioned above, and will not be repeated here.

[0090] The unique Auxiliary interface of the quay crane is used to control the main switch of the quay crane and auxiliary equipment such as floodlights in various parts such as the overhang, pitch, trolley, gantry, and walkway.

[0091] Overall, these interface diagrams, through intuitive buttons, status displays, and monitoring views, collectively support the various operational controls and status monitoring required by quay cranes and yard cranes in automated lifting processes, ensuring efficient and safe operation of terminal operations.

[0092] It should be noted that the communication connection method is not limited in the embodiments of this application. There can be a variety of communication connection methods. For example, the communication connection method can be a wireless communication connection method such as ZigBee, LoRa, Wi-Fi, Bluetooth, etc., or a wired communication connection method such as fiber optic or power line.

[0093] By coordinating a remote control programmable controller with a touchscreen controller integrating dual-model operation interfaces, compatibility and adaptation for both quay cranes and yard cranes have been successfully achieved. This application eliminates the need to deploy and maintain multiple isolated HMI systems for different models, fundamentally avoiding redundant investment and significantly reducing initial deployment costs and subsequent maintenance burdens.

[0094] Meanwhile, the touchscreen controller can automatically switch between the quay crane and yard crane operation pages according to instructions, eliminating the need for operators to frequently adapt to the different machine models' interfaces. This design reduces the learning cost and operational tedium of interface switching, helping operators quickly improve their operational proficiency and work efficiency, while also effectively reducing the risk of misoperation caused by interface differences, further ensuring the stability and safety of operations.

[0095] refer to Figure 7 and Figure 8 , Figure 7 This invention illustrates a flowchart of the connection between a touchscreen controller and a remote control programmable controller according to an embodiment of the present invention. Figure 8 An initialization interface diagram of an embodiment of the present invention is shown.

[0096] like Figure 7 As shown, before the remote control programmable controller and the touch screen controller can interact with each other, they need to be connected. Specifically, this includes:

[0097] S110 obtains the serial number data through the touch screen controller.

[0098] S120, the touch screen controller establishes a communication connection between the remote control programmable controller corresponding to the numbered data based on the numbered data.

[0099] For reference Figure 8 The initial interface, Figure 50, includes a remote console editor number input box 51. The user-inputted remote console editor number (HMI-ID) tag value (range 1-3) is obtained through this input box, triggering the ConnectToPlc function to execute IP switching logic. The core process is as follows: After reading the HMI-ID value, a switch-case structure is used to match the corresponding ROS PLC IP address. Specifically, a value of 1 points to 10.45.22.11, a value of 2 points to 10.45.22.12, and a value of 3 points to 10.45.22.13. Simultaneously, the corresponding status code is written to the IP-FLAG tag. If an invalid value is entered, the connection is made to 192.168.1.1 by default, and IP-FLAG is marked as 99. Finally, the SysFct.ChangeConnection method dynamically updates the IP address of Connection_1 to the address selected by the user. This solution combines a hard-coded IP address mapping table with a dynamic connection interface to reduce the PLC switching operation response time to less than 100ms, while eliminating the need for engineering maintenance.

[0100] The initial interface's import and export buttons incorporate file writing and reading scripts. They write the user-inputted "HMI_ID" to a file and then read it back to ensure a connection to the correct ROS PLC. The WriteFileToHmi function handles user export operations: it reads the integer value of the HMI-ID tag, converts it to a string, and calls the FileSystem.WriteFile method to write the data to the fixed path / home / industrial / textfile.txt in UTF-8 encoding. Upon successful writing, it sets the ExportFlag=1 status flag and outputs the tracking code and data content; otherwise, it returns an error code. The ReadFileFromHmi function handles import and initialization operations: it uses a Promise asynchronous architecture to read a file at the same path. Upon successful parsing, it converts the text content to an integer and writes it to the HMI-ID tag, updates the ImportFlag=1 flag, and outputs the status code and text content; otherwise, it returns an error diagnosis. This mechanism ensures file operations are completed within 200ms by hard-coding file paths and using UTF-8 encoding, constructing a complete diagnostic chain and realizing closed-loop management from user input to PLC connection configuration. The configuration data remains valid after the system restarts by automatically reading the file.

[0101] It should be noted that the ConnectToPlc function mentioned above is the core communication connection function integrated into the human-machine interface. It is specifically used to establish a communication link between the software and the ROS PLC. It is the prerequisite for realizing data interaction between the two. Without the connection established by this function, the human-machine interface software cannot interact with the ROS PLC.

[0102] The IP-FLAG mentioned above is a custom IP address associated status flag in industrial control software / systems. It is not a standard term in general network protocols (such as TCP or IP), but an identification field designed for device communication configuration and status management. Its core is to mark the IP configuration status, connection attributes or adaptation rules related to ROS PLC communication through enumeration values.

[0103] The SysFct.ChangeConnection mentioned above is a connection switching or change function (or method) integrated in the System Function (SysFct, short for System Function) module. It is specifically used to dynamically change the communication connection configuration between the human-machine interface software and the ROS PLC when adapting to multiple models (quay crane or yard crane), switching multiple PLC devices, or adjusting communication parameters, so as to achieve seamless switching of connection targets and parameters. It is one of the core functions that support dual-model compatibility.

[0104] The FileSystem.WriteFile method mentioned above is the core file writing method encapsulated in the system file operation module (FileSystem). It is used to persistently write data (such as logs, configuration parameters, running records, etc.) to local files or storage media. It is the basic function for realizing system data retention and configuration saving. Simply put, it is the tool for software / system to "write data into files".

[0105] The ReadFileFromHmi function mentioned above is a file reading function integrated into remote control systems (such as DataView software or ROS PLC control programs). It is specifically used to read file data from the local storage or associated storage media of the HMI (Handheld Controller). As an operator interaction terminal, the HMI stores data such as configuration files, operation logs, and machine parameters. This function is the core tool for the upper-level system (or PLC) to obtain the file content from the HMI.

[0106] The aforementioned Promise asynchronous architecture is a standardized solution for handling asynchronous operations in programming languages ​​such as JavaScript (and is also widely used in other languages ​​or frameworks that support asynchronous programming). Its core is to separate the execution of asynchronous operations from the processing of results, and to manage the state and results of asynchronous tasks in an object-oriented way. This avoids the code mess caused by traditional nested callback functions (callback hell), making asynchronous logic clearer and more maintainable.

[0107] Simply put, a Promise is like a contract for an asynchronous task: it represents an operation that is not yet completed but will eventually have a result (success or failure), and allows you to predefine what to do when it succeeds and what to do when it fails, without blocking the main thread to wait for the result.

[0108] refer to Figure 9 , Figure 9 A data transmission diagram according to an embodiment of the present invention is shown.

[0109] like Figure 9As shown, in an embodiment of the present invention, one of the displays of the remote control console is equipped with data terminal application software for storing and displaying monitoring data received by the remote control console's programmable controller. Each remote control console's data display terminal interacts with its remote control console's programmable controller (ROS PLC) via an independent OPC interface (OLE for process control). This assists the remote control console in remotely controlling the crane. For example, depending on the crane model, different pages can be switched to display real-time fault information, crane real-time status information, remote control console status information, current task information, etc., and to perform tasks such as handling abnormalities. Furthermore, the data display terminal requires the remote control operator to log in with their own ID account to use it.

[0110] The login process of the data display terminal includes: the touch screen controller obtains the login request and transmits the login request to the remote control programmable controller, wherein the login request is used to verify the operator's identity.

[0111] After receiving the login request, the remote control programmable controller verifies the login request through the control data processing terminal. Once the verification is successful, it retrieves the monitoring information from the data processing terminal to control the touch screen controller to display the operation page corresponding to the target crane.

[0112] It should be noted that when using the touchscreen controller (MTP), a secure connection must first be established with the data display terminal through a login authentication process. When the operator enters login credentials on the MTP to initiate a login request, this request is sent to the data display terminal via the ROS PLC for identity verification and authorization checks. After successful verification by the data display terminal, the operator is authorized to access the system. The ROS PLC forwards information obtained from the data display terminal, such as the types of devices that the currently authorized operator can connect to, device IDs, and crane operating status data, to the MTP. The MTP receives and utilizes this device identification and status information to dynamically switch and display the corresponding device's operation page, providing the operator with an operational context and status display. The MTP relies on the data display terminal for initial login authorization and continuous operation permission management, and drives the interface display based on the device information provided by the data display terminal, thus forming a secure and top-down perceptive operating environment.

[0113] like Figure 9As shown, the remote control console also includes a video processing module, which can be equipped with closed-circuit television (CCTV) monitoring system software. This CCTV monitoring system software is client-side video processing software that can receive video stream signals from CCTV cameras installed at various locations on the crane (including quay cranes and yard cranes). The video processing module can communicate with the remote control console's programmable controller (ROS PLC), allowing for data exchange between them to assist the remote control console in remotely controlling the crane. For example, when the remote control console is connected to a crane (quay crane or yard crane), the crane's programmable controller (Crane PLC) sends information such as the crane's type, number, location, and screen-switching mode to the ROS PLC via bus communication. The ROS PLC then forwards this information to the remote control console's video data processing module via Ethernet communication, enabling video operations such as screen switching and zooming. The real-time operating screen of the quay crane or yard crane is then displayed on the remote control console's monitor.

[0114] The remote-controlled programmable controller can also acquire the video stream corresponding to the target crane from the video processing module based on user commands, and control the touch screen controller to display the video stream.

[0115] It's important to note that the MTP provides operators with a crucial window into the context of on-site video monitoring. Through the ROS PLC, the MTP can receive and display core status information from the video processing module in real time, particularly the device type and its unique identifier corresponding to the current video monitoring screen, as well as the screen switching status (screen-switching mode), along with the relevant crane operating status. This allows operators to clearly understand the crane's operational video information captured by the cameras mounted on the crane on the MTP display. Therefore, the MTP becomes a vital bridge connecting operator commands with the on-site monitoring screen, ensuring that the operator's intentions and the monitored footage are perfectly aligned when remotely controlling the equipment, thus significantly improving the safety and accuracy of remote operation.

[0116] The following section provides a detailed explanation of the cross-model integrated remote control human-machine interface switching method of this application.

[0117] refer to Figure 10 , Figure 10 A flowchart of the cross-model integrated remote control HMI switching method according to an embodiment of the present invention is shown.

[0118] like Figure 10 As shown, this application provides a method for switching cross-model integrated remote control human-machine interface, applied to the aforementioned cross-model integrated remote control human-machine interface system, including:

[0119] S130, Receive user instructions, wherein the user instructions include the selected target crane, which may be a quay crane or a yard crane.

[0120] S140: Based on user instructions, determine the target crane from multiple cranes to obtain monitoring information corresponding to the target crane. The monitoring information includes the type, number, operating status data, current task information, and remote control console status information of the target crane.

[0121] S150 controls the touchscreen controller to display the operation page corresponding to the target crane based on monitoring information, wherein the operation page includes monitoring information.

[0122] It should be noted that when the touchscreen controller communicates with the remote programmable controller, it typically uses the S7 protocol, which is based on Ethernet. The touchscreen controller will display the corresponding device's operating interface based on the type of currently connected device fed back by the remote programmable controller.

[0123] The interface content transmitted from the touchscreen controller to the remote control programmable controller includes: heartbeat signals, active connection operation commands, automatic equipment operation commands (such as anti-torsion, anti-shake, and automatic box-mounting), basic control commands (such as pitch, trolley anchoring, control activation, and bypass), and remote connection information (such as video data processing module device information, bypass permissions, and login / logout). The aforementioned heartbeat signals are used to detect whether the touchscreen controller and the remote control programmable controller are operating normally.

[0124] It should be noted that the above-mentioned control engagement refers to the engagement of the main contactor on the crane equipment that powers the drive.

[0125] The interface content transmitted between the remote control programmable controller and the touch screen controller includes: heartbeat signals, connection status and waiting time of all devices (for operators to select connected devices), the type of currently connected device (such as a rail-mounted gantry crane or quay crane) and its number, execution feedback information of single-machine control commands, and partial operating status of the crane; crane equipment information, such as spreader dimensions and guide plate status, user-defined information (mainly used for custom indicator lights in the title bar), spreader speed parameters (written with the spreader extension / retraction time to achieve synchronization between the spreader on the screen and the actual device), CCTV monitoring system software login information, and video data processing module device information. The heartbeat signals mentioned above will not be repeated here.

[0126] The beneficial effects of the above methods can be found in the section on cross-level converged remote control HMI systems mentioned above, and will not be elaborated here.

[0127] Therefore, compared with existing technologies, the cross-model integrated remote control human-machine interface system and switching method provided in this application innovatively integrates the monitoring and operation functions of yard cranes and quay cranes into a remote control HMI system, dynamically controlling the model and ID displayed on the touch screen through ROSPLC. This system grants remote operators the authority to flexibly select and configure any operating equipment from any remote control console. This solution significantly reduces the R&D and debugging costs of multi-model adaptation and brings core value to terminal users: enabling flexible and universal hardware equipment in different operating scenarios and seamless personnel turnover, effectively reducing redundant investment in equipment and manpower, and comprehensively improving the automation and intelligence level of the terminal, achieving simple and efficient operation.

[0128] This application provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. When the processor loads and executes the instruction or program, the electronic device performs the cross-model integrated remote control human-machine interface switching method described in the above embodiments. Its specific functions and corresponding technical effects can be found in the above embodiments. Figures 1-10 The cross-model integrated remote control human-machine interface system and switching method explained earlier will not be repeated here. The following section will combine... Figure 11 The electronic devices described in the embodiments of this application will be described in detail.

[0129] refer to Figure 11 The diagram shows a block diagram of an electronic device 1200 according to one embodiment of this application. The electronic device 1200 may include one or more processors 1201 coupled to a controller hub 1203. In at least one embodiment, the controller hub 1203 communicates with the processor 1201 via a multi-branch bus such as a front side bus (FSB) 1210, a point-to-point interface such as a quick path interconnect (QPI), or a similar connection. The processor 1201 executes instructions controlling general types of data processing operations. In one embodiment, the controller hub 1203 includes, but is not limited to, a graphics memory controller hub (GMCH) (not shown) and an input / output hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0130] Electronic device 1200 may also include a coprocessor 1202 and a memory 1204 coupled to a controller hub 1203. Alternatively, one or both of the memory and the GMCH may be integrated within the processor (as described in this application), with memory 1204 and coprocessor 1202 directly coupled to processor 1201 and controller hub 1203, which resides on a single chip with the IOH. Memory 1204 may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of both. In one embodiment, coprocessor 1202 is a dedicated processor, such as, for example, a high-throughput MIC (many integerized core) processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose computing on GPU (GPGPU), or an embedded processor, etc. Optional properties of coprocessor 1202 are indicated by dashed lines. Figure 11 middle.

[0131] As a computer-readable storage medium, memory 1204 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. For example, memory 1204 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device such as one or more hard-disk drives (HDDs), one or more compact disc (CD) drives, and / or one or more digital versatile disc (DVD) drives.

[0132] In one embodiment, electronic device 1200 may further include a network interface controller (NIC) 1206. Network interface 1206 may include a transceiver for providing a radio interface for electronic device 1200 to communicate with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, network interface 1206 may be integrated with other components of electronic device 1200. Network interface 1206 can implement the functions of the communication unit in the above embodiments.

[0133] Electronic device 1200 may further include input / output (I / O) device 1205. I / O device 1205 may include: a user interface designed to enable a user to interact with electronic device 1200; a peripheral component interface designed to enable peripheral components to also interact with electronic device 1200; and / or sensors designed to determine environmental conditions and / or location information related to electronic device 1200.

[0134] It is worth noting that, Figure 11 This is merely an example. That is, although... Figure 11 The electronic device 1200 shown includes multiple devices such as a processor 1201, a coprocessor 1202, a controller hub 1203, and a memory 1204. However, in practical applications, devices using the methods of this application may include only a portion of the devices in the electronic device 1200. For example, it may include only the processor 1201 and the network interface 1206. Figure 11 The properties of the optional devices are shown in dashed lines. According to some embodiments of this application, the memory 1204, which is a computer-readable storage medium, stores instructions or programs that, when executed on a computer, perform the cross-model integrated remote control human-machine interface switching method described in the above embodiments. For details, please refer to the methods in the above embodiments, which will not be repeated here.

[0135] Now for reference Figure 12 The diagram shown is a block diagram of a system-on-chip (SoC) 1300 according to an embodiment of this application. Figure 12 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 12 In this SoC 1300, the following are included: an interconnect unit 1350 coupled to an application processor 1310; a system proxy unit 1380; a bus controller unit 1390; an integrated memory controller unit 1340; a group or one or more coprocessors 1320, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1330; and a direct memory access (DMA) unit 1360. In one embodiment, the coprocessor 1320 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0136] The static random access memory (SRAM) cell 1330 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of those instructions. These instructions may include, when executed by at least one unit in the processor, causing the SoC 1300 to perform the cross-model fusion remote control human-machine interface switching method according to the above embodiments. Specific details can be found in the methods of the above embodiments and will not be repeated here.

[0137] This application provides a computer-readable storage medium storing at least one instruction or at least one program. The instruction or program is loaded and executed by a processor to implement the cross-model integrated remote control human-machine interface switching method described in the above embodiments. Its specific functions and corresponding technical effects can be referred to the above embodiments. Figures 1-10 The method for switching between cross-model integrated remote control human-machine interfaces, as explained above, will not be repeated here.

[0138] This application provides a computer program product, including: computer instructions, which, when executed on an electronic device, cause the electronic device to implement the cross-model integrated remote control human-machine interface switching method described in the above embodiments. Its specific functions and corresponding technical effects can be found in the above embodiments. Figures 1-10 The cross-model integrated remote human-machine interface system and switching method explained herein will not be repeated here.

[0139] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0140] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0141] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0142] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0143] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0144] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0145] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0146] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0147] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A cross-model integrated remote control human-machine interface system, characterized in that, include: A touchscreen controller for receiving user instructions, the user instructions including a selected target crane, the target crane being a quay crane or a yard crane; The remote control programmable controller is configured to communicate with the touch screen controller and multiple cranes. The remote control programmable controller determines the target crane from the multiple cranes based on the user instructions to obtain monitoring information corresponding to the target crane. The monitoring information includes the type, number, operating status data, current task information, and remote control status information of the target crane. The remote control programmable controller controls the touch screen controller to display the operation page corresponding to the target crane based on the monitoring information, and the operation page includes the monitoring information.

2. The system according to claim 1, characterized in that, The touchscreen controller is also used to receive numbered data and communicate with the remote control programmable controller corresponding to the numbered data.

3. The system according to claim 1, characterized in that, Also includes: The video processing module is communicatively connected to the remote control programmable controller and the camera mounted on each of the cranes, and is used to acquire the video stream captured by the camera. The remote control programmable controller is also used to acquire a video stream corresponding to the target crane from the video processing module based on the user instructions, and control the touch screen controller to display the video stream.

4. The system according to claim 3, characterized in that, The remote control programmable controller is also used to acquire video switching data corresponding to the target crane, and control the video processing module to perform screen switching and zooming on the video stream corresponding to the target crane based on the video switching data. The video switching data includes the type, number, location and screen switching mode of the target crane.

5. The system according to claim 1, characterized in that, Also includes: The data display terminal is communicatively connected to the remote control programmable controller and is used to store and display the monitoring information; The remote control programmable controller controls the data display terminal to switch to the display page corresponding to the target crane based on the type of the target crane. The display page includes the monitoring information.

6. The system according to claim 5, characterized in that, The touchscreen controller is also used to obtain a login request, which is used to verify the operator's identity. The remote control programmable controller controls the touchscreen controller to display the operation page corresponding to the target crane based on the monitoring information, including: The remote control programmable controller controls the data processing terminal to verify the login request, and after the verification is successful, it obtains the monitoring information from the data processing terminal to control the touch screen controller to display the operation page corresponding to the target crane.

7. A method for switching cross-model integrated remote control human-machine interface, characterized in that, include: Receive user instructions, the user instructions including the selected target crane, the target crane including a quay crane or a yard crane; Based on the user instruction, the target crane is determined from multiple cranes to obtain monitoring information corresponding to the target crane. The monitoring information includes the type, number, operating status data, current task information, and remote control console status information of the target crane. Based on the monitoring information, the touch screen controller is controlled to display an operation page corresponding to the target crane, and the operation page includes the monitoring information.

8. An electronic device, characterized in that, include: processor; and a memory, in which computer program instructions are stored. Wherein, when the computer program instructions are executed by the processor, the processor performs the method of claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of claim 7.

10. A computer program product, characterized in that, Includes a computer program / instruction that, when processed, implements the method of claim 7.

Images