Intelligent display screen man-machine interaction system and engineering machinery

Abstract

The invention discloses an intelligent display screen man-machine interaction system and engineering machinery, and the system comprises a display screen display operation module which is used for achieving the state display and function operation of a whole vehicle; the physical key operation module is used for starting functions of the whole vehicle and comprises a whole vehicle function key panel, an auxiliary operation key panel, a rocker switch, a light adjusting panel and / or a windscreen wiper adjusting panel; the controller is used for collecting and processing complete machine information and making a decision, and the complete machine communication module is used for communication between the display screen and the controller and internal communication of the controller. According to the invention, a multi-modal interaction architecture is constructed, and through collaborative design of three modules, the problems of single display mode, lack of dynamic adaptation capability and poor environmental adaptability of a traditional engineering machinery interaction interface are solved; by developing a multi-channel interaction protocol stack, the cooperation of touch control and physical keys is realized, and the man-machine work efficiency and the operation safety of the engineering machinery are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of engineering machinery, and more specifically, relates to an intelligent display screen human-computer interaction system and engineering machinery. Background Art

[0002] As construction machinery evolves toward intelligent and digital capabilities, traditional human-machine interaction systems are no longer able to meet the operational demands of complex working conditions. Traditional construction machinery often relies on a combination of physical buttons and a single LCD screen. This results in a fragmented interface layout and complex functional hierarchy, making it prone to operator errors in critical situations and limiting the user interface.

[0003] The existing display screen has a single information display mode, lacks dynamic adaptation capabilities, and suffers from information overload and operation delays; the existing system mostly relies on manual operation, lacks real-time analysis and early warning functions of mechanical status (such as hydraulic pressure and engine load), and cannot actively push optimization suggestions (such as energy-saving mode switching and fault pre-diagnosis). Summary of the Invention

[0004] The purpose of the present invention is to address the above shortcomings and provide an intelligent display human-machine interaction system and engineering machinery, which combines a physical button panel, integrates multimodal interaction and real-time data processing technology, and improves the operating efficiency and safety of engineering machinery by optimizing the human-machine interface (HMI) design.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, the present invention provides a human-computer interaction system for an intelligent display screen, comprising: Display screen display operation module, used to realize vehicle status display and function operation; Physical button operation module, used to activate vehicle functions, including vehicle function button panel, auxiliary operation button panel, rocker switch, light adjustment panel and / or wiper adjustment panel; Controller, used to collect and process machine information and make decisions, The vehicle communication module is used for communication between the display screen and the controller and for internal communication within the controller.

[0006] Furthermore, the display screen display operation module includes a vehicle status monitoring unit, a physical button display unit, a fault diagnosis unit, an intelligent weighing unit, a user setting unit, a reversing image unit, a driving guide and a 3D visual manual.

[0007] Furthermore, the vehicle status monitoring unit is used to display the vehicle status and operating conditions, including the operating status of the working device, the engine status, the gearbox status and / or the host information; The physical button display unit is used to display whether the physical button is turned on; for example, when the hydraulic lock button is pressed, a hydraulic lock icon appears in the status prompt bar of the display screen; The fault diagnosis unit is used to display tire temperature and pressure, IO diagnostic status, engine fault information, transmission fault information and / or air conditioning fault information; The intelligent weighing unit is used to implement intelligent weighing operation functions, including weighing calibration, material selection, truck selection, weighing data printing and / or last bucket function; The user setting unit is used to implement user-defined operating parameters; it can realize the preference setting of multiple operators and can perform personalized settings; The reversing image unit is used to display the distance to obstacles around the vehicle and provide different levels of prompts; the reversing image system can be activated automatically after engaging reverse gear or manually; The driving instructions and 3D visual manual are used to provide vehicle information and operating guidelines, which can guide novice drivers to quickly master driving skills; specifically, the driving instructions include a guide for using the buttons, handles, panels, etc. in the cab, and by touching the virtual buttons on the display screen, you can learn how to operate and activate the button functions; by scanning the QR code on the display screen, you can view the 3D visual manual online, including vehicle technical specifications, safety instructions, operating guidelines, maintenance guidelines, and fault diagnosis, and obtain vehicle information more intuitively through 3D animation demonstrations.

[0008] Furthermore, the working device includes a bucket, a boom, an implement, etc., and the operating status includes the angle, opening, current, etc. of the working device; the IO diagnostic status includes DI, AI, DO, PWM port values, and pin status; the engine fault information includes SPN, FMI, OC, and fault description; the transmission fault information includes SPN, FMI, OC, and fault description; the last bucket is a weighing function. When the bucket weight is greater than the current target total weight (stack / truck), the last bucket function is turned on to shake the material to remove the difference between the current bucket weight and the target total weight.

[0009] Furthermore, the hardware structure of the display operation module includes a touch screen, a driving circuit, a touch sensor (capacitive / resistive), a graphics processing unit (GPU), and an interface (HDMI) with the controller, etc., and each component is physically connected through a wiring harness (including power supply, communication, and wake-up); The human-computer interaction software algorithm of the display screen display operation module includes user interface design, touch feedback algorithm, gesture recognition (such as sliding, zooming, and long pressing), multi-level menu logic, and dynamic data visualization (such as vehicle status and fault diagnosis).

[0010] Furthermore, the controller includes a main controller and a sub-controller; the vehicle communication module includes a main controller and sub-controller communication unit, and a main controller and display screen communication unit.

[0011] Furthermore, the main controller and the sub-controller communication unit communicate through the controller's CAN1 channel. The main controller serves as the decision-making core (coordination layer), and the sub-controller serves as the execution layer data acquisition node, and data interaction is achieved through the CAN1 bus. The main controller and display screen communication unit includes the CAN1 channel and CAN2 channel of the display screen and the CAN1 channel and CAN3 channel of the controller. A group of setting functions of the display screen display operation module communicate with the controller through the CAN1 channel of the display screen and the CAN3 channel of the controller. Another group of setting functions of the display screen display operation module communicate with the controller through the CAN2 channel of the display screen and the CAN1 channel of the controller. The physical button operation module communicates with the controller through the CAN3 channel of the controller. The main controller also communicates with other parts of the vehicle through the CAN2 channel and CAN4 channel of the controller, such as program download and steering system. The main controller collects and processes the whole machine information and transmits it to the display screen through the CAN channel. The display screen completes the visual display of the vehicle status and operation status.

[0012] Furthermore, the vehicle function button panel is used to select the desired engine power mode; it also includes operation calibration (such as boom bucket angle, mainly used for automatic lifting, automatic lowering, automatic bucket retraction, and automatic dumping functions), and the opening and closing of engine functions, transmission functions, and stability module functions. Among them, the engine functions include delayed engine shutdown, automatic shutdown, regeneration, and prohibition of regeneration, and the transmission functions include manual and automatic modes, power cut, etc. The stability module function includes real-time monitoring and active intervention when the construction machinery is operating (such as loading, lifting, and transporting by a loader) to prevent or reduce the risk of vehicle body roll, slippage, or even tipping, thereby improving operational safety and stability. The auxiliary operation button panel is used to realize intelligent weighing, auxiliary shoveling and custom shortcut macro instructions with one button; The rocker switch is used to turn on or off the lights and various functional modules with one click, including engine functions, transmission functions, emergency steering functions, etc., which can be operated through the touch screen or physical buttons, and customized configuration can be made according to actual needs; for example, the double flash switch, when the switch is pressed, the display screen shows the double flash icon, and the vehicle width indicator lights flash at the same time; The light adjustment panel includes gear positions for the position lights, low beam lights, front and rear working lights, left and right working lights, and rotating warning lights; The wiper adjustment panel is used to adjust the gear position of the wiper, including intermittent gear, low gear and high gear.

[0013] Furthermore, the vehicle's function key panel is a membrane key panel, comprising an elastic interlayer and a circuit layer. The elastic interlayer utilizes a composite structure of conductive and insulating silicone. Self-cleaning comb-shaped contacts are located at the base of the springs, with a surface coating thickness of ≥0.1μm. The circuit layer integrates a multi-point matrix scanning circuit and utilizes piezoelectric ceramics for dynamic adjustment of tactile feedback. The mechanical structure of the auxiliary operation key panel is identical to that of the vehicle's function key panel.

[0014] Furthermore, the rocker switch includes a base, a panel body and a rocker body, the base and the panel body are connected by a snap-on structure, and a dual redundant contact group is provided inside the rocker body; The wiper adjustment panel is integrated with a knob-type combination switch, in which a multi-stage toggle switch is provided; the knob-type combination switch is integrated with a Hall sensor, the magnetic field strength of which is linearly related to the knob angle, and outputs a digital signal through the SPI protocol.

[0015] In a second aspect, the present invention provides an engineering machine, which is equipped with the intelligent display human-computer interaction system described in the first aspect.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention builds a multimodal interaction architecture. Through the collaborative design of three modules (display display operation module / vehicle communication module / physical button operation module), it solves the problems of traditional engineering machinery interactive interfaces with a single display mode, lack of dynamic adaptation capabilities, and poor environmental adaptability. By developing a multi-channel interaction protocol stack, it achieves the coordination of touch and physical buttons, shortening the operation response time by 40% under complex working conditions. Field tests have verified that the system can reduce the operator error rate by 62%, supports stable operation in a wide temperature range of -40°C to 85°C, and significantly improves the ergonomics and operational safety of engineering machinery. The present invention deeply integrates the vehicle status monitoring system, fault diagnosis system, intelligent weighing system, user setting system, reversing image system, etc., and can realize the preference setting of multiple operators and customize the operation parameters. Combined with real-time monitoring display and graded early warning mechanism, it improves the operation accuracy and equipment maintenance efficiency, comprehensively enhances the operation safety, and ultimately realizes the intelligent upgrade of the whole process of construction machinery from status monitoring, function control to decision-making and maintenance. This invention uses a multi-CAN channel interactive vehicle communication module to achieve 10ms-level real-time communication between the main controller and the display screen. It also establishes a hierarchical architecture design of main and sub-controllers to ensure the real-time and stable transmission of key instructions. In combination with multiple function keys and algorithms, it can effectively cope with the harsh challenges of harsh working conditions such as mining and extreme cold. The physical key operation module of the present invention retains the physical key operation of core functions such as emergency braking and power mode switching through a layered layout design, and at the same time develops a composite membrane key panel with an IP67 protection grade, integrating intelligent weighing and status indication functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A structural diagram of a human-computer interaction system for an intelligent display screen provided by an embodiment of the present invention; Figure 2 A communication topology diagram of a human-computer interaction system for an intelligent display screen provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings and specific examples.

[0019] Example 1: This embodiment provides a human-computer interaction system for an intelligent display screen, such as Figure 1 As shown, it includes a controller, a display operation module, a vehicle communication module and a physical button module; The display screen display operation module is used to realize the vehicle status display and function operation; The physical key operation module is used to activate some functions of the vehicle, including the vehicle function membrane key panel, auxiliary operation membrane key panel, rocker switch, light adjustment panel and wiper adjustment panel; Controller, used to collect and process machine information and make decisions, The vehicle communication module is used for communication between the display screen and the controller and for internal communication within the controller.

[0020] The display screen display operation module described in this embodiment includes a vehicle status monitoring unit, a physical button display unit, a fault diagnosis unit, an intelligent weighing unit, a user setting unit, a reversing image unit, a driving guide and a 3D visual manual.

[0021] The vehicle status monitoring unit is used to display the vehicle status and operating conditions, including the operating status of the working device, engine status, gearbox status and host information; the working device can be a bucket, boom, machinery, etc., and the operating status includes the angle, opening, current, etc. of the working device.

[0022] The physical button display unit is used to display whether the physical button is turned on; for example, when the hydraulic lock button is pressed, a hydraulic lock icon appears in the status prompt bar of the display screen.

[0023] The fault diagnosis unit is used to display tire temperature and pressure, IO diagnostic status, engine fault information, transmission fault information and air conditioning fault information; the IO diagnostic status includes DI, AI, DO, PWM port values, and pin status; the engine fault information includes SPN, FMI, OC and fault description; the transmission fault information includes SPN, FMI, OC and fault description.

[0024] The intelligent weighing system is used to implement intelligent weighing operations, including weighing calibration, material selection, truck selection, weighing data printing, and a last bucket function. The last bucket is a weighing function that, when the bucket weight exceeds the current target total weight (stockpile / truck), triggers the last bucket function to shake out the material, removing the difference between the current bucket weight and the target total weight.

[0025] The user setting system is used to enable users to customize operating parameters; it can realize the preference settings of multiple machine operators and can perform personalized settings.

[0026] The reversing imaging system is used to display the distance of obstacles around the vehicle body and provide different degrees of prompts; the reversing imaging system can be activated in two ways: automatically triggered after engaging reverse gear and manually activated.

[0027] The driving instructions and 3D visual manual are used to provide vehicle information and operating guidelines, which can guide novice drivers to quickly master driving skills; specifically, the driving instructions include a guide for using the buttons, handles, panels, etc. in the cab, and by touching the virtual buttons on the display screen, you can learn how to operate and activate the button functions; by scanning the QR code on the display screen, you can view the 3D visual manual online, including vehicle technical specifications, safety instructions, operating guidelines, maintenance guidelines, and fault diagnosis, and obtain vehicle information more intuitively through 3D animation demonstrations.

[0028] In this embodiment, the controller includes a main controller and a sub-controller, and the vehicle communication module includes a main controller and sub-controller communication unit and a main controller and display screen communication unit.

[0029] like Figure 2As shown, the main controller and the sub-controller communication unit communicate through the controller's CAN1 channel. The main controller serves as the decision-making core (coordination layer), and the sub-controller serves as the execution layer data acquisition node, and data interaction is achieved through the controller's CAN1 channel. The main controller and the display screen communication unit include the display screen's CAN1 channel and CAN2 channel and the controller's CAN1 channel and CAN3 channel (the CAN channels marked in the figure are all controller's CAN channels), wherein a set of setting functions of the display screen display operation module communicates with the controller through the display screen's CAN1 channel and the controller's CAN3 channel (main controller IO diagnostic information), and the physical button operation module communicates with the controller through the controller's CAN3 channel. The display screen displays another set of setting functions of the operation module and communicates with the controller through the CAN2 channel of the display screen and the CAN1 channel of the controller (human-computer interaction, sub-controller IO diagnostic information); other components of the vehicle such as centralized lubrication, engine, gearbox, GPS, and operating handle communicate with the main controller through the CAN3 channel of the controller, and the air conditioner communicates with the main controller through the CAN1 channel of the controller; the main controller also communicates with other parts of the vehicle through the CAN2 channel and CAN4 channel of the controller, such as program download and steering system communicate with the main controller through the CAN2 channel of the controller; the main controller collects and processes the whole machine information, and transmits it to the display screen through the CAN channel. The display screen completes the visual display of the vehicle status and operation status.

[0030] In this embodiment, the vehicle function button panel is used to select the desired engine power mode; it also includes operation calibration (such as boom bucket angle, mainly used for automatic lifting, automatic lowering, automatic bucket retraction, and automatic dumping functions), and the opening and closing of engine functions, transmission functions, and stability module functions. Among them, the engine functions include delayed engine shutdown, automatic shutdown, regeneration, and regeneration prohibition, and the transmission functions include manual and automatic modes and power cutoff. The stability module function includes real-time monitoring and active intervention when the construction machinery is operating (such as loading, lifting, and transporting by a loader) to prevent or reduce the risk of vehicle body roll, slippage, or even tipping, thereby improving operational safety and stability. The auxiliary operation button panel is used to realize intelligent weighing, auxiliary shoveling and custom shortcut macro instructions with one button; The rocker switch is used to turn on or off the lights and various functional modules with one click, including engine functions, transmission functions, emergency steering functions, etc., which can be operated through the touch screen or physical buttons, and customized configuration can be made according to actual needs; for example, the double flash switch, when the switch is pressed, the display screen shows the double flash icon, and the vehicle width indicator lights flash at the same time; The light adjustment panel includes gear positions for the position lights, low beam lights, front and rear working lights, left and right working lights, and rotating warning lights; The wiper adjustment panel is used to adjust the gear position of the wiper, including intermittent gear, low gear and high gear.

[0031] Example 2: This embodiment provides a human-computer interaction system for an intelligent display screen. The technical solution is further refined on the basis of Example 1 to achieve better technical effects. For contents not described in detail, please refer to Example 1 and will not be repeated in this embodiment.

[0032] In this embodiment, the display screen operation module includes a hardware structure and a human-computer interaction software algorithm.

[0033] In this embodiment, the hardware structure of the display screen display operation module includes a touch screen, a driving circuit, a touch sensor (capacitive / resistive), a graphics processing unit (GPU), and an interface (HDMI) with the controller, etc. The various components are physically connected through a wiring harness (including power supply, communication, and wake-up).

[0034] In this embodiment, the human-computer interaction software algorithm of the display screen display operation module includes user interface design, touch feedback algorithm, gesture recognition (such as sliding, zooming, and long pressing), multi-level menu logic, and dynamic data visualization (such as vehicle status and fault diagnosis).

[0035] In this embodiment, the vehicle's functional membrane keypad panel comprises an elastic interlayer and a circuit layer. The elastic interlayer utilizes a composite structure of conductive and insulating silicone. The elastic interlayer features self-cleaning comb-shaped contacts at the bottom of the springs, with a surface coating thickness of ≥0.1μm. The circuit layer integrates a multi-point matrix scanning circuit and utilizes piezoelectric ceramics to achieve dynamic adjustment of tactile feedback. The mechanical structure of the auxiliary operation keypad panel is identical to that of the vehicle's functional keypad panel.

[0036] In this embodiment, the rocker switch includes a base, a panel body, and a rocker body. The base and the panel body are connected via a snap-on structure, and a dual redundant contact group is provided inside the rocker body.

[0037] In this embodiment, the wiper adjustment panel integrates a knob-type combination switch, in which a multi-level toggle switch is provided; the knob-type combination switch integrates a Hall sensor, whose magnetic field strength is linearly related to the knob angle, and outputs a digital signal through the SPI protocol.

[0038] Example 3: This embodiment provides an engineering machine, which is equipped with the smart display human-computer interaction system described in Example 1 or Example 2.

[0039] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0040] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0041] While various embodiments of the present invention have been described above, the foregoing description is intended to be illustrative and non-exhaustive, and is not intended to be limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those skilled in the art without departing from the scope and technical principles of the described embodiments, and such modifications and variations should be considered within the scope of the present invention.

Claims

1. A smart display screen human-computer interaction system, characterized in that: include: Display screen display operation module, used to realize vehicle status display and function operation; Physical button operation module, used to activate vehicle functions, including vehicle function button panel, auxiliary operation button panel, rocker switch, light adjustment panel and / or wiper adjustment panel; Controller, used to collect and process machine information and make decisions, The vehicle communication module is used for communication between the display screen and the controller and for internal communication within the controller.

2. The intelligent display screen human-computer interaction system according to claim 1, characterized in that: The display screen display operation module includes a vehicle status monitoring unit, a physical button display unit, a fault diagnosis unit, an intelligent weighing unit, a user setting unit, a reversing image unit, a driving guide and a 3D visual manual.

3. The intelligent display screen human-computer interaction system according to claim 2, characterized in that: The vehicle status monitoring unit is used to display the vehicle status and operating conditions, including the operating status of the working device, engine status, gearbox status and / or host information; The physical button display unit is used to display whether the physical button is turned on; The fault diagnosis unit is used to display tire temperature and pressure, IO diagnostic status, engine fault information, transmission fault information and / or air conditioning fault information; The intelligent weighing unit is used to implement intelligent weighing operation functions, including weighing calibration, material selection, truck selection, weighing data printing and / or last bucket function; The user setting unit is used to implement user-defined operation parameters; The reversing image unit is used to display the distance to obstacles around the vehicle body and provide different levels of prompts; the reversing image unit can be turned on automatically after engaging reverse gear or manually; The driving instructions and 3D visual manual are used to provide vehicle information and operating instructions.

4. The intelligent display screen human-computer interaction system according to claim 1, characterized in that: The hardware structure of the display screen display operation module includes a touch screen, a driving circuit, a touch sensor, a graphics processing unit and an interface with a controller; The human-computer interaction software algorithm of the display screen display operation module includes user interface design, touch feedback algorithm, gesture recognition, multi-level menu logic, and dynamic data visualization.

5. The intelligent display screen human-computer interaction system according to claim 1, characterized in that: The controller includes a main controller and a sub-controller, and the vehicle communication module includes a main controller and sub-controller communication unit and a main controller and display screen communication unit.

6. The intelligent display screen human-computer interaction system according to claim 5, characterized in that: The main controller and the sub-controller communication unit communicate through the controller's CAN1 channel; The main controller and display screen communication unit includes the CAN1 channel and CAN2 channel of the display screen and the CAN1 channel and CAN3 channel of the controller, wherein a group of setting functions of the display screen display operation module communicates with the controller through the CAN1 channel of the display screen and the CAN3 channel of the controller, another group of setting functions of the display screen display operation module communicates with the controller through the CAN2 channel of the display screen and the CAN1 channel of the controller, and the physical button operation module communicates with the controller through the CAN3 channel of the controller.

7. The intelligent display screen human-computer interaction system according to claim 1, characterized in that: The vehicle function button panel is used to select the desired engine power mode; it also includes operation calibration, opening and closing of engine functions, transmission functions and stability module functions; And / or, the auxiliary operation button panel is used to realize intelligent weighing, auxiliary shoveling and custom shortcut macro instructions with one button; And / or, the rocker switch is used to turn on or off the lights and various functional modules with one click; And / or, the light adjustment panel includes gear positions for position marker lights, low beam lights, front and rear work lights, left and right work lights, and rotating warning lights; And / or, the wiper adjustment panel is used to adjust the gear position of the wiper.

8. The intelligent display screen human-computer interaction system according to claim 7, characterized in that: The vehicle function key panel is a membrane key panel, including an elastic interlayer and a circuit layer. The elastic interlayer adopts a composite structure of conductive silicone and insulating silicone, and a self-cleaning comb-shaped contact is provided at the bottom of the spring. The contact surface coating thickness is ≥0.1μm. The circuit layer integrates a multi-point matrix scanning circuit and realizes dynamic adjustment of tactile feedback through piezoelectric ceramic sheets.

9. The intelligent display screen human-computer interaction system according to claim 7, characterized in that: The rocker switch includes a base, a panel body and a rocker body, wherein the base and the panel body are connected via a snap-on structure, and a dual redundant contact group is provided inside the rocker body; And / or, the wiper adjustment panel is integrated with a knob-type combination switch, in which a multi-stage toggle switch is provided; the knob-type combination switch is integrated with a Hall sensor, the magnetic field strength of which is linearly related to the knob angle, and outputs a digital signal through the SPI protocol.

10. An engineering machine, characterized in that: The engineering machinery is equipped with an intelligent display screen human-computer interaction system as described in any one of claims 1 to 9.

Images