Intelligent display system, control method and device of intelligent display system

Abstract

The application relates to the technical field of industrial control, and discloses an intelligent display system, a control method and device of the intelligent display system; the system comprises an intelligent display and a display group server; the intelligent display sends a preprocessed face image of an operator to the server after collecting the face image; the server completes identity matching by comparing the face image with a user library through an image analysis unit; corresponding operating system and industrial control software permission configuration data are called by a system distribution and login unit, and a system image preinstalled with the industrial control software is matched; finally, the image and the configuration data are sent to intelligent display client software, interface rendering and permission display are completed, and the application can automatically distribute operating systems, industrial control software configurations and corresponding operating permissions.

Description

Technical Field

[0001] This application relates to the field of industrial control technology, and more specifically, to an intelligent display system, a control method for the intelligent display system, and an apparatus. Background Technology

[0002] In industrial control scenarios, maintenance, scheduling, and operation personnel rely on smart displays to carry out core tasks. Through smart displays, they can view the operating parameters and equipment status of the industrial site in real time, and complete operations such as industrial control program configuration, parameter adjustment, command issuance, and fault diagnosis. As a key terminal for interaction between personnel and the industrial control system, smart displays support the visualization and precise execution of various industrial control operations.

[0003] In existing industrial control technologies, multi-user shared smart displays are already being used. The same smart display or group of smart displays is not exclusively used by a single worker. Personnel with different positions and responsibilities can take turns logging in and performing related industrial control operations on the smart display, thereby achieving the sharing and reuse of smart display hardware.

[0004] However, existing multi-user shared intelligent display technology has some drawbacks: the operating permissions of centralized control personnel are tied to their own management scope, and due to the rigid limitations of their work area, their operations are confined to fixed industrial control computers and associated displays. They cannot flexibly switch operating terminals or adapt to different work scenarios, resulting in insufficient flexibility in the entire industrial control operating system and making it difficult to meet the ever-changing operational needs of industrial sites. Therefore, it is urgent to solve this technical problem. Summary of the Invention

[0005] In view of the above, this application provides an intelligent display system, a control method and apparatus for the intelligent display system, which aims to solve the above problems or at least partially solve the above problems.

[0006] In a first aspect, embodiments of this application provide an intelligent display system, the system comprising: at least one intelligent display and a display cluster server; The smart display includes: The face detection unit is used to capture the facial image of the current display operator; preprocess the facial image to obtain preprocessed data and send it to the display cluster server; The system client software is used to receive and parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, load the corresponding operating system running environment, perform interface rendering calculations, generate the rendered digital interface image signal, and send it to the display unit. The display unit is used to receive and convert the digital interface image signal to display the corresponding operating system and pre-installed industrial control software interface. The display cluster server includes: The system server software uses its internal virtualization technology to build multiple operating system images, each of which has an industrial control software client installed. The user database is used to store and configure user information; the user information includes at least: the operator's facial image, terminal operating system configuration data, and industrial control software permission configuration data; The image data analysis unit is used to receive the preprocessed data and match it with user information in the user database to generate a matching result; The system allocation and login unit is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data and industrial control software permission configuration data to the system client software.

[0007] Secondly, embodiments of this application also provide a control method for an intelligent display system, the control method being deployed on a display cluster server of the intelligent display system as described in the first aspect, the control method comprising: Acquire the facial image of the operator currently displaying the monitor; The facial image is matched with user information in the user database to generate a matching result; If the matching result is successful, the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator are retrieved from the user database. The system client software of the smart display is then sent to the system client software of the display group, which matches the corresponding operating system image based on the terminal operating system configuration data. The system client software parses the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, loads the corresponding operating system runtime environment, performs interface rendering calculations, generates the rendered digital interface image signal, and sends it to the display unit.

[0008] Thirdly, embodiments of this application also provide a control device for an intelligent display system, the control device being deployed on a display cluster server of the intelligent display system as described in the first aspect, the control device comprising: The acquisition module acquires the facial image of the operator currently displaying the screen. The user matching module is used to match the facial image with user information in the user database and generate matching results. The system allocation module is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data, and industrial control software permission configuration data to the system client software of the smart display. This allows the system client software to parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, load the corresponding operating system runtime environment, perform interface rendering calculations, generate the rendered digital interface image signal, and send it to the display unit.

[0009] Fourthly, embodiments of this application also provide a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the steps of the control method for the intelligent display system as described in the second aspect.

[0010] Using the above technical solutions, the intelligent display system, control method, and apparatus provided in this application embodiment acquire facial images of the current display operator through a face detection unit, perform preprocessing, and transmit the preprocessed data to an image data analysis unit. The image data analysis unit matches the facial image with the operator's facial images stored in the user database to obtain a matching result. Meanwhile, the system server software, using internal virtualization technology, pre-builds multiple operating system images with industrial control software clients installed. After successful matching, the system allocation and login unit retrieves the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current operator, and then matches the corresponding operating system image based on the terminal operating system configuration data. The operating system image, terminal operating system configuration data, and industrial control software permission configuration data are then sent together to the system client software. The system client software receives and parses the relevant data, loads the corresponding operating system runtime environment, performs interface rendering calculations, generates digital interface image signals, and transmits them to the display unit. Finally, the display unit converts the signals and displays the corresponding operating system and pre-installed industrial control software interface. As can be seen, the complete technical solution provided in this embodiment relies on facial recognition to accurately identify the operator's identity. It eliminates the need for repeated permission configuration when multiple positions share display devices. It can automatically complete the configuration of the operating system and industrial control software and the allocation of corresponding operation permissions based on the identified current user. This effectively solves the technical problems of low configuration freedom of monitors and industrial control computers and complex user login permission management in the existing industrial control environment. It breaks the situation where the operation behavior of central control personnel is restricted by fixed industrial control computers and supporting monitors, allowing central control personnel to flexibly switch operating terminals and adapt to different work scenarios, greatly improving the flexibility of the industrial control operating system.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram of the structure of the intelligent display system provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating the control method of the intelligent display system provided in an embodiment of this application is shown. Figure 3A flowchart illustrating a control method for an intelligent display system according to another embodiment of this application is shown; Figure 4 This paper shows a schematic diagram of the structure of the control device of the intelligent display system provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0016] As mentioned earlier, existing multi-user shared intelligent display technology has some drawbacks: the operating permissions of centralized control personnel are tied to their own management scope, and due to the rigid limitations of their work scope, their operations are restricted to fixed industrial control computers and associated displays. They cannot flexibly switch operating terminals or adapt to different work scenarios, resulting in insufficient flexibility in the entire industrial control operating system and difficulty in meeting the ever-changing operational needs of industrial sites. Based on this, this invention proposes an intelligent display system, a control method for the intelligent display system, and a device for it. The following detailed description of specific embodiments further illustrates this application.

[0017] Figure 1 This paper shows a schematic diagram of the structure of an intelligent display system provided in an embodiment of this application. Figure 1 As can be seen, the intelligent display system provided in this application embodiment includes: at least one intelligent display 101 and a display cluster server 102; The smart display 101 includes: The face detection unit 1011 is used to acquire the facial image of the current display operator; preprocess the facial image to obtain preprocessed data and send it to the display cluster server 102; The system client software 1012 is used to receive and parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server 102, load the corresponding operating system running environment, perform interface rendering calculations, generate the rendered digital interface image signal, and send it to the display unit 1013. The display unit 1013 is used to receive and convert the digital interface image signal to display the corresponding operating system and pre-installed industrial control software interface. The display cluster server 102 includes: The system server software 1021 uses its internal virtualization technology to build multiple operating system images, and each operating system image has an industrial control software client installed. User database 1022 is used to store and configure user information; the user information includes at least: the operator's facial image, terminal operating system configuration data, and industrial control software permission configuration data; The image data analysis unit 1023 is used to receive the preprocessed data and match it with the user information in the user database 1022 to generate a matching result; The system allocation and login unit 1024 is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database 1022 if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data and industrial control software permission configuration data to the system client software 1012.

[0018] As can be seen, the intelligent display system provided in this application embodiment acquires the facial image of the current display operator through a face detection unit, performs preprocessing, and transmits the preprocessed data to an image data analysis unit. The image data analysis unit matches the facial image with the operator's facial images stored in the user database to obtain a matching result. Meanwhile, the system server software, with the help of internal virtualization technology, pre-builds multiple operating system images with industrial control software clients installed. After successful matching, the system allocation and login unit retrieves the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current operator, and then matches the corresponding operating system image based on the terminal operating system configuration data. The operating system image, terminal operating system configuration data, and industrial control software permission configuration data are sent together to the system client software. After receiving and parsing the relevant data, the system client software loads the corresponding operating system runtime environment, performs interface rendering calculations, generates digital interface image signals, and transmits them to the display unit. Finally, the display unit converts the signals and displays the corresponding operating system and pre-installed industrial control software interface. As can be seen, the complete technical solution provided in this embodiment relies on facial recognition to accurately identify the operator's identity. It eliminates the need for repeated permission configuration when multiple positions share display devices. It can automatically complete the configuration of the operating system and industrial control software and the allocation of corresponding operation permissions based on the identified current user. This effectively solves the technical problems of low configuration freedom of monitors and industrial control computers and complex user login permission management in the existing industrial control environment. It breaks the situation where the operation behavior of central control personnel is restricted by fixed industrial control computers and supporting monitors, allowing central control personnel to flexibly switch operating terminals and adapt to different work scenarios, greatly improving the flexibility of the industrial control operating system.

[0019] The following is combined with Figure 1 This will further explain the structure of the aforementioned intelligent display system.

[0020] See Figure 1 As shown in the embodiments of this application, the intelligent display system includes: multiple intelligent displays and a display cluster server.

[0021] The smart display includes a face detection unit, system client software, and a display unit. Specifically, the face detection unit can be a front-facing camera of the smart display, used to preprocess the facial image of the operator currently using the display. The preprocessed data is then sent to the display cluster server for identity verification. In practice, for example, the face detection unit acquires the operator's original facial image, sequentially performs irrelevant background removal, image noise reduction, lighting deviation correction, facial key point alignment, and size standardization, then compresses and encodes the image to obtain standardized preprocessed image data, which is then uploaded to the display cluster server to provide compliant input data for subsequent identity verification.

[0022] The system client software is configured to receive and parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data transmitted by the display group server, load the corresponding operating system operating environment, perform interface rendering calculation, generate a rendered digital interface image signal, and transmit the signal to the display unit. The processing flows of data receiving and parsing, operating environment loading, interface rendering calculation, and interface image signal distribution executed by the above system client software can be implemented with reference to existing conventional technical solutions.

[0023] The display unit is configured to receive and convert the digital interface image signal, and display the corresponding operating system and pre-installed industrial control software interface.

[0024] It is understandable that the smart display further includes a communication unit, which is used for fast data communication between the smart display and the display group server as well as other systems.

[0025] The display group server includes system server software, a user library, an image data analysis unit, and a system allocation and login unit.

[0026] The system server software constructs multiple sets of operating system images through its internal virtualization technology, and each set of operating system images is installed with an industrial control software client to ensure real-time response of interfaces and functions.

[0027] The user library is configured to store and configure user information; the user information includes: facial images of operators, terminal operating system configuration data, and industrial control software permission configuration data. The terminal operating system configuration data is related control data for standardizing the operating parameters, environment configuration and basic service scheduling of the smart display terminal system, so as to ensure the stable and standardized operation of the terminal system; the industrial control software permission configuration data is constraint data of operation scope, function access and usage permission set for the industrial control software supporting the equipment, which is used to divide the software operation boundaries and usage permissions of different operators. In some embodiments, the user information further includes user identity information.

[0028] The image data analysis unit is configured to receive preprocessed data, match the preprocessed data with the user information in the user library, and generate a matching result. Specifically, similarity calculation is performed between the preprocessed data and the facial images of operators stored in the user library. If there is user information with a similarity greater than a preset similarity threshold, the matching is successful and the user identity is determined to be legal; if the matching is unsuccessful, a feedback instruction of "identity verification failure" is generated and transmitted back to the smart display for interface prompting.

[0029] The system allocation and login unit is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data and industrial control software permission configuration data to the system client software of the smart display screen.

[0030] Understandably, the display cluster server also includes a communication unit, which works in conjunction with the communication unit of the smart display to achieve high-speed data transmission between systems.

[0031] In some embodiments, the user information also includes industrial control software and third-party system accounts and passwords; the system allocation and login unit is further used for: Retrieve the user information corresponding to the current monitor operator from the user database; In response to receiving a login request from an associated system, the user information is processed using preset verification rule data to generate an account legitimacy verification result; Based on the account validity verification result, a login execution command is generated; Based on the login execution command, automatic login to the associated system is completed.

[0032] In this embodiment, the associated system refers to the software system that interacts with or needs to be accessed by the current intelligent display system in the industrial control environment. Specifically, it includes industrial control software such as programming configuration software and monitoring software of field control systems, as well as third-party systems such as database systems. These systems also require independent input of account and password to log in.

[0033] The preset verification rule data is a set of logical rules used to determine whether user information (token set) is valid. For example, the preset verification rule data includes: account password consistency verification rule, which verifies whether the account password in the retrieved user information belongs to the preset account password set of the associated system; account validity verification rule, which verifies whether the account in the user information is in a normal and usable state and has not been canceled or frozen; and account permission matching verification rule, which verifies whether the user account has the operation permission to access the current associated system.

[0034] In practice, after receiving a login request from the associated system, the system allocation and login unit extracts the account and password corresponding to the industrial control software and third-party system from the retrieved user information. The extracted account and password information is then substituted into the preset verification rule data to perform verification calculations. The consistency of the account and password, the validity of the account, and the matching of permissions are checked in turn. Based on the passing status of all verification items, the legality of the account is determined, and finally, the account legality verification result is generated.

[0035] Then, based on the account validity verification result, a branch judgment is made. If the verification result is valid, an execution instruction for automatic login to the associated system is generated, and the corresponding instruction is then executed. If valid, a login session for the associated system is created directly to complete the automatic login.

[0036] During implementation, if the verification result is invalid, an execution instruction is generated to redirect to the user login interface, redirecting the operation interface to the manual account and password login interface of the associated system.

[0037] In this embodiment, the system allocation and login unit can retrieve the industrial control software and third-party system accounts and passwords of the current display operator from the user database. Then, it verifies the user information using preset verification rules to generate an account legitimacy verification result. Subsequently, it generates a corresponding login execution command based on the verification result. Relying on the user database to uniformly store user-related information, the system can automatically complete the account login verification and login operation for the corresponding system and software without manual input of account and password after the user's facial recognition is completed. This effectively simplifies the operation steps for switching logins between multiple systems. Ultimately, this embodiment provides a simpler and more efficient login process for multiple industrial control software systems in industrial control scenarios, achieving single sign-on.

[0038] In practical applications, the adjustment of industrial control system parameters mainly relies on manual, experience-based operation. However, this method depends on subjective experience, resulting in inconsistent adjustment standards and poor control accuracy. Furthermore, human response is delayed, failing to adapt to dynamic changes in operating conditions in a timely manner; insufficient operational standardization may also cause fluctuations in equipment operation. Therefore, in some embodiments of this application, the intelligent display further includes a computational macro unit; the display group server further includes a rule base for storing industrial control parameter adjustment rules; the industrial control parameter adjustment rules include trigger conditions and corresponding computational macro programs. The industrial control software in the display cluster server is used to collect real-time operating condition parameter data of the industrial control system and send it to the computing macro unit. The computational macro unit includes a control unit, used for: The real-time operating condition parameter data is matched with the trigger conditions in the rule base to generate matching result data; Based on the matching result data, the corresponding calculation macro program is invoked; The real-time operating condition parameter data is processed using the aforementioned computational macro program to generate parameter adjustment data; the parameter adjustment data is then loaded into the associated control interface. Based on the user's confirmed operation instructions, generate the final control instruction data; The final control command data is sent to the industrial control system through the command sending module.

[0039] In this embodiment, real-time operating condition parameter data refers to the current operating status values ​​collected by field devices of the industrial control system, such as level gauges, pressure sensors, and flow meters. Specifically, the signal detection unit of the industrial control software can periodically read the digital signals on the PLC or fieldbus and convert them into standardized data structures such as floating-point numbers or integer values ​​to obtain real-time operating condition parameter data.

[0040] The triggering condition is a pre-defined logical judgment expression in the rule base used to match operating condition parameters, such as "the liquid level drop rate is greater than 2 centimeters per second". In this embodiment, the execution subject reads rules containing comparison operators, thresholds and logical combinations from the rule base in the form of structured records such as database table rows or JSON objects.

[0041] A computational macro is a predefined parameter adjustment logic or script that is called when a trigger condition is matched to calculate recommended parameter values, i.e., parameter tuning data. In specific implementation, the execution entity in this embodiment loads the executable code corresponding to the successfully matched trigger condition, such as a Python script, Lua function, or compiled dynamic link library, from the rule base into memory for execution.

[0042] The associated control interface refers to a graphical window on a smart display used to show and allow users to modify parameter data. Specifically, based on the current industrial control software and the parameter being adjusted, the corresponding interface definition file (such as an XML layout or front-end component) can be dynamically loaded to generate an interactive window object.

[0043] In practice, firstly, the control unit of the computational macro unit receives real-time operating condition parameter data from the industrial control software and temporarily stores it as the current state variable; secondly, it reads the trigger conditions of all industrial control parameter adjustment rules from the rule base and sorts them according to a preset priority; then, it iterates through each trigger condition, comparing the real-time parameter value with the comparison items in the condition one by one, such as determining whether the liquid level change rate is greater than the threshold or whether the temperature is out of range; finally, if a trigger condition is fully satisfied (all sub-conditions are true), it generates result data containing matching rule identifiers such as rule ID and a matching success flag; if all conditions are not satisfied, it generates an empty matching result.

[0044] Extract the matching rule ID from the matching result data; find the corresponding operation macro program storage path or function name in the rule base based on the rule ID; then load the operation macro program into memory, initialize the interpreter environment if it is a scripting language, and directly obtain the function pointer if it is a compiled binary function; finally, save the successfully loaded program handle or reference in the context object for the next step of execution.

[0045] Then, using real-time operating condition parameter data as input parameters, the loaded computational macro program is called to perform calculations, resulting in a set of parameter adjustment data, such as the valve opening value and pump frequency value. Based on the current industrial control scenario and parameter type, the identifier of the associated control interface to be opened is determined, such as the window name or URL. Then, through the interface management interface provided by the industrial control software, the corresponding control window is dynamically generated or refreshed, and the parameter adjustment data is automatically filled into the corresponding input controls in the window, such as numerical boxes and sliders. Finally, the control interface is displayed on the smart display and activated to await user interaction.

[0046] Finally, in response to the user's confirmation operation command input through the display interface, the final control command data is generated; the sending interface provided by the command issuing module is called to pass in the final control command data and the address information of the target industrial control system; the command issuing module converts the command data into data frames on the network or bus according to the communication protocol of the target system; then, the data frames are sent to the designated field controller through a communication unit such as an Ethernet card or fieldbus interface.

[0047] In some embodiments, for the final control command data, user input events such as key presses, mouse wheel movements, or touch swipes on all editable controls on the associated control interface can be continuously monitored, and the temporary control parameter object in memory can be updated whenever the value changes; the click event of the confirmation button can be monitored, and when the user clicks confirmation, the confirmation operation data, including the confirmation timestamp and user identifier, can be captured; then, the value in the current temporary control parameter object is merged with the confirmation operation data, and the source display information is added; finally, according to the instruction format required by the industrial control system, such as Modbus command structure or OPC UA write request, these data are encapsulated into byte sequences or structured messages to form the final control command data.

[0048] Optionally, it can also receive execution confirmation messages returned by the industrial control system and record the sending status and feedback results in the log for subsequent tracking.

[0049] This embodiment proposes an intelligent control method for industrial control parameters. It relies on a rule base of a display cluster server to centrally store industrial control parameter adjustment rules, uniformly incorporating trigger conditions and computational macro programs. This provides a standardized execution basis for condition judgment and data computation. The industrial control software collects industrial control system operating parameters in real time and pushes them synchronously to the computational macro unit, ensuring that computational processing is based on real-time dynamic operating data from the field. The computational macro unit matches and identifies the received real-time parameters with the built-in trigger conditions in the rule base, accurately locating the operating scenario requiring parameter adjustment and calling the appropriate computational macro program. Standardized computational logic generates adjustment data and automatically loads it into the corresponding control interface, simplifying manual operation processes and reducing the difficulty of parameter adjustment. Combined with user confirmation, it generates compliant final control commands, integrating the advantages of automatic predictive adjustment and manual review. The command issuance module then reliably issues and executes the control commands. Ultimately, this embodiment achieves a more stable, accurate, efficient, and flexible control effect for industrial control system operating parameters, significantly shortening the response cycle.

[0050] During implementation, in addition to rules for adjusting industrial control parameters, the rule base can also store user permission rules. These rules define the operational boundaries of different roles in the system. For example, ordinary operators are prohibited from modifying any industrial control parameters to prevent accidental operation. When a user needs to access a third-party system, the system will verify whether their job permissions match the access requirements of the corresponding system. For high-risk operations involving equipment safety or significant production impact, only administrator accounts are allowed to execute them after an additional unlocking process. At the same time, the system will automatically assign default permission ranges based on the user's job position (e.g., monitoring staff can only view data, and maintenance staff can adjust regular parameters but not high-risk parameters). All permission verifications are performed in the logic operation unit to ensure that each instruction meets the pre-set constraints before it is issued.

[0051] In some embodiments, the computational macro unit further includes an operation recording module, used for: Collect raw data of operational steps input by personnel when responding to changes in operating parameters; Trigger condition feature data and operation step feature data are extracted from the original data of trigger conditions corresponding to changes in operating parameters and the original data of operation steps. Based on the trigger condition feature data and the operation step feature data, industrial control parameter adjustment rules are generated and stored in the rule base.

[0052] In this embodiment, in order to construct a rule base, the original data of triggering conditions and operation steps when manually responding to changes in operating parameters are first collected.

[0053] The trigger condition raw data refers to the set of working condition parameters when a person encounters a change in specific working condition parameters (such as the rate of liquid level drop in a container exceeding 3 cm per second), including parameter type, rate of change, current value, etc. For example: the liquid level drop rate is 3.5 cm / second, the current liquid level is 0.8 meters, and the water supply valve opening is 30%.

[0054] The raw data of the operation steps refers to a series of interface operations and parameter input actions that are performed by the operator in order to solve the change in the working condition. For example: first open the water supply valve adjustment interface, change the opening setting value from 30% to 70%, then open the water pump frequency setting interface, and change the frequency from 40Hz to 25Hz.

[0055] In practice, the operation recording module in the computational macro unit continuously monitors user operation events on the industrial control software interface. Through hook technology or event callback mechanism, it captures working condition snapshots when parameter changes are triggered, records the numerical sequence entered by the user on the control and the interface jump path, and captures the click behavior of the confirmation button and the associated verification feedback results. All data is temporarily stored in memory in the form of structured logs or time series key-value pairs.

[0056] Next, trigger condition feature data and operation step feature data are extracted from the original trigger condition data and operation step data. The core of this step is to clean and generalize the original data, removing details irrelevant to individual users and specific times, and retaining key features that can be used for rule matching and parameter generation. Trigger condition feature data is an abstract condition extracted from the original data to describe the pattern of changes in operating conditions. For example, "the liquid level drops from 1.0 meter to 0.5 meter in 10 seconds" is abstracted as "the rate of liquid level drop > 0.05 m / s". Operation step feature data is an ordered action template extracted from a series of specific operations, such as "open interface A -> set parameter X to a certain variable -> open interface B -> set parameter Y to another variable", where the original specific numerical value (e.g., 70%) is replaced by parameter variables (e.g., the increase in opening degree).

[0057] Finally, based on the trigger condition feature data and operation step feature data, industrial control parameter adjustment rules are generated and stored in the rule base. This step assembles the above two types of feature data into a rule record that can be executed by the computational macro unit. In specific implementation, the trigger condition feature data can be the conditional expression of the rule, the operation step feature data can be the execution body of the rule, where the original numerical positions are replaced with parameter placeholders, and the confirmation logic feature data can be the pre-execution verification script of the rule. Then, the two are encapsulated into an executable rule object, such as a JSON structure or a lightweight rule language script. Finally, the storage interface of the rule base is called to persist the object to the database or configuration file, and a unique rule ID and version number are assigned to it for subsequent matching and calling by the computational macro unit.

[0058] In other embodiments, when multiple adjustments (e.g., two or more, possibly from the same user at different times or different users) encounter the same trigger condition raw data, multiple raw parameter adjustment records with the same trigger condition feature data can be identified first. Secondly, a weight is assigned to the parameter adjustment operation steps (e.g., opening setpoint, frequency setpoint) and confirmation logic corresponding to each record. This weight is determined based on the historical accuracy of the parameter adjustment feedback loop (e.g., users with high accuracy or records with good adjustment results have higher weights). Then, a weighted average is calculated for the same control parameter (e.g., valve opening value) to obtain the fused parameter value. Simultaneously, for discrete features such as the interface call order in the operation steps, majority voting or weighted voting can be used to determine the fused operation flow. Finally, the fused trigger condition features, operation step features, and confirmation logic features are combined into a comprehensive industrial control parameter adjustment rule and stored in the rule base, thereby achieving intelligent fusion and iteration of multi-user experience.

[0059] In some embodiments, the user information further includes user identity information; the display cluster server further includes a logic operation unit, used for: Based on the user identity information and logged-in user information corresponding to the final control instruction data, permission validity determination data is generated. If the permission validity determination data is valid, the parameter tuning data received from the control unit will be input into the virtual controller of the industrial control software to generate a logic simulation result. If the logic simulation result is that the simulation is successful, the logic simulation result is sent to the control unit. After receiving the logic simulation result, the control unit sends the final control command data to each industrial control system through the command sending module. If the logic simulation result is abnormal, an error alarm is generated and sent to the smart display to issue an alarm.

[0060] In this embodiment, firstly, based on the user identity information and logged-in user information corresponding to the final control command data, permission validity determination data is generated. Here, user identity information is basic identification information that can uniquely distinguish different operators and represent personnel identity attributes, and may include personnel-specific identification identifiers, job categories, operation identity levels, and other data.

[0061] Logged-in user information refers to the user information currently recorded by the display cluster server for users who have completed identity verification and maintained a session on the smart display.

[0062] In practice, the logic operation unit receives the final control command data from the control unit, parses its message header or specified fields to obtain the user's identity information; at the same time, it queries the currently logged-in user information based on the smart display ID; it compares whether the two are completely consistent; if they are consistent and the user has the authority to perform parameter tuning operations, it generates the judgment data of valid authority; otherwise, it generates the judgment data of invalid authority.

[0063] If the authorization is deemed valid, the received parameter tuning data from the control unit is input into the virtual controller of the industrial control software to generate a logic simulation result. Here, the virtual controller refers to a software simulation environment provided by the industrial control software that simulates the logic of an actual PLC or DCS controller. It is not connected to a real physical device and can safely receive input parameters and simulate execution effects. In specific implementation, after confirming the authorization, the logic operation unit extracts the parameter tuning data from the message sent by the control unit; locates the virtual controller instance corresponding to the target device or process section in the industrial control software; writes each parameter in the parameter tuning data into the corresponding register or variable of the virtual controller; calls the simulation run function to simulate one or more control cycles; captures the output status, abnormal event queue, and error log after the simulation ends; if there are no abnormalities and the output meets the expected logic, such as the simulated liquid level stabilizing within the target range, a "simulation passed" result is generated; otherwise, a "simulation abnormal" result is generated.

[0064] The logic operation unit determines the type of the logic simulation result. If it passes, it encapsulates a confirmation message and sends it to the control unit. Upon receiving the message, the control unit calls the sending interface of the instruction issuing module, encodes the final control instruction data into a data frame according to protocols such as ModbusTCP or OPC UA, and sends it to the field controller through the server communication unit, while receiving execution confirmation feedback. If the simulation is abnormal, the logic operation unit directly generates error alarm data and pushes it to the designated smart display through the communication channel between the server and the smart display (such as WebSocket). Upon receiving the data, the smart display pops up a red alarm window or provides a voice prompt, and simultaneously stores the alarm record in the local storage area. The error alarm data refers to a structured alarm message containing information such as error code, error description, timestamp, and user ID, for example, {"errorCode":"E001","message":"Simulation abnormality: water pump frequency adjustment caused simulation overload","timestamp":"2025-04-27T15:30:45Z","userId":"operator_01"}.

[0065] This embodiment relies on the logic operation unit to combine the user identity information associated with the control command with the logged-in user information to complete the permission compliance judgment. It can identify the operation qualifications of the management personnel from the source of operation, effectively prevent the hidden dangers of equipment operation caused by unauthorized operation. After the permission is verified, the parameter adjustment data is input into the virtual controller to carry out the pre-logic simulation, simulate the actual operation logic of the command execution in advance, and check the logic conflicts and operation faults in the parameter adjustment process in advance. Based on the simulation results, the subsequent process is promoted in a differentiated manner. After the simulation is qualified, the final control command is released and transmitted. If the simulation is abnormal, alarm data is generated immediately and pushed to the smart display to realize real-time warning. In the end, this embodiment can obtain a safer, more standardized and fault-controllable industrial control command issuance and management mechanism.

[0066] In some embodiments, the logic operation unit is further configured to: Using preset stable operating condition data, and based on the adjusted operating condition parameters collected by the industrial control software, adjustment result data is generated; If the adjustment result data is good, the parameter adjustment data, system recovery steady-state time data, and fluctuation amplitude data will be stored in the rule base as reference data.

[0067] In this embodiment, firstly, the logic operation unit uses preset stable state operating condition data and, based on the adjusted operating condition parameters collected by the industrial control software, generates adjustment result data. Here, the adjusted operating condition parameters refer to the sequence of actual operating condition feedback values ​​collected by the industrial control software from field sensors or controllers after the parameter adjustment command is issued, such as the liquid level height values ​​recorded every 0.5 seconds during the liquid level adjustment process [0.55m, 0.65m, 0.85m, 0.95m, 1.00m, 1.02m, 1.00m]. The preset stable state operating condition data refers to the benchmark conditions predefined by the system for evaluating whether the parameter adjustment effect is good, including at least the maximum allowable stability time and the maximum allowable fluctuation range.

[0068] In practice, the logic unit obtains the time series of adjusted operating parameters from the industrial control software; reads preset stable state operating data from the rule base or configuration file: stable time threshold and fluctuation threshold; calculates the actual time required to recover to steady state (the time from the moment the parameter adjustment command is issued to the time it takes for the parameter to first enter the target value ± allowable error range and no longer exceed it), and the actual maximum fluctuation amplitude (the maximum absolute deviation of all values ​​in the sequence from the target value); if the actual time required to recover to steady state is ≤ stable time threshold and the actual maximum fluctuation amplitude is ≤ fluctuation threshold, "good" adjustment result data is generated; otherwise, "poor" adjustment result data is generated.

[0069] If the adjustment results are satisfactory, the parameter adjustment data, system recovery steady-state time data, and fluctuation amplitude data will be stored in the rule base as reference data.

[0070] In practice, for example, after the logic operation unit determines that the adjustment result data is "good", it extracts the parameter adjustment data from the context of the current parameter adjustment operation; obtains the system recovery steady-state time data and fluctuation amplitude data; assembles these three types of data together with the current user ID, device ID, operating condition type, and timestamp into a structured record object; calls the rule base's storage interface to write the record object into the "parameter adjustment reference basis" table or configuration file; after successful storage, the rule base returns the record ID, the logic operation unit records the log, and the reference basis data is saved.

[0071] In this embodiment, the logic operation unit can compare and analyze the adjusted actual operating parameters and preset stable state data to objectively evaluate the actual application effect of a single parameter adjustment, accurately reflect the actual adaptation of the operating condition control, and automatically collect and retain the content of this parameter adjustment, the system steady-state recovery time, and the effective operating data such as parameter fluctuation amplitude when the control effect meets the standard requirements. These data are then uniformly stored in the rule base as a reference for subsequent control optimization, and the field operating condition adaptation samples are continuously accumulated to improve the control rule system.

[0072] In some embodiments, the logic operation unit is further configured to: If the adjustment result data is poor, a recording command is sent to the face detection unit; and a screen recording command is sent to the smart display; The face detection unit receives the recording instruction, starts the recording mode, and generates a fault handling process recording video; The smart display receives the screen recording command, starts screen recording, and generates a screen recording video of the fault handling process. The smart display also includes a storage area for storing screen recordings and video recordings of the fault handling process.

[0073] In this embodiment, when the logic operation unit determines that the adjustment result data is poor, it sends a recording command to the face detection unit and a screen recording command to the smart display. Here, the recording command is used to trigger the face detection unit to start recording, and its content includes parameters such as recording resolution, frame rate, and encoding format. For example, the start command format is {"cmd":"start_video","resolution":"640x480","fps":30}. The screen recording command is used to trigger the system client software to start capturing the screen operation process, for example, {"cmd":"start_screen_recording", "area":"full","fps":15}.

[0074] In practice, after receiving the adjustment result data as a difference, the logic operation unit first constructs a recording instruction data packet, specifies the recording parameters, and then sends the instruction to the IP address or device ID corresponding to the face detection unit through the server communication unit; simultaneously, it constructs a screen recording instruction data packet and sends it to the system client software of the smart display through the same communication channel. Optionally, an instruction sending log is recorded after the sending is completed.

[0075] The face detection unit activates recording mode and generates a fault handling process video. This video is a continuous image file of the user's face and surrounding environment recorded by the front-facing camera from the start to the end of recording, capturing the user's actions, expressions, and postures while handling the fault. Specifically, upon receiving a recording command, the face detection unit initializes the camera sensor and allocates a video encoding buffer; it begins capturing images frame by frame, compressing each frame and writing it to a temporary file; recording continues until a stop command is received, which can be sent automatically after a system timeout; after recording ends, the temporary file is converted into the final video file, and an index record containing metadata such as start time, end time, file size, and associated user ID is generated.

[0076] The smart monitor initiates screen recording, generating a troubleshooting process screen recording video. This troubleshooting process screen recording video refers to a video file captured by the smart monitor's operating system of the screen operation process, recording all displayed content such as user-opened parameter adjustment windows, mouse movements, keyboard input, and interface switching. In practice, after receiving the screen recording command, the system client software on the smart monitor calls the screen capture API provided by the operating system (such as DXGI or X11), sets the capture area to full screen or a specified window; begins capturing screen images frame by frame, compresses them, and writes them to a video file; timestamp watermarks and user identifiers can be superimposed during screen recording; recording continues until a stop command is received; after recording ends, the file is closed, and metadata is generated.

[0077] After the video recording and screen recording are completed, the smart display's system client software obtains the temporary paths of the two video files; it calls the file system API to move these two files to the specified directory in the storage area (e.g., " / storage / accident / 2025-04-27 / "); at the same time, it generates an association record, writing information such as the two video file names, fault ID, user ID, and operation time range to the local database or metadata file; if storage space is insufficient, it deletes the oldest recall video file in chronological order to ensure that new data can be written; after storage is completed, it can send a storage confirmation message to the server.

[0078] During implementation, effective trigger condition feature data (such as parameter anomaly type and rate of change threshold) and operation step feature data (such as interface call order and parameter input value) can be extracted from the two video files to form a structured experience description. Then, these feature data are compared with the existing rule base. If they are identified as typical processing flows that have not yet been included, a new industrial control parameter adjustment rule is generated and stored in the rule base. This allows the corresponding operation interface to be automatically called up and parameters to be preset when similar abnormal conditions are encountered in the future, thus completing a closed-loop optimization from anomaly occurrence, recording, analysis to rule iteration.

[0079] In this embodiment, when the adjusted result data is poor, the logic operation unit synchronously sends screen recording and video recording commands to the face detection unit and the smart display, enabling synchronous acquisition of on-site images and the operation interface. The face detection unit starts the video recording mode to record the entire fault handling process, which can completely capture the on-site working environment information. The smart display starts the screen recording function to record the human-machine operation process, and completely retains the operation details of parameter adjustment. Relying on the storage area of ​​the smart display to uniformly store image data, it can ensure the stable storage and subsequent retrieval of fault record data. Finally, this embodiment can obtain a more comprehensive and detailed industrial control anomaly tracing system that facilitates fault source tracing and experience summarization.

[0080] Based on the same concept, this application also provides a control method for an intelligent display system, wherein the control method is deployed on the display cluster server of the intelligent display system provided in any of the foregoing embodiments. Figure 2 This document illustrates a flowchart of a control method for an intelligent display system provided in an embodiment of this application. (See attached diagram.) Figure 2 As shown, the control method for the intelligent display system provided in this application embodiment includes: Step S21: Obtain the facial image of the operator currently displaying the screen; Step S22: Match the facial image with the user information in the user database to generate a matching result; Step S23: If the matching result is a successful match, retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database; Step S24: Match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data, and industrial control software permission configuration data to the system client software of the smart display. This allows the system client software to parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, load the corresponding operating system runtime environment, perform interface rendering calculations, generate the rendered digital interface image signal, and send it to the display unit.

[0081] This embodiment first collects facial images of on-site operators, and then compares and verifies their identities using facial data and user database information. This allows for rapid and accurate identification of operators, effectively preventing unauthorized personnel from accessing the system. Once the identity is verified, the system automatically retrieves the dedicated terminal system configuration and industrial control software permission configuration data from the user database, eliminating the need for manual input and setting of related parameters. Based on the system configuration data, the system accurately matches the corresponding operating system image and sends the image file and both types of configuration data to the system client software of the smart display. The client automatically parses the data, loads the runtime environment, and renders the interface. This fully automated process of identity verification system adaptation, permission allocation, and interface display simplifies the manual configuration and debugging process and accurately matches dedicated runtime environments and operating permissions for different operators, significantly improving the efficiency of smart display system login and the security of industrial control operation permission management.

[0082] Figure 3 A flowchart illustrating a control method for an intelligent display system according to another embodiment of this application is shown. See also... Figure 3 As shown, the implementation process of this application embodiment is as follows: The smart display's face detection unit scans the user's face and transmits the data to the display cluster server via the communication unit. The image data analysis unit of the display cluster server processes the data and retrieves stored user database information. The user database matches the stored user image, the user's corresponding operating system and software username and password, and the user's corresponding operating system and software access permissions, forming a data chain group, which is sent to the system server software and the system allocation and login unit. The system allocation and login unit performs unified login for the software within the operating system based on the user database data chain. The system server software, based on the user database data chain group, performs operating system allocation, data interaction, and interface rendering for the corresponding smart display system client software. The smart display's display unit displays the system and software interface corresponding to the system client software and records the screen. The smart display's storage area stores the screen recording data and computational macros. When the industrial control software of the display cluster server detects a change in preset industrial control parameters, it... The communication unit transmits data information to the computational macro of the intelligent display. The computational macro drives the operation interface to open the corresponding parameter adjustment window and preset the parameter adjustment data. After the user confirms manually or automatically, the control command is sent to the logic operation unit. The logic operation unit performs user permission verification, industrial control logic judgment, and parameter adjustment simulation. If there are no errors, the command is sent to the industrial control software. The industrial control software sends the control command to the field control system to perform parameter adjustment. The industrial control software collects the feedback value of the parameter adjustment. The logic operation unit evaluates the adjustment effect. If the effect is not good, the rule base of the display group server and the computational macro of the intelligent display are modified. The accident replay function is entered simultaneously. If the effect is good, the adjusted parameters, the time for the system to recover to steady state after adjustment, the fluctuation range, and other information are stored as a reference for the rule base. Subsequently, the industrial control software collects whether the system operation is abnormal in real time. If there is an abnormality, the accident replay is executed. When abnormal parameter changes are collected, the face detection unit automatically starts the recording mode and performs screen recording operation, which is saved in the storage area. If there is no abnormality, the process ends.

[0083] Those skilled in the art will understand that in the above-described method of the specific embodiments, the order in which the steps are written does not imply a strict execution order, but constitutes no limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0084] It should be noted that in practical applications, all the above-described possible implementation methods can be combined in any way to form possible embodiments of this application, and will not be described in detail here. The information (including but not limited to device information, user information, etc.) and data (including but not limited to data used for analysis, storage, and display) involved in this application are all information and data authorized by the user or fully authorized by all parties. The software tools or components appearing in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0085] Based on the same concept, this application also provides a control device for an intelligent display system, which corresponds one-to-one with the control method of the intelligent display system in the above embodiments. Figure 4 A schematic diagram of the control device for the intelligent display system provided in this application embodiment is shown. See also: Figure 4 As shown, the control device 400 of the intelligent display system provided in this application embodiment includes: The acquisition module 401 acquires the facial image of the operator currently displaying the screen; User matching module 402 is used to match the facial image with user information in the user database and generate a matching result; The system allocation module 403 is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data and industrial control software permission configuration data to the system client software of the smart display, so that the system client software parses the operating system image, terminal operating system configuration data and industrial control software permission configuration data sent by the display group server, loads the corresponding operating system running environment, performs interface rendering calculations, generates the rendered digital interface image signal and sends it to the display unit.

[0086] Specific limitations regarding the control device of the intelligent display system can be found in the limitations of the control method for the intelligent display system described above, and will not be repeated here. Each module in the control device of the aforementioned intelligent display system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0087] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Figure 5 As shown, at the hardware level, this electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or it may include non-volatile memory, such as at least one disk drive. Of course, this electronic device may also include other hardware required for other business operations.

[0088] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0089] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0090] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming the control device of the intelligent display system at the logical level. The processor executes the program stored in memory and specifically performs the aforementioned methods.

[0091] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0092] This electronic device can execute the control method of the intelligent display system provided in several embodiments of this application, and is implemented as a control device for the intelligent display system. Figure 4 The functions of the embodiments shown are not described in detail here.

[0093] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform the control method of the intelligent display system provided in several embodiments of this application.

[0094] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0096] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0097] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0098] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0099] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0100] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0101] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0102] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An intelligent display system, characterized in that, The system includes: at least one smart display and a display cluster server; The smart display includes: The face detection unit is used to capture the facial image of the current display operator; preprocess the facial image to obtain preprocessed data and send it to the display cluster server; The system client software is used to receive and parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, load the corresponding operating system running environment, perform interface rendering calculations, generate the rendered digital interface image signal, and send it to the display unit. The display unit is used to receive and convert the digital interface image signal to display the corresponding operating system and pre-installed industrial control software interface. The display cluster server includes: The system server software uses its internal virtualization technology to build multiple operating system images, each of which has an industrial control software client installed. The user database is used to store and configure user information; the user information includes at least: the operator's facial image, terminal operating system configuration data, and industrial control software permission configuration data; The image data analysis unit is used to receive the preprocessed data and match it with user information in the user database to generate a matching result; The system allocation and login unit is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data and industrial control software permission configuration data to the system client software.

2. The intelligent display system according to claim 1, characterized in that, The user information also includes industrial control software and third-party system accounts and passwords; the system allocation and login unit is also used for: Retrieve the user information corresponding to the current monitor operator from the user database; In response to receiving a login request from an associated system, the user information is processed using preset verification rule data to generate an account legitimacy verification result; Based on the account validity verification result, a login execution command is generated; Based on the login execution command, the system will automatically log in to the associated system or redirect to the user login interface.

3. The intelligent display system according to claim 1, characterized in that, The intelligent display also includes a computational macro unit; the display cluster server also includes a rule base for storing industrial control parameter adjustment rules; the industrial control parameter adjustment rules include trigger conditions and corresponding computational macro programs; The industrial control software in the display cluster server is used to collect real-time operating condition parameter data of the industrial control system and send it to the computing macro unit. The computational macro unit includes a control unit, used for: The real-time operating condition parameter data is matched with the trigger conditions in the rule base to generate matching result data; Based on the matching result data, the corresponding calculation macro program is invoked; The real-time operating condition parameter data is processed using the aforementioned computational macro program to generate parameter adjustment data; the parameter adjustment data is then loaded into the associated control interface. Based on the user's confirmed operation instructions, generate the final control instruction data; The final control command data is sent to the industrial control system through the command sending module.

4. The intelligent display system according to claim 3, characterized in that, The computational macro unit also includes an operation recording module, used for: Collect raw data of operational steps input by personnel when responding to changes in operating parameters; Trigger condition feature data and operation step feature data are extracted from the original data of trigger conditions corresponding to changes in operating parameters and the original data of operation steps. Based on the trigger condition feature data and the operation step feature data, industrial control parameter adjustment rules are generated and stored in the rule base.

5. The intelligent display system according to claim 1, characterized in that, The user information also includes user identity information; the display cluster server also includes a logic operation unit for: Based on the user identity information and logged-in user information corresponding to the final control instruction data, permission validity determination data is generated. If the permission validity determination data is valid, the parameter tuning data received from the control unit will be input into the virtual controller of the industrial control software to generate a logic simulation result. If the logic simulation result is that the simulation is successful, the logic simulation result is sent to the control unit. After receiving the logic simulation result, the control unit sends the final control command data to each industrial control system through the command sending module. If the logic simulation result is abnormal, an error alarm is generated and sent to the smart display to issue an alarm.

6. The intelligent display system according to claim 5, characterized in that, The logic operation unit is also used for: Using preset stable operating condition data, and based on the adjusted operating condition parameters collected by the industrial control software, adjustment result data is generated; If the adjustment result data is good, the parameter adjustment data, system recovery steady-state time data, and fluctuation amplitude data will be stored in the rule base as reference data.

7. The intelligent display system according to claim 6, characterized in that, The logic operation unit is also used for: If the adjustment result data is poor, a recording command is sent to the face detection unit; and a screen recording command is sent to the smart display; The face detection unit receives the recording instruction, starts the recording mode, and generates a fault handling process recording video; The smart display receives the screen recording command, starts screen recording, and generates a screen recording video of the fault handling process. The smart display also includes a storage area for storing screen recordings and video recordings of the fault handling process.

8. A control method for an intelligent display system, characterized in that, The control method is deployed on the display cluster server of any of the intelligent display systems described in claims 1-7, and the control method includes: Acquire the facial image of the operator currently displaying the monitor; The facial image is matched with user information in the user database to generate a matching result; If the matching result is successful, the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator are retrieved from the user database. The system client software of the smart display is then sent to the system client software of the display group, which matches the corresponding operating system image based on the terminal operating system configuration data. The system client software parses the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, loads the corresponding operating system runtime environment, performs interface rendering calculations, generates the rendered digital interface image signal, and sends it to the display unit.

9. A control device for an intelligent display system, characterized in that, The control device is deployed on the display cluster server of the intelligent display system as described in any one of claims 1-7, and the control device includes: The acquisition module acquires the facial image of the operator currently displaying the screen. The user matching module is used to match the facial image with user information in the user database and generate a matching result; The system allocation module is used to retrieve the terminal operating system configuration data and industrial control software permission configuration data corresponding to the current display screen operator from the user database if the matching result is successful; match the corresponding operating system image according to the terminal operating system configuration data, and send the operating system image, terminal operating system configuration data, and industrial control software permission configuration data to the system client software of the smart display. This allows the system client software to parse the operating system image, terminal operating system configuration data, and industrial control software permission configuration data sent by the display group server, load the corresponding operating system runtime environment, perform interface rendering calculations, generate the rendered digital interface image signal, and send it to the display unit.

10. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by an electronic device including multiple applications, the electronic device performs the steps of the control method for the intelligent display system as described in claim 8.

Images