Touch screen control operation method for deep learning and visual servo of power plant central control room

Abstract

The invention discloses a touch screen control operation method for deep learning and visual servo of a power plant central control room. The touch screen control operation method comprises the following steps of data acquisition and preprocessing, deep learning model construction and training, visual servo control, operation decision and execution and personalized operation interface customization. According to the touch screen control operation method for deep learning and visual servo of the power plant central control room provided by the invention, a series of innovative technologies such as data acquisition and preprocessing, deep learning model construction and training, visual servo control, operation decision and execution, personalized operation interface customization and the like are organically combined; the intelligent level, the high efficiency, the accuracy and the individuation level of the operation of the power plant central control room are comprehensively improved. In practical application, the method can remarkably improve the operation efficiency, reduce the misoperation risk, improve the operation accuracy and provide personalized operation experience, and powerful technical support is provided for safe, stable and efficient operation of a power plant.

Description

Technical Field

[0001] The present invention relates to the fields of image processing technology and visual servo technology, and particularly relates to a touch screen control operation method for deep learning and visual servo in a power plant centralized control room. Background Art

[0002] In modern power plant centralized control rooms, operators need to perform real-time monitoring and control operations on a large number of equipment operation parameters. Traditional operation methods mainly rely on mice, keyboards, and fixed operation interface buttons, etc. However, with the continuous expansion of power plant scale and the increase in equipment complexity, these traditional operation methods have exposed many problems.

[0003] On the one hand, traditional operation methods require operators to manually click a large number of menus and buttons to complete various operations, and the operation process is cumbersome and error-prone. For example, in an emergency, operators may make misoperations due to nervousness, resulting in serious consequences. On the other hand, the operation interfaces in power plant centralized control rooms are often designed based on fixed layouts and preset operation logics, and it is difficult to flexibly adjust according to the actual operation scenarios and the needs of operators. This makes the operation efficiency of operators low when facing complex and changeable working conditions, and it is difficult to quickly and accurately obtain the required information and execute corresponding operations.

[0004] In addition, with the continuous development of artificial intelligence technology, deep learning and visual servo technologies have been widely applied in various fields. However, in the field of power plant centralized control room operations, how to effectively integrate these advanced technologies and apply them to touch screen control operations to improve the intelligent level of operations is still an urgent problem to be solved. Summary of the Invention

[0005] In view of the above deficiencies in the prior art, the present invention provides a touch screen control operation method for deep learning and visual servo in a power plant centralized control room, and its purpose is to solve... and other problems.

[0006] In order to achieve the above invention purpose, the technical solution adopted by the present invention is as follows:

[0007] A touch screen control operation method for deep learning and visual servo in a power plant centralized control room, comprising the following steps:

[0008] S1: Data acquisition and preprocessing, obtaining the image data of the centralized control room interface and performing preprocessing on it, collecting the RGB image of the display screen through a camera, and then performing normalization and noise injection operations on the image;

[0009] S2: Deep learning model construction and training. Use a deep learning framework to construct a Convolutional Neural Network (CNN) model. The construction of the model includes defining the network structure, setting hyperparameters, loading the dataset and training. During the training process, optimize the parameters through the backpropagation algorithm to ensure that the model can accurately identify the interface characters and pose information;

[0010] S3: Visual servo control. Visual servo technology detects and identifies the target position through vision, and combines the pose measurement information to drive the robotic arm to complete precise operations. Detect the character buttons that need to be clicked through the image template matching method, and combine with the CNN to determine the pose information of the display screen to achieve precise operation of the control interface;

[0011] S4: Operation decision-making and execution. With the support of the deep learning model and visual servo technology, the system quickly judges the state of the current working page, generates corresponding control instructions according to the image recognition results of the operation interface, and transmits them to the robotic arm through the PLC control system to complete the automated operation;

[0012] S5: Personalized operation interface customization. Intuitively adjust the interface layout and function modules through touch screen and gesture control technology.

[0013] Further, in the step S1, the data acquisition and preprocessing include the following steps:

[0014] S11: Image data acquisition: Install multiple high-definition cameras in the operation area of the power plant centralized control room to collect the hand movement images and the display images of the operation interface during the touch screen operation of the operator;

[0015] S12: Sensor data acquisition: Integrate pressure sensors and capacitive sensors on the touch screen device to collect the pressure and touch position information when the operator touches the screen;

[0016] S13: Data preprocessing: Denoise, grayscale, and normalize the collected image data.

[0017] Further, in the step S2, the deep learning model construction and training include the following steps:

[0018] S21: Action recognition: Use a Convolutional Neural Network (CNN) to construct a hand action recognition model, and accurately recognize the click, slide, and zoom gestures of the operator by learning the hand action images;

[0019] S22: Equipment status prediction: Use a recurrent neural network (RNN) or a long short-term memory network (LSTM) to construct an equipment status prediction model. Use the historical parameter data of the equipment operation and the current operation information as inputs to predict the future operation status of the equipment. By analyzing the voltage and current change trends over a period of time and the adjustment operations of the operator, predict whether the equipment will malfunction.

[0020] Further, in step S3, the visual servo control includes the following steps:

[0021] S31: Target detection and positioning: Based on a deep learning-based target detection algorithm, perform real-time detection and positioning of the control elements on the operation interface. When the hand movement of the operator is recognized, quickly determine the position of the control element corresponding to this movement through the visual servo system to achieve accurate operation positioning;

[0022] S32: Operation guidance and feedback: When the operator performs a touch screen operation, the visual servo system provides real-time operation guidance on the screen according to the recognized hand movement and the position of the target control element. After the operation is completed, the system promptly feedbacks the operation result and informs the operator whether the operation is successful through color changes and animation effects.

[0023] Further, in step S4, the operation decision-making and execution include the following steps:

[0024] S42: Operation decision-making formulation: According to the hand movement recognized by the deep learning model, the predicted equipment status, and the target positioning information provided by the visual servo system, formulate an operation decision. When the operator makes a gesture to click on a certain control button and the equipment status prediction model shows that the equipment is currently in normal operation, the system determines that this operation conforms to the operation rules and allows execution;

[0025] S42: Operation execution and monitoring: Convert the formulated operation decision into specific control instructions and send them to the control system of the power plant to execute the corresponding operations. Monitor the operation execution process in real time. If abnormal situations are found during the operation process, including equipment response timeout and abnormal parameter changes, stop the operation immediately and issue an alarm.

[0026] Further, in step S5, the personalized operation interface customization includes the following steps:

[0027] S51: User preference learning: By analyzing the historical operation data of the operator, including operation habits and frequently used function modules, use machine learning algorithms to learn the personalized preferences of each operator, and count the viewing frequencies of different equipment parameters by the operator at different time periods, as well as the usage times of various operation functions;

[0028] S52: Interface customization generation: Based on the learned user preferences, a personalized operation interface is generated for each operator. On the interface, the frequently used function modules and control elements of the operator are placed in prominent and easily operable positions. At the same time, the display parameters such as the color and font size of the interface are adjusted according to the operator's habits.

[0029] The beneficial effects of the present invention are as follows:

[0030] Improve operation efficiency: Through gesture recognition and operation guidance, operators can complete various operations more quickly, without the need to click on menus and buttons tediously, greatly shortening the operation time and improving work efficiency. When adjusting device parameters, operators can quickly locate the parameters to be adjusted and modify them through simple gesture operations, saving a large amount of time compared to traditional methods.

[0031] Reduce the risk of misoperation: The deep learning model can monitor the operation intention of the operator in real time and make operation decisions based on the device status and operation rules. When a possible misoperation is detected, the system issues a warning prompt in time to prevent the operator from causing equipment failures or accidents due to misoperation. When an operator attempts to perform a dangerous operation on a running device, the system will automatically pop up a warning window to block the execution of the operation.

[0032] Improve operation accuracy: The visual servo system realizes precise positioning and operation feedback of the operation target, ensuring that each operation action of the operator can accurately act on the corresponding control element, improving the operation accuracy. When fine-tuning device parameters, operators can precisely control the adjustment amplitude through visual feedback, avoiding excessive parameter deviation caused by inaccurate operation.

[0033] Personalized operation experience: The personalized operation interface customization module provides an interface that conforms to each operator's personal operation habits, improving the operator's operation comfort and job satisfaction. Operators can customize the interface according to their own needs, making the operation more convenient and efficient, and reducing the learning cost and operation inconvenience caused by adapting to a general interface. Description of the Drawings

[0034] Figure 1 It is a schematic diagram of the touch screen control operation method of the deep learning and visual servo in the power plant centralized control room of the present invention;

[0035] Figure 2 It is a schematic diagram of data acquisition and preprocessing of the present invention;

[0036] Figure 3 It is a schematic diagram of the construction and training of the deep learning model of the present invention;

[0037] Figure 4Schematic diagram of visual servo control for the present invention;

[0038] Figure 5 Schematic diagram of operation decision-making and execution for the present invention;

[0039] Figure 6 Schematic diagram of personalized operation interface customization for the present invention. Detailed implementation manners

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0041] As Figures 1 to 6 shown, a touch screen control operation method for deep learning and visual servo in a power plant centralized control room includes the following steps:

[0042] Data acquisition and preprocessing:

[0043] Image data acquisition:

[0044] In the operation area of the power plant centralized control room, a plurality of high-definition cameras are arranged. The cameras are responsible for capturing the hand movement images of the operator during the touch screen operation and the display images of the operation interface. The cameras are deployed from multiple angles, such as above, left and right sides, and in front of the operator of the operation screen. 5 - 8 high-definition cameras are installed in a large power plant centralized control room, with a resolution of 1920×1080 and a frame rate maintained at 30fps, which can clearly and smoothly record every operation detail, providing a rich and accurate data basis for subsequent analysis and processing.

[0045] Sensor data acquisition:

[0046] A pressure sensor and a capacitance sensor are integrated on the touch screen device. The pressure sensor can accurately detect the pressure value applied by the operator when touching the screen, with an accuracy of 0.1N. The capacitance sensor can accurately measure the touch position, with the error controlled within 1mm. At the same time, by establishing a stable data connection with the control system of the power plant, key parameters of the equipment operation are obtained in real time, such as the voltage, current, and frequency of the generator, as well as the temperature, rotation speed, etc. of various equipment. The sensor data and the image data complement each other, providing multi-dimensional data support for the system to comprehensively understand the operation behavior of the operator and the operation state of the equipment.

[0047] Data preprocessing:

[0048] For the collected image data, first, the Gaussian filtering algorithm is used for denoising. Gaussian filtering can effectively remove the noise interference in the image, making the image clearer and cleaner. Then, the color image is converted into a grayscale image, which simplifies the subsequent image processing process, reduces the data processing volume, and at the same time retains the key structural and texture information in the image. Finally, the grayscale image is normalized, and the pixel values of the image are uniformly mapped to the standard range of [0,1], so that the deep learning model can better receive and process the data, improving the training efficiency and accuracy of the model. For the sensor data, the Kalman filtering algorithm is used for filtering. Kalman filtering can perform optimal estimation on the data according to the dynamic model of the system and the measurement data, effectively remove the noise, improve the stability and reliability of the data, and ensure that the data can truly reflect the operating state of the device and the operation of the operator.

[0049] Construction and Training of Deep Learning Model:

[0050] Action Recognition:

[0051] A convolutional neural network (CNN) is used to construct a hand action recognition model. Through in-depth learning of a large number of rich hand action images, the CNN model can accurately recognize various gestures of the operator, such as common clicks, swipes, zooms, etc. In the process of model construction, multiple convolutional layers, pooling layers, and fully connected layers are designed. A hand action recognition CNN model contains 5 convolutional layers, each convolutional layer uses a 3×3 convolutional kernel, the stride is set to 1, and the padding is 1. The convolutional kernel configuration can effectively extract local features in the image. Subsequently, there are 3 pooling layers, using 2×2 max pooling operations to downsample the feature maps output by the convolutional layers, reducing the data volume while retaining important feature information. Finally, there are 2 fully connected layers, and the number of neurons in the fully connected layers is 128 and 64 respectively. Through the fully connected layers, the features extracted previously are integrated and classified to output the final recognition result. During the training process, the model is trained using a large number of labeled gesture image datasets, which cover click, swipe, and zoom gesture images in various different scenarios and at different angles. By continuously adjusting the parameters of the model, such as the weights and biases of the convolutional kernels, the recognition accuracy of the model is continuously improved. After a large number of experiments and optimizations, on the prepared test set, the recognition accuracy of the model can reach more than 95%.

[0052] Device Status Prediction:

[0053] Construct a device status prediction model using a Recurrent Neural Network (RNN) or a Long Short-Term Memory Network (LSTM). The device status prediction model can fully consider the time series characteristics of device operation, taking the historical parameter data of device operation and the current operation information as input to predict the future operation status of the device. For example, by analyzing the voltage and current change trends of power plant equipment in the past few hours or even days, as well as various adjustment operations performed by operators on the equipment, such as power adjustment, valve opening adjustment, etc., the model can predict whether the device is likely to fail.

[0054] An LSTM device status prediction model, which contains 2 LSTM layers and 1 fully connected layer. The LSTM layer can effectively handle the long-term dependencies in time series data and capture the changing patterns of the device operation status. The fully connected layer then integrates the features output by the LSTM layer to output the prediction results of the future operation status of the device. During the training process, the device operation data of the power plant in the past year or even longer is used for training. By continuously adjusting the parameters of the model, the model can accurately predict the operation status of the device within the next 1 hour or even several hours. After actual application verification, the prediction accuracy of the model can reach over 85%, providing strong support for the preventive maintenance and safe operation of the device.

[0055] Visual servo control:

[0056] Object detection and localization:

[0057] Based on the deep learning object detection algorithm, various control elements on the operation interface, such as buttons, sliders, text boxes, etc., are detected and localized in real time. When the hand movements of the operator are recognized by the previously constructed action recognition model, the visual servo system quickly determines the position of the control element corresponding to the recognized gesture information on the operation interface in combination with the object detection algorithm, achieving precise operation positioning. For a circular control button, the system can accurately calculate its center position and radius size within an extremely short time. In actual applications, the object detection algorithm can quickly and accurately detect various control elements in images with dozens of frames per second, and the detection accuracy is over 98%, ensuring that the operation intention of the operator can be accurately conveyed to the system.

[0058] Operation guidance and feedback:

[0059] When the operator performs touch screen operations, the visual servo system provides real-time operation guidance on the screen based on the recognized hand movements and the positions of the target control elements. When the operator makes a click gesture and the target button is located far away, the system will display a clear operation guidance line on the screen from the starting point of the gesture to the center of the button, guiding the operator to accurately click the button. After the operation is completed, the system promptly feedbacks the operation result, informing the operator whether the operation is successful in an intuitive way such as color change and animation effect. When the operation is successful, the button will turn green and accompanied by a short flashing animation; when the operation fails, the button will turn red and display the corresponding error message. This real-time operation guidance and feedback mechanism greatly improves the operator's operation experience and operation accuracy, reducing the possibility of operation errors.

[0060] Operation Decision-making and Execution:

[0061] Operation Decision-making:

[0062] Based on the hand movements recognized by the deep learning model, the predicted device states, and the target positioning information provided by the visual servo system, combined with the power plant operation rules and safety policies, operation decisions are made. When the operator makes a gesture to click the start button of a certain device, and the device state prediction model shows that the device is currently in the standby state and all parameters are normal, the operation decision-making module will quickly determine that this operation complies with the operation rules and allows it to be executed. On the contrary, if the device state prediction model shows that the device has potential failure risks, such as the temperature of a certain key component is too high, even if the operator makes a start gesture, the system will determine that this operation does not meet the safety requirements, prevent the operation from being executed, and issue an alarm prompt to the operator. The operation decision-making module ensures that every operation decision is safe and reasonable.

[0063] Operation Execution and Monitoring:

[0064] The formulated operation decisions are converted into specific control instructions and sent to the PLC control system of the power plant through a reliable communication protocol to execute the corresponding operations. During the operation execution process, the system monitors the running state of the device in real time and continuously obtains various running parameters of the device. If abnormal situations occur during the operation, such as device response timeout, normally the device should respond to the operation instruction within a few seconds, if it still does not respond after exceeding the set time threshold, it is determined as response timeout; or abnormal parameter changes, such as a sudden large increase in current exceeding the normal range, the system will immediately stop the operation, issue an alarm to the operator, and record the detailed information of the abnormal situation for subsequent fault troubleshooting and analysis. This real-time monitoring and abnormal handling mechanism ensures the safety and stability of operation execution, effectively avoiding serious accidents caused by device failures or operation errors.

[0065] Personalized Operation Interface Customization:

[0066] User preference learning:

[0067] By deeply analyzing the historical operation data of operators, including operation habits, frequently used function modules, etc., advanced machine learning algorithms are used to learn the personalized preferences of each operator. By statistically analyzing the viewing frequency of different equipment parameters by operators at different time periods, it is found that an operator often views parameters such as the rotation speed and temperature of the steam turbine during the morning inspection period every day; at the same time, by counting the number of times of using various operation functions, it is found that this operator is accustomed to using the quick operation menu to control the start and stop of equipment. Through the analysis and mining of a large amount of similar data, the machine learning algorithm can accurately grasp the personalized needs and operation habits of each operator, providing strong data support for the customization of personalized operation interfaces.

[0068] Interface customization and generation:

[0069] According to the learned user preferences, a dedicated personalized operation interface is generated for each operator. On the interface, the frequently used function modules and control elements of the operator are placed in prominent and easy-to-operate positions. For operators who often view the parameters of the steam turbine, the display area of the relevant parameters of the steam turbine is placed in the center of the screen, and the font is enlarged for easy and clear information acquisition by the operator. At the same time, a quick operation bar is set on one side of the screen, containing common function buttons such as equipment start / stop and parameter adjustment. The size and position of the buttons are optimized according to the operator's habits, enabling easy clicking and operation. In addition, the display parameters such as the interface color and font size are adjusted according to the operator's personal preferences to create a comfortable and convenient operation environment. After using the personalized interface, the operation efficiency of the operator is significantly improved, and the satisfaction with the operation interface is also greatly enhanced, effectively reducing the learning cost and operation inconvenience caused by adapting to the general interface.

[0070] In the traditional operation mode of the power plant central control room, operators need to search for the required operation options among numerous menus and buttons, and the operation process is cumbersome and time-consuming. With the gesture recognition and operation guidance functions of the present invention, operators can complete various operations more quickly. Taking the adjustment of equipment parameters as an example, in the traditional mode, operators may need to switch through multiple menu levels to find the parameters to be adjusted and make modifications, and the whole process may take dozens of seconds or even several minutes. By using the method of the present invention, operators only need to perform simple gesture operations, such as swiping and zooming, to quickly locate the parameters to be adjusted and make modifications, and the whole process may only take a few seconds, saving a large amount of time compared with the traditional mode, and the operation efficiency has been significantly improved. According to actual application statistics, in daily operations, using the method of the present invention, the average operation time of operators is shortened by more than 60%, greatly improving the work efficiency and making the operation management of the power plant more efficient and smooth.

[0071] In the operation of the power plant central control room, misoperations may cause serious equipment failures or even safety accidents. The deep learning model of the present invention can monitor the operation intention of operators in real time and make operation decisions according to the equipment status and operation rules. When a possible misoperation is detected, the system will issue a warning prompt in time. For example, when an operator attempts to perform a dangerous operation on a running device, such as directly performing maintenance on internal components without stopping the device, the system will immediately pop up a warning window automatically, prevent the execution of the operation, and explain the operation risks and correct operation procedures to the operator in detail. Through this intelligent risk warning and operation control mechanism, misoperations caused by operator negligence or misjudgment are effectively avoided, and the probability of equipment failures and accidents is reduced. Actual application data shows that after adopting the method of the present invention, the misoperation rate in the power plant central control room is reduced by more than 80%, significantly improving the safety and stability of power plant operation.

[0072] The operation of the power plant central control room requires extremely high precision, and minor operation deviations may have an adverse impact on equipment operation. The visual servo system of the present invention realizes precise positioning and operation feedback of the operation target. When fine-tuning equipment parameters, operators can accurately control the adjustment amplitude through visual feedback. For example, when adjusting the voltage parameter of a certain device, the operator adjusts the parameter value through gesture operation, and the system will display the change of the current parameter value and the expected effect after adjustment on the screen in real time. Operators can accurately control the adjustment amplitude according to this visual feedback information to avoid excessive parameter deviation caused by inaccurate operation. Actual test results show that when using the method of the present invention for operation, the accuracy of equipment parameter adjustment is improved by more than 90%, effectively ensuring the stable operation and performance optimization of the equipment.

[0073] Each operator has their own unique operating habits and requirements, and it is difficult for traditional general operating interfaces to meet personalized requirements. The personalized operating interface customization module of the present invention provides an interface that conforms to each operator's personal operating habits. Operators can customize the interface according to their own needs to make operations more convenient and efficient. For example, for operators who are accustomed to using large fonts for display, the interface customization module can enlarge the font to a suitable size; for operators who prefer a simple operating interface, the interface layout can be simplified to only retain the commonly used function modules. Through this personalized customization, operators can handle the operating interface more proficiently, and the operation comfort and job satisfaction are greatly improved. According to the user feedback survey, after adopting the personalized operating interface, the satisfaction of operators with the operating interface has increased by more than 95%, effectively reducing the learning cost and operation inconvenience caused by adapting to the general interface, and further improving work efficiency and operation accuracy.

[0074] A touch screen control operation method for deep learning and visual servo in a power plant centralized control room proposed by the present invention comprehensively improves the intelligence, efficiency, accuracy, and personalization level of power plant centralized control room operations through the organic combination of a series of innovative technologies such as data acquisition and preprocessing, deep learning model construction and training, visual servo control, operation decision-making and execution, and personalized operating interface customization. In practical applications, this method can significantly improve operation efficiency, reduce the risk of misoperation, improve operation accuracy, and provide a personalized operation experience, providing strong technical support for the safe, stable, and efficient operation of power plants.

[0075] The above description is only a preferred embodiment of the present invention patent and is not intended to limit the present invention patent. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention patent shall be included within the protection scope of the present invention patent.

Claims

1. A touch - screen control operation method for deep learning and visual servo in a power plant's centralized control room, characterized in that, It includes the following steps: S1: Data acquisition and preprocessing. Obtain the image data of the centralized control room interface and preprocess it. Collect the RGB images of the display screen through a camera, and then perform standardization and noise injection operations on the images; S2: Deep learning model construction and training. Use the deep learning framework to construct a convolutional neural network (CNN) model. The construction of the model includes defining the network structure, setting hyperparameters, loading the dataset and training. During the training process, optimize the parameters through the backpropagation algorithm to ensure that the model can accurately identify the interface characters and pose information; S3: Visual servo control. The visual servo technology detects and identifies the target position through vision, and combines the pose measurement information to drive the robotic arm to complete precise operations. Detect the character buttons to be clicked through the image template matching method, and combine the CNN to determine the pose information of the display screen to achieve precise operations on the control interface; S4: Operation decision-making and execution. With the support of the deep learning model and visual servo technology, the system quickly judges the state of the current working page, and generates corresponding control instructions according to the image recognition results of the operation interface, and transmits them to the robotic arm through the PLC control system to complete the automated operation; S5: Personalized operation interface customization. Intuitively adjust the interface layout and function modules through touch screen and gesture control technologies.

2. A touch screen control operation method for deep learning and visual servo in a power plant centralized control room according to claim 1, characterized in that: In the step S1, the data acquisition and preprocessing include the following steps: S11: Image data acquisition: Install multiple high-definition cameras in the operation area of the power plant centralized control room to collect the hand movement images of the operator during the touch screen operation and the display images of the operation interface; S12: Sensor data acquisition: Integrate pressure sensors and capacitive sensors on the touch screen device to collect the pressure and touch position information when the operator touches the screen; S13: Data preprocessing: Denoise, grayscale, and normalize the collected image data.

3. A touch screen control operation method for deep learning and visual servo in a power plant centralized control room according to claim 1, characterized in that: In the step S2, the deep learning model construction and training include the following steps: S21: Action recognition: Use a convolutional neural network (CNN) to construct a hand movement recognition model, and accurately recognize the click, slide, and zoom gestures of the operator by learning the hand movement images; S22: Equipment status prediction: Use a recurrent neural network (RNN) or long short-term memory network (LSTM) to construct an equipment status prediction model. Use the historical parameter data of the equipment operation and the current operation information as inputs to predict the future operation status of the equipment. Predict whether the equipment fails by analyzing the voltage and current change trends and the adjustment operations of the operator over a period of time.

4. A touch screen control operation method for deep learning and visual servo in a power plant centralized control room according to claim 1, characterized in that: In the step S3, the visual servo control includes the following steps: S31: Target detection and positioning: Based on the deep learning-based target detection algorithm, perform real-time detection and positioning of the control elements on the operation interface. When the hand movement of the operator is recognized, quickly determine the position of the control element corresponding to the action through the visual servo system to achieve precise operation positioning; S32: Operation guidance and feedback: When the operator performs touch screen operations, the visual servo system provides real-time operation guidance on the screen based on the recognized hand movements and the positions of the target control elements. After the operation is completed, the system promptly feedbacks the operation result, and informs the operator whether the operation is successful through color changes and animation effects.

5. A touch screen control operation method for deep learning and visual servo in a power plant centralized control room according to claim 1, characterized in that: In the step S4, the operation decision-making and execution include the following steps: S42: Operation decision-making formulation: According to the hand movements recognized by the deep learning model, the predicted device states, and the target positioning information provided by the visual servo system, formulate operation decisions. When the operator makes a gesture to click on a certain control button and the device state prediction model shows that the device is currently in a normal operating state, the system determines that the operation complies with the operation rules and allows execution; S42: Operation execution and monitoring: Convert the formulated operation decisions into specific control instructions, send them to the control system of the power plant to execute the corresponding operations, and monitor the operation execution process in real time. If abnormal situations occur during the operation, including device response timeouts and abnormal parameter changes, stop the operation immediately and issue an alarm.

6. A touch screen control operation method for deep learning and visual servo in a power plant centralized control room according to claim 1, characterized in that: In the step S5, the personalized operation interface customization includes the following steps: S51: User preference learning: By analyzing the historical operation data of the operator, including operation habits and frequently used function modules, use machine learning algorithms to learn the personalized preferences of each operator, and count the viewing frequencies of different device parameters by the operator at different time periods, as well as the usage times of various operation functions; S52: Interface customization generation: Generate a personalized operation interface for each operator according to the learned user preferences. On the interface, place the frequently used function modules and control elements of the operator in prominent and easy-to-operate positions, and at the same time adjust the color, font size, and display parameters of the interface according to the operator's habits.

Images