Non-drive cross-computer control and data acquisition system

Abstract

The invention provides a drive-free cross-computer control and data acquisition system, which comprises a display, a video acquisition terminal, a host, a simulation mouse and a target intranet, and is characterized in that the display is connected with the video acquisition terminal through an HDMI, the simulation mouse is connected with the host through a USB control line and is connected with the target intranet through a local area network, and the target intranet is connected with the host through a USB control line. The video acquisition terminal captures pictures of the display in real time and converts the pictures into data streams, the simulation mouse receives control instructions from the host and converts the control instructions into operation data, the target intranet receives the operation data and the data streams for processing, automatic control over controlled equipment is achieved, the system does not need to be driven and installed, deployment and use are simplified, and the operation efficiency is improved. The method has high compatibility and safety, is suitable for industrial automation, remote operation and other scenes, and has good real-time performance, stability and operation convenience.

Description

Technical Field

[0001] The invention relates to a driverless cross-computer control and data acquisition system. Background Art

[0002] In the field of computer technology, with the acceleration of informatization and the continuous growth of automation needs, the demand for cross-computer operations and data acquisition is becoming more and more common, especially in multiple application scenarios such as industrial automation, remote office, technical support, and data monitoring.

[0003] Existing cross-device data transmission solutions usually rely on the underlying drivers of the operating system or the installation of special control software. Traditional solutions such as remote desktops and KVM switches require the installation of client software or drivers on the controlled device to establish a stable communication connection. These methods require the installation of specific client programs or drivers on the controlled computer, which cannot be effectively implemented in an environment where the controlled device cannot install additional drivers for security reasons.

[0004] Therefore, how to achieve driverless cross-computer control and data acquisition is a problem that needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to solve the above shortcomings of the prior art and provide a driverless cross-computer control and data acquisition system, which realizes cross-computer automatic control and data acquisition by simulating a mouse and a video acquisition terminal, and does not need to install any driver or software on the controlled computer. The purpose of the present invention is achieved as follows:

[0006] The present invention provides a driverless cross-computer control and data acquisition system, comprising a display, a video acquisition terminal, a host, a simulated mouse and a target intranet, wherein the display and the video acquisition terminal are connected via HDMI, the host and the video acquisition terminal are connected via HDMI, the simulated mouse is connected to the host via a USB control line, the simulated mouse is connected to the target intranet via a local area network, and the video acquisition terminal is connected to the target intranet via a local area network; the display is used to display pictures from the host; the video acquisition terminal is used to acquire all pictures on the display in real time; the simulated mouse is used to receive control instructions from the host and convert the instructions into operation data of the simulated mouse; the video acquisition terminal is used to capture the screen image of the display and convert the screen image into a data stream; the target intranet is used to receive the operation data and the data stream.

[0007] Furthermore, the video acquisition terminal includes an image processing module, which is used to preprocess and analyze the displayed image through an image recognition model, wherein the preprocessing includes denoising, grayscale, normalization, and cropping and scaling the captured display image in sequence, and the analysis includes feature extraction, target positioning, classification recognition, and operation decision generation for the preprocessed image through the image recognition model.

[0008] Furthermore, the image processing module is trained through the following steps: collecting and annotating image data of the display, the data including image samples of different resolutions, color modes and application program interfaces; selecting a model architecture based on a convolutional neural network, and preliminarily configuring the number of network layers, activation functions, loss functions, etc. of the model architecture; inputting the image samples into the model architecture for forward propagation, calculating the output of the model architecture and comparing it with the true label; calculating the error through the back-propagation algorithm and adjusting the parameters of the model architecture to minimize the loss function; evaluating the trained model on the validation set and adjusting the hyperparameters; repeating the forward propagation, back-propagation and evaluation and tuning steps until the recognition accuracy of the model reaches the set requirements.

[0009] Furthermore, the simulated mouse is used to receive control instructions from the host, and convert the instructions into operation data of the simulated mouse, including: receiving control instructions from the host, the control instructions including mouse position, click events and scrolling events; converting the received control instructions into operation data.

[0010] Furthermore, the conversion of the received control instruction into operation data includes: parsing the x, y coordinate values ​​of the mouse position, parsing the trigger state of the click event, parsing the direction of the scroll instruction, and parsing the number of pixels of the scroll instruction; converting the coordinate value, the trigger state, the direction of the scroll instruction and the number of pixels of the scroll instruction into relative incremental data of mouse movement.

[0011] Furthermore, the target intranet includes a data exchange module, a network management module, a data processing module and a controlled device. The data exchange module receives the data stream and operation data from the video acquisition terminal, parses the data stream and operation data and forwards them to the network management module; the network management module parses the data stream and operation data according to the control protocol to generate target data, and passes the parsed target data to the data processing module; the data processing module combines the received target data with the target image data of the target intranet to generate automated control instructions for the controlled device.

[0012] Furthermore, the data exchange module receives the data stream and operation data from the video acquisition terminal, parses the data stream and operation data, and forwards them to the network management module, including: the data exchange module receives the data stream and operation data from the video acquisition terminal through the network interface; extracts the image data in the video frame for decoding processing, and converts the decoded image data into the original image format; parses the control instructions in the operation data, extracts the actions input by the user; and forwards the parsed data stream and operation data to the network management module at the same time.

[0013] Furthermore, the target data is an instruction set that can drive the controlled device to perform operations.

[0014] Furthermore, the data processing module combines the received target data with the target image data in the target intranet to generate an automation control instruction for the controlled device, including: the data processing module connects the target data and the target image data in a time or frame order.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: by simulating the combination of a mouse and a video acquisition terminal, cross-computer automated control and data acquisition are realized, without the need to install any driver or dedicated software on the controlled computer, thereby simplifying the deployment and use of the system; the system avoids the cumbersome device configuration and driver installation process in traditional solutions by simulating mouse control and the transmission of image data streams, and the operation process is more concise. Users do not need to master complex hardware configurations and can quickly deploy and use the system, greatly improving the convenience of operation and the operability of the system; since the system does not need to install a driver, system failures that may be caused by software or driver conflicts are avoided, thereby enhancing the stability and security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the connection structure of a driverless cross-computer control and data acquisition system;

[0017] Figure 2 It is a schematic diagram of the device structure of a driverless cross-computer control and data acquisition system;

[0018] In the figure: 100, display, 200, host, 300, video acquisition terminal, 310, image processing module, 400, simulated mouse, 500, target intranet, 510, data exchange module, 520, network management module, 530, data processing module, 540, controlled device. DETAILED DESCRIPTION

[0019] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with embodiments and drawings. The embodiments are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0020] like Figure 1-2 As shown, an embodiment of the present invention provides a driverless cross-computer control and data acquisition system, including a display 100, a video acquisition terminal 300, a host 200, a simulated mouse 400 and a target intranet 500, wherein the display 100 and the video acquisition terminal 300 are connected via HDMI, the host 200 and the video acquisition terminal 300 are connected via HDMI, the simulated mouse 400 is connected to the host 200 via a USB control line, the simulated mouse 400 is connected to the target intranet 500 via a local area network, and the video acquisition terminal 300 is connected to the target intranet 500 via a local area network; the display 100 is used to display the screen from the host 200; the video acquisition terminal 300 is used to acquire all the screens on the display 100 in real time; the simulated mouse 400 is used to receive the control instructions from the host 200 and convert the instructions into the operation data of the simulated mouse 400; the video acquisition terminal 300 is used to capture the screen image of the display 100 and convert the screen image into a data stream; the target intranet 500 is used to receive the operation data and the data stream.

[0021] It should be noted that: the display 100 is a hardware device for displaying the image output by the host 200, which is composed of an LCD, LED or OLED panel and can present the image or video output content of a computer or other device. In the present invention, the display 100 is used to display the screen from the host 200 and transmit it to the user for visual feedback; the video acquisition terminal 300 is a device connected to the display 100 via an HDMI interface, which can acquire the image or video displayed on the display 100 in real time and convert it into a digital data stream for processing. In the present invention, the terminal is responsible for capturing the image of the display 100 and converting the image into a data stream for transmission to the target intranet 500; the simulated mouse 400 is a device for receiving control instructions from the host 200. The device is connected to the host 200 via a USB control line and connected to the target intranet 500 via a local area network to achieve cross-computer control; the target intranet 500 is a local area network, connecting the controlled device 540, the video acquisition terminal 300 and the simulated mouse 400. It is used to receive operation data and data streams, and coordinate data exchange to achieve cross-computer control and data acquisition, so that the controlled device 540 can perform corresponding automated operations; the controlled device 540 is a controlled device in the target intranet 500, usually referring to a controlled computer, mechanical device or other industrial control equipment, or it can also be a screen.

[0022] Optionally, in one use scenario of the present invention, in the power industry, especially in the power grid monitoring center, it is necessary to monitor the operating status and data collection of multiple power devices in real time, and control and manage these devices at the same time. Since the devices are widely distributed and many controlled devices 540 do not allow any additional software or drivers to be installed, it is required to implement an automatic control and data acquisition system that does not require the installation of drivers or software on the controlled devices 540. The specific use steps are as follows:

[0023] The power grid monitoring host 200 generates power grid control instructions. The power grid control instructions are to adjust the power grid load, switch the power source, start or stop the equipment. The control instructions are displayed on the power grid control panel through the interaction between the host 200 and the display 100, and the display 100 displays the relevant information in real time; the video acquisition terminal 300 is connected to the display 100 via HDMI, and captures the power grid control panel screen on the display 100 in real time, and converts the image data into a data stream and transmits it to the target intranet 500; the monitor operates the simulated mouse 400 through the host 200 interface to select a specific device or function in the power grid control panel. The simulated mouse 400 converts the control instructions into operation data and transmits it to the target intranet 500; the target intranet The data exchange module 510 of 500 receives the image data stream from the video acquisition terminal 300 and the operation data of the simulated mouse 400, parses and transmits it to the network management module 520; the network management module 520 parses the operation data according to the control protocol and generates the target data, and transmits it to the data processing module 530; the data processing module 530 combines the image data and the control instruction to generate the automatic control instruction for the power grid equipment, and finally transmits it to the controlled device 540 through the intranet to realize the automatic control of the equipment; the controlled device 540 performs the corresponding operation, and the control feedback is updated on the display 100 of the monitoring center through the image of the video acquisition terminal 300, and the monitor can adjust the control strategy according to the feedback. Through this system, the power grid monitoring center can realize cross-device automatic control and data acquisition without installing any driver on the controlled device 540, which improves the efficiency and safety of power grid management. At the same time, since the system does not rely on the driver of the controlled device 540, it avoids the compatibility and safety problems caused by the driver installation.

[0024] In this embodiment, by simulating the combination of the mouse 400 and the video acquisition terminal 300, cross-computer automated control and data acquisition are achieved without installing any driver or dedicated software on the controlled computer, thereby simplifying the deployment and use of the system. The system avoids the cumbersome device configuration and driver installation process in the traditional solution by simulating the transmission of the mouse 400 control and image data stream, and the operation process is more concise. The user does not need to master complex hardware configuration and can quickly deploy and use it, which greatly improves the convenience of operation and the operability of the system. Since the system does not need to install a driver, system failures that may be caused by software or driver conflicts are avoided, thereby enhancing the stability and security of the system.

[0025] Furthermore, the video acquisition terminal 300 includes an image processing module 310, which is used to pre-process and analyze the displayed image through an image recognition model, wherein the pre-processing includes denoising, graying, normalizing, and cropping and scaling the captured display 100 image in sequence, and the analysis includes feature extraction, target positioning, classification recognition, and operation decision generation for the pre-processed image through the image recognition model. By optimizing and analyzing the image obtained from the display 100, the accuracy and real-time performance of image processing are ensured, the control accuracy and response speed of the system are improved, the image interference and recognition accuracy problems in complex scenes are further solved, and the quality of automatic control and data acquisition is optimized.

[0026] Furthermore, the image processing module 310 is trained through the following steps: collecting and annotating image data of the display 100, the data including image samples of different resolutions, color modes and application program interfaces; selecting a model architecture based on a convolutional neural network, and preliminarily configuring the number of network layers, activation functions, loss functions, etc. of the model architecture; inputting the image samples into the model architecture for forward propagation, calculating the output of the model architecture and comparing it with the true label; calculating the error and adjusting the parameters of the model architecture through the back-propagation algorithm to minimize the loss function; evaluating the training model on the validation set and adjusting the hyperparameters; repeating the forward propagation, back-propagation and evaluation and tuning steps until the recognition accuracy of the model reaches the set requirements. It is ensured that the image recognition model can accurately understand the characteristics of various display 100 images, and effectively classify and judge according to different devices and application scenarios, thereby improving the system's automatic control capabilities in uncertain environments, and further solving the problem of poor adaptability of the model to different application interfaces.

[0027] Furthermore, the simulated mouse 400 is used to receive control instructions from the host 200, and convert the instructions into operation data of the simulated mouse 400, including: receiving control instructions from the host 200, the control instructions including mouse position, click events and scroll events; converting the received control instructions into operation data. By accurately parsing the control instructions and converting them into executable operation data, the dependence on the driver is effectively avoided, and cross-device control without installing additional software is realized, which further solves the problem of driver dependence in traditional cross-device control, so as to make the system more simple and compatible.

[0028] Furthermore, the conversion of the received control instruction into operation data includes: parsing the x and y coordinate values ​​of the mouse position, parsing the trigger state of the click event, parsing the direction of the scroll instruction, parsing the number of pixels of the scroll instruction; converting the coordinate value, the trigger state, the direction of the scroll instruction and the number of pixels of the scroll instruction into relative incremental data of the mouse movement. By accurately parsing the details of each control instruction, the accuracy and operability of the operation of the simulated mouse 400 are ensured, and the precise positioning and operation problems in various complex control scenarios are further solved, which improves the user's operation experience and control accuracy.

[0029] Furthermore, the target intranet 500 includes a data exchange module 510, a network management module 520, a data processing module 530 and a controlled device 540. The data exchange module 510 receives the data stream and operation data from the video acquisition terminal 300, parses the data stream and operation data and forwards them to the network management module 520; the network management module 520 parses the data stream and operation data respectively according to the control protocol to generate target data, and transmits the parsed target data to the data processing module 530; the data processing module 530 combines the received target data with the target image data of the target intranet 500 to generate an automated control instruction for the controlled device 540. Through an efficient data parsing and forwarding mechanism, the image data and operation instructions are correctly transmitted to the target device, ensuring that the controlled device 540 can accurately execute the control instruction, further optimizing the transmission efficiency and accuracy of the control instruction, solving the compatibility and data transmission delay problems between different systems, and improving the response speed and control accuracy of the entire system.

[0030] Furthermore, the data exchange module 510 receives the data stream and operation data from the video acquisition terminal 300, parses the data stream and operation data, and forwards them to the network management module 520, including: the data exchange module 510 receives the data stream and operation data from the video acquisition terminal 300 through the network interface; extracts the image data in the video frame for decoding processing, and converts the decoded image data into the original image format; parses the control instructions in the operation data, extracts the user input action; and forwards the parsed data stream and operation data to the network management module 520 at the same time. Through this process, the data exchange module 510 not only ensures the fast decoding and parsing of the data stream and operation data, but also improves the efficiency of the network management module 520 in processing data, further solves the synchronization problem between the data stream and the control instructions, ensures the real-time performance and accuracy, and enhances the stability and fluency of the system.

[0031] Furthermore, the target data is an instruction set that can drive the controlled device 540 to perform operations. The control instructions that can drive the device to execute are generated to ensure that the system can perform automatic control through accurate target data, further solve the multi-task coordination problem in complex control tasks, and improve the accuracy and intelligence level of device operation.

[0032] Furthermore, the data processing module 530 combines the received target data with the target image data of the target intranet 500 to generate an automated control instruction for the controlled device 540, including: the data processing module 530 connects the target data and the target image data in the order of time or frame. By combining the target data and the image data, it is ensured that the automated control instruction can accurately reflect the image changes and real-time operation requirements, further solving the problem of the mismatch between the control instruction and the real-time image information, ensuring a more accurate and dynamic control response, and improving the timeliness and reliability of the device control.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A driverless cross-computer control and data acquisition system, characterized in that: It includes a display, a video acquisition terminal, a host, a simulated mouse and a target intranet, the display and the video acquisition terminal are connected via HDMI, the host and the video acquisition terminal are connected via HDMI, the simulated mouse is connected to the host via a USB control line, the simulated mouse is connected to the target intranet via a local area network, and the video acquisition terminal is connected to the target intranet via a local area network; the display is used to display images from the host; The video acquisition terminal is used to acquire all images on the display in real time; the simulated mouse is used to receive control instructions from the host and convert the instructions into operation data of the simulated mouse; The video acquisition terminal is used to capture the screen image of the display and convert the screen image into a data stream; The target intranet is used to receive the operation data and the data stream.

2. A driverless cross-computer control and data acquisition system according to claim 1, characterized in that: The video acquisition terminal includes an image processing module, which is used to preprocess and analyze the displayed image through an image recognition model, wherein the preprocessing includes denoising, grayscale, normalization, and cropping and scaling the captured display image in sequence, and the analysis includes feature extraction, target positioning, classification recognition, and operation decision generation for the preprocessed image through the image recognition model.

3. A driverless cross-computer control and data acquisition system according to claim 2, characterized in that: The image processing module is trained through the following steps: collecting and annotating image data of the display, the data including image samples of different resolutions, color modes and application program interfaces; selecting a model architecture based on a convolutional neural network, and preliminarily configuring the number of network layers, activation functions, loss functions, etc. of the model architecture; inputting the image samples into the model architecture for forward propagation, calculating the output of the model architecture and comparing it with the true label; calculating the error through the back-propagation algorithm and adjusting the parameters of the model architecture to minimize the loss function; evaluating the trained model on the validation set and adjusting the hyperparameters; repeating the forward propagation, back-propagation and evaluation and tuning steps until the recognition accuracy of the model meets the set requirements.

4. The driverless cross-computer control and data acquisition system according to claim 1 is characterized in that: The simulated mouse is used to receive control instructions from the host and convert the instructions into operation data of the simulated mouse, including: receiving control instructions from the host, the control instructions including mouse position, click events and scroll events; converting the received control instructions into operation data.

5. A driverless cross-computer control and data acquisition system according to claim 4, characterized in that: The converting of the received control instruction into operation data includes: parsing the x, y coordinate values ​​of the mouse position, parsing the trigger state of the click event, parsing the direction of the scroll instruction, and parsing the number of pixels of the scroll instruction; converting the coordinate value, the trigger state, the direction of the scroll instruction and the number of pixels of the scroll instruction into relative incremental data of mouse movement.

6. The driverless cross-computer control and data acquisition system according to claim 1 is characterized in that: The target intranet includes a data exchange module, a network management module, a data processing module and a controlled device. The data exchange module receives data streams and operation data from the video acquisition terminal, parses the data streams and operation data, and forwards them to the network management module. The network management module parses the data stream and the operation data according to the control protocol to generate target data, and transmits the parsed target data to the data processing module; The data processing module combines the received target data with the target image data in the target intranet to generate an automation control instruction for the controlled device.

7. A driverless cross-computer control and data acquisition system according to claim 6, characterized in that: The data exchange module receives the data stream and operation data from the video acquisition terminal, parses the data stream and operation data and forwards them to the network management module, including: the data exchange module receives the data stream and operation data from the video acquisition terminal through the network interface; extracts the image data in the video frame for decoding processing, and converts the decoded image data into the original image format; parses the control instructions in the operation data, extracts the actions input by the user; and forwards the parsed data stream and operation data to the network management module at the same time.

8. The driverless cross-computer control and data acquisition system according to claim 6 is characterized in that: The target data is an instruction set that can drive the controlled device to perform operations.

9. The driverless cross-computer control and data acquisition system according to claim 6, characterized in that: The data processing module combines the received target data with the target image data in the target intranet to generate an automation control instruction for the controlled device, including: the data processing module connects the target data with the target image data in a time or frame order.

Images