Monitoring control system and control method of numerical control production equipment and storage medium

Abstract

The invention is suitable for the technical field of intelligent manufacturing, and provides a monitoring control system and control method of numerical control production equipment and a storage medium, the monitoring control system is connected with the numerical control production equipment, and the monitoring control system is configured to obtain screen content of display equipment when the numerical control production equipment displays operation state data through the display equipment, acquiring a frame image according to the screen content, and acquiring operation state data of the numerical control production equipment according to the frame image so as to perform centralized management on the numerical control production equipment; according to the technical scheme, the problem that data of existing numerical control production equipment cannot be obtained in real time and managed in a centralized mode is solved, real-time monitoring of the equipment state is achieved, manual intervention is reduced, the production efficiency is improved, and the operation cost is reduced; meanwhile, by copying different protocol data, the compatibility problem among different devices is solved, and automation and intelligentization of production management are promoted.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing technology, and in particular to a monitoring and control system, a control method, and a storage medium for numerically controlled production equipment. Background Art

[0002] In the field of intelligent manufacturing, CNC production equipment, such as CNC machine tools, punch presses, and semiconductor manufacturing equipment, has been widely used, enabling high-precision, high-efficiency automated production. However, the data from these CNC production equipment is often not standardized. Different brands of equipment use their own proprietary protocols and lack a unified data interface, making it difficult for production management systems to collect critical operational data and achieve centralized management. Without real-time access to equipment status, factories still require a large number of personnel to maintain operations, increasing operating costs and reducing production efficiency. Summary of the Invention

[0003] The present application provides a monitoring and control system, a control method and a storage medium to solve the problem in the prior art that the equipment status cannot be obtained in real time, the factory still needs a large number of manpower to be stationed, which increases operating costs and reduces production efficiency.

[0004] In a first aspect, a monitoring and control system for a CNC production device is provided, wherein the monitoring and control system is connected to the CNC production device and is configured to:

[0005] When the CNC production equipment displays operating status data through a display device, the screen content of the display device is obtained, a frame image is obtained based on the screen content, and the operating status data of the CNC production equipment is collected based on the frame image to centrally manage the CNC production equipment.

[0006] In the above solution, screen content acquisition and image recognition technology are used to achieve unified data collection and centralized management of CNC production equipment of different brands, without changing the internal system of the equipment, and it has strong adaptability. This solution can extract the operating status of the equipment in real time, reduce manual inspections, improve management efficiency, optimize production scheduling, and effectively improve the production efficiency and operation and maintenance level of intelligent manufacturing.

[0007] In some embodiments, a monitoring and control system is provided, wherein the numerically controlled production equipment includes a control device and a first display device, and the monitoring and control system includes a monitoring module and a second display device, wherein the monitoring module is connected to the control device and the first display device, respectively; when the numerically controlled production equipment displays operating status data via the display device, before acquiring the screen content of the display device, the monitoring module is configured to:

[0008] Receive an original display signal output by the control device, obtain a first display signal and a second display signal that are identical to the original display signal based on the original display signal, send the first display signal to the first display device, and send the second display signal to the second display device.

[0009] In some embodiments, the monitoring module obtains a first display signal and a second display signal that are identical to the original display signal according to the original display signal, and is configured to:

[0010] The original display signal is copied into a first display signal and a second display signal through a signal distributor;

[0011] Alternatively, the original display signal is set as the first display signal, and the original signal is copied to obtain the second display signal.

[0012] In some embodiments, a monitoring and control system is provided. When the numerical control production equipment displays operating status data through a display device, the screen content of the display device is obtained, a frame image is obtained based on the screen content, and the operating status data of the numerical control production equipment is collected based on the frame image. The monitoring module is configured to:

[0013] Decoding the second display signal into a video stream, and displaying the video stream through the second display device;

[0014] Taking a screenshot of each frame of video displayed by the second display device to obtain a screenshot image;

[0015] The screenshot image is identified to obtain the operating status data of the CNC production equipment.

[0016] In the above solution, video data enters the chip through the hardware interface; the compressed video data is decoded into original video frames through the hardware decoder; the decoded frames become a new video stream, which can be used for subsequent remote operations; static frames in the video stream are periodically extracted for screenshots (a certain number of frames per second or a specific time interval); the screenshot images are processed using AI algorithms to extract important information or perform object recognition; text recognition is performed on the screenshot images, and the extracted text is converted into data for subsequent analysis and decision-making.

[0017] In some embodiments, a monitoring and control system is provided, wherein the monitoring module identifies the screenshot image to obtain the operating status data of the CNC production equipment, and is configured to:

[0018] Dividing the screenshot image into multiple areas according to coordinate areas;

[0019] Identify each area and obtain the type of each area;

[0020] Call the OCR text model or AI model for recognition according to the type of each area;

[0021] The identified data is returned and stored in the requested key format.

[0022] The above solution achieves efficient extraction of equipment operating status data from video streams through video decoding, frame extraction, AI analysis, and OCR recognition. First, the compressed video is decoded to ensure that the video data can be subsequently processed in an uncompressed format. Then, through frame extraction technology, key frames are extracted at set time intervals. Next, the AI module analyzes the screenshot image to extract equipment status information, and combines OCR technology to identify numerical and text data in the image and convert it into a processable text format. Ultimately, this solution can achieve automated monitoring of equipment operating status, reduce manual intervention, improve the accuracy and real-time nature of data collection, and provide efficient support for intelligent production management and automated decision-making.

[0023] In some embodiments, a monitoring and control system is provided, which centrally manages the CNC production equipment and is then configured to:

[0024] analyzing the operating status data;

[0025] When the operating status data meets the preset normal conditions, maintaining the current state;

[0026] When the operating status data meets a preset abnormal condition, a response operation is performed.

[0027] In the above solution, through real-time analysis of the equipment operation status data, an automatic execution mechanism based on conditional triggering is established, which effectively improves production efficiency and reduces the risk of failure; by monitoring key parameters (such as temperature, pressure, speed, etc.) and comparing them with preset conditions, once the equipment operation status meets the preset abnormal conditions, relevant operations such as alarm, automatic adjustment or shutdown operations can be automatically executed. The automated control provided by this technical solution can respond to equipment abnormalities in a timely manner, reduce manual intervention and reaction time, and ensure the continuity and safety of the production process. In addition, by flexibly adjusting the preset conditions, the system can adapt to different equipment and working conditions, and has strong scalability and flexibility, thereby effectively ensuring the normal operation of the equipment, improving production efficiency and reducing operating costs.

[0028] In some embodiments, a monitoring control system is provided, wherein the monitoring module is further connected to a keyboard and a mouse, respectively, and the monitoring module is further configured to:

[0029] receiving a keyboard signal output by the keyboard and / or a mouse signal output by the mouse;

[0030] Record the motion trajectory of the keyboard signal and / or the mouse signal during execution, and generate an automated operation script.

[0031] In the above scheme, automatic control and operation script generation are achieved by receiving input signals from the keyboard and mouse and recording the operation trajectories of these signals; by accurately capturing keyboard key operations and every movement and click of the mouse, the user's manual operations can be converted into repeatable automated scripts, significantly improving operational efficiency and accuracy; the generated scripts can automatically execute complex operation processes without human intervention, thereby reducing human errors and improving production efficiency; this implementation method not only supports accurate recording of the operation process, but also has flexible script optimization and custom adjustment functions to ensure execution in different application scenarios.

[0032] In some embodiments, a monitoring and control system is provided, wherein the monitoring module is further connected to a storage module, the storage module is further connected to the control device, and the monitoring module is further configured to:

[0033] The configuration file in the storage module is updated, and the updated configuration file is written into the control device.

[0034] In this solution, the dynamic updating and application of configuration files is achieved by monitoring the connection between the control system and the storage module. By receiving update requests through external input or the system interface, the device can read and modify the configuration files in the storage module. The updated configuration files are successfully written to the control device, which then adjusts its operating mode and parameters based on the new configuration, improving the automation level, operational efficiency, and stability of the equipment. This implementation effectively simplifies the configuration management of production equipment, enhances system flexibility and responsiveness, reduces manual intervention, and improves production efficiency and facilitates equipment maintenance.

[0035] In a second aspect, a monitoring and control method for numerical control production equipment is provided, which is applied to a monitoring and control system connected to the numerical control production equipment. The method comprises the following steps:

[0036] When the CNC production equipment displays operating status data through a display device, the screen content of the display device is obtained, a frame image is obtained based on the screen content, and the operating status data of the CNC production equipment is collected based on the frame image to centrally manage the CNC production equipment.

[0037] In a third aspect, an electronic device is provided, comprising the above-mentioned monitoring and control system.

[0038] In a fourth aspect, an electronic device is provided, comprising a device for executing the control method of the monitoring control system in the second aspect.

[0039] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a display control device, the control device executes the control method of the monitoring control system in the second aspect.

[0040] In a sixth aspect, a computer program product is provided, which includes: a computer program, which, when executed by an electronic device, enables the control device to execute the control method of the monitoring control system in the second aspect.

[0041] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 This is a schematic diagram of a first connection structure of a monitoring and control system for a numerical control production equipment in one embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of a second connection structure of a monitoring and control system for a numerical control production equipment in one embodiment of the present invention;

[0045] Figure 3 This is a flow chart of a monitoring and control system for a numerical control production equipment in one embodiment of the present invention for acquiring equipment operating status data;

[0046] Figure 4 This is a flow chart of a monitoring and control system for a numerical control production equipment in one embodiment of the present invention for identifying a screenshot image;

[0047] Figure 5 This is a flow chart of a monitoring and control system for a numerical control production equipment in one embodiment of the present invention for acquiring and analyzing equipment operating status data;

[0048] Figure 6 This is a schematic diagram of a third connection structure of a monitoring and control system for a numerical control production device in one embodiment of the present invention;

[0049] Figure 7This is a flow chart of a monitoring and control system for numerically controlled production equipment controlling a keyboard and a mouse in one embodiment of the present invention;

[0050] Figure 8 This is a schematic diagram of a fourth connection structure of a monitoring and control system for a numerical control production equipment in one embodiment of the present invention;

[0051] Figure 9 This is a schematic diagram of a fifth connection structure of a monitoring and control system for a numerical control production device in one embodiment of the present invention;

[0052] Figure 10 is a structural diagram of an electronic device in one embodiment of the present invention;

[0053] In the figure: 100, CNC production equipment; 101, control equipment; 102, monitoring module; 103, display device; 104, remote server; 105, keyboard; 106, mouse; 107, storage module; 110, keyboard; 111, first display device; 112, first mouse; 113, first keyboard; 114, second mouse; 115, second keyboard; 116, second display device; 200, monitoring and control system. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] It should be understood that when used in the present specification and the appended claims, unless otherwise specified, the term " / " means or. For example, A / B can mean A or B. "And / or" in this document is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

[0056] In the description of the present invention and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their collections. It should also be understood that the term "and / or" used in the present invention description and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0058] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0059] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0060] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0061] To facilitate a further understanding of the technical solutions in some embodiments of the present application, the following describes in detail the technical solutions of the monitoring and control system, control method, and storage medium of CNC production equipment, as well as how the technical solutions solve the above-mentioned technical problems, in conjunction with some specific embodiments and drawings. The various embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are part of the embodiments of the present application, not all of them.

[0062] In some embodiments, as Figure 1 As shown, a monitoring and control system 200 of a numerical control production device 100 is provided. The monitoring and control system 200 is connected to the numerical control production device 100 and is configured as follows:

[0063] When the CNC production equipment 100 displays the operating status data through the display device 103, the screen content of the display device 103 is obtained, the frame image is obtained according to the screen content, and the operating status data of the CNC production equipment 100 is collected according to the frame image to centrally manage the CNC production equipment 100.

[0064] Among them, the monitoring and control system 200 is used to monitor CNC production equipment 100 (such as CNC machine tools, automated assembly lines, punching machines, semiconductor manufacturing equipment, etc.). The monitoring and control system 200 is connected to the CNC production equipment 100 and can obtain data from the CNC production equipment 100 or interact with it. The CNC production equipment 100 displays operating status data through the display device 103, which means that the operating status of the CNC equipment (such as temperature, speed, output, alarm information, etc.) will be visually presented on the display device (such as a screen, HMI touch screen). The display device can be a display device built into the CNC production equipment 100 or an external display device. Acquiring the screen content of the display device means that the monitoring and control system 200 will capture or read the screen content of the display device to obtain the operating information of the CNC equipment. A frame image refers to the screen image at a certain moment, usually a static image. Acquiring a frame image based on the screen content means that the monitoring and control system 200 will capture the screen content, which may be obtained by screenshots, photography, or screen acquisition technology to obtain frame-by-frame image data. Collecting the operating status data of the CNC production equipment 100 based on the frame image means extracting the operating status data of the CNC equipment (such as temperature values, equipment alarm information) from the collected frame image through technologies such as image recognition (OCR text recognition, machine vision analysis). Centrally managing the CNC production equipment 100 means that through the above-mentioned data collection process, the monitoring and control system 200 can uniformly monitor the operating status of multiple CNC production equipment 100 to achieve centralized management, for example, remote monitoring of equipment status, recording and analyzing production data, warning of abnormal conditions, automatic control of the operation of CNC production equipment 100, generating reports, etc. The difference between this embodiment and the prior art is that the monitoring and control system 200 does not obtain data through the communication interface of the equipment, but through visual monitoring (reading data information on the screen). This method is applicable to a variety of different models of equipment that do not have a unified standard or equipment that cannot directly obtain data.

[0065] The technical effect of this embodiment is that: through screen content acquisition and image recognition technology, unified data collection and centralized management of CNC production equipment 100 of different brands can be achieved without changing the internal system of the equipment, and it has strong adaptability; this solution can extract the equipment operation status in real time, reduce manual inspections, improve management efficiency, optimize production scheduling, and effectively improve the production efficiency and operation and maintenance level of intelligent manufacturing.

[0066] As an implementation method, Figure 2As shown, the numerical control production equipment 100 includes a control device 101 and a first display device 111, and the monitoring and control system 200 includes a monitoring module 102 and a second display device 116. The monitoring module 102 is connected to the control device 101 and the first display device 111 respectively. When the numerical control production equipment 100 displays operating status data through the display device, before obtaining the screen content of the display device, the monitoring module is configured as follows:

[0067] The receiving control device 101 receives the original display signal output, obtains a first display signal and a second display signal identical to the original display signal according to the original display signal, sends the first display signal to the first display device 111, and sends the second display signal to the second display device 116.

[0068] The control device 101 generates, processes, and outputs raw display signals, coordinates and manages various types of information during the production process, provides data input to the monitoring module 102, and determines the equipment's operating status and actions. Based on parameters such as production demand, equipment status, and production schedule, the control device 101 generates raw display signals. These signals may include equipment operating status, production parameters, and real-time data (such as temperature, pressure, and speed). The control device 101 transmits these raw display signals to the monitoring module 102 via a communication interface. These signals, which may be digital or analog, have specific formats and content for subsequent display and processing. The control device 101 is also responsible for scheduling and managing equipment during the production process, ensuring that production tasks are executed as planned. It coordinates and controls the operation of the entire production system through communication with other production equipment. The functions of the monitoring module 102 include signal reception and processing, forwarding and uploading display signals, and real-time monitoring of production status. The monitoring module 102 receives the raw display signals from the control device 101 via an input interface, processes them, and generates two identical display signals. The first display signal is transmitted to the first display device 111 for on-site display, while the second display signal is further processed and output to the second display device 116. The monitoring module 102 monitors real-time data (such as temperature, speed, and equipment status) during the production process and, via the second display signal, feeds this data into the system's internal processing flow. The monitoring module 102 can automatically perform certain operations, such as adjusting equipment settings and starting / stopping the production line, to ensure normal production. The second display device 116 is a display unit directly connected to the monitoring module 102, used to display real-time data and equipment status during the current production process. It serves as a tool for on-site operators to monitor the production process in real time. The first display device 111 receives the first display signal from the monitoring module 102, which contains real-time production data, equipment status, parameter settings, and other information. Based on the received display signal, the first display device 111 updates various information about the production process in real time. The displayed content may include temperature, pressure, speed, production task progress, etc., allowing on-site operators to keep abreast of production conditions. The second display device 116 can display the same content or, based on user instructions, other content. The second display signal contains real-time information about the equipment operation, such as production status, equipment parameters, sensor readings, temperature, humidity, pressure, etc. The monitoring module 102 monitors the operation of the equipment in real time and stores the received display signals and equipment status data in the database. The stored data can be real-time data or historical data for subsequent query, analysis or decision support.The monitoring module 102 performs real-time analysis on received equipment data, for example, to determine whether parameters such as temperature and humidity exceed preset safety thresholds, and whether there are any equipment anomalies or signs of failure. Using rules or algorithms, the data is processed in real time to analyze the equipment's operating status and production progress. If an abnormality is detected (such as excessively high equipment temperature or unstable voltage), an alarm signal is immediately issued. This alarm information is notified to the relevant operator or management personnel via the network, allowing them to take necessary preventative measures, such as adjusting equipment settings or activating automatic protection mechanisms. Through the monitoring module 102, management personnel can view the operating status, production progress, and operating parameters of the CNC production equipment 100 in real time.

[0069] As an example, consider a large factory with multiple sets of CNC production equipment and display devices. Each device is connected to a CNC production machine and a display device. Each CNC production machine generates real-time display signals reflecting the machine's operating status, operational data, and alarm information. These signals are typically displayed to operators via local display devices. However, to improve management efficiency, the factory also desires centralized remote monitoring and operation. In this scenario, the monitoring and control system can function as follows: It receives raw display signals from the CNC production equipment. Each production machine generates display signals through the CNC production equipment. These signals may include real-time operating data (such as temperature, pressure, and speed), status information (normal, fault, warning, etc.), and operational prompts. Based on the received raw display signals, the monitoring and control system generates two display signals. The first display signal is sent to the local display device, allowing on-site operators to view the machine's operating status and perform operations. The second display signal is fed into the system's internal processing flow and displayed on a second display device. Remote servers and networks can also be used for centralized management and remote monitoring, serving as a centralized management platform to aggregate data from different devices. Factory managers can monitor the operating status of all equipment through a centralized system, providing real-time access to the status of each piece of production equipment, regardless of its physical location. Managers can view the display information of each piece of production equipment through the remote system and, if problems are discovered, remotely operate the system. This can involve simulating a keyboard and mouse to adjust device parameters, start or stop equipment, and even perform troubleshooting operations. This allows managers to manage multiple pieces of equipment on behalf of on-site operators, reducing the time and labor costs of traveling to each individual facility.

[0070] The technical effect of this technical solution is that by dividing the display signal of the monitoring module 102 into two paths, one path is transmitted to the local display device, and the other path is brought into the internal processing flow of the system, the problem that the existing equipment data cannot be obtained in real time and centrally managed is solved. This technical solution realizes real-time monitoring of the equipment status, reduces manual intervention, improves production efficiency, and reduces operating costs; at the same time, by copying data of different protocols, it solves the compatibility problem between different devices and promotes the automation and intelligence of production management.

[0071] As an embodiment, the monitoring module 102 generates a first display signal and a second display signal that are identical to the display signal according to the display signal, and is configured as follows:

[0072] The original display signal is copied into a first display signal and a second display signal through a signal distributor;

[0073] Alternatively, the original display signal is set as the first display signal, and the original signal is copied to obtain the second display signal.

[0074] In this technical solution, the monitoring module 102 generates the first display signal and the second display signal identical to the original display signal according to the original display signal, including but not limited to the following implementations:

[0075] The first embodiment is a hardware embodiment, in which the original display signal is copied into a first display signal and a second display signal through a signal distributor. The signal distributor is a hardware module that copies the original display signal, and the circuit inside the signal distributor divides the original signal into two identical signals. Generally speaking, the signal distributor uses electronic components (such as amplifiers, current drivers, etc.) to enhance signal strength and prevent signal attenuation. In this technical solution, the signal distributor receives the original display signal output by the control device 101 and copies it into two identical signals. The first display signal will be directly transmitted to the display device 103 for the on-site operator or monitoring personnel to view the operating status of the production equipment. The second display signal is sent to the remote server 104 for remote monitoring and data analysis, helping managers to grasp the operating status of the equipment in real time.

[0076] The second implementation method is a software implementation method, in which the original display signal is set as the first display signal, and the second display signal is obtained by copying the original signal. The original display signal is configured as the first display signal, and the first display signal is directly transmitted to the display device 103 to show the real-time status of the device to the operator. The second display signal is obtained by copying the original display signal. Through the copying process, the second display signal is exactly the same as the original signal, ensuring the integrity and consistency of the signal. For example, by copying the display signal through software, it can be displayed on the display device. The main steps include: obtaining the screen image in real time through the API or graphics library provided by the operating system; compressing and encoding the captured display content for efficient transmission; sending the signal to the remote device through the network protocol, commonly used protocols include RTSP, HLS, WebRTC, etc.; receiving and decoding the signal, and displaying it on the screen to achieve two-way interaction.

[0077] The technical effect of this embodiment is that: directly copying the display signal through the signal distributor, or setting the original signal as the first display signal and then copying it to obtain the second display signal, can ensure the accurate transmission and real-time synchronization of the original signal. Through this mechanism, the local display of the device and the data acquisition system of the remote server 104 can work synchronously to achieve full monitoring and management of the device.

[0078] As an embodiment, after the monitoring module 102 generates the first display signal and the second display signal identical to the original display signal according to the original display signal, the monitoring module 102 is further configured to:

[0079] The second display signal is identified to obtain the equipment operation status data.

[0080] The second display signal is identified to obtain the device operation status data. The specific process is as follows:

[0081] The monitoring module 102 replicates the original display signal through a signal distributor or software, generating two identical signals: a first display signal and a second display signal. The second display signal is used in the subsequent recognition process. The second display signal is typically a video signal containing text, graphics, color changes, and other content. The intelligent device converts the second display signal into a processable data format, involving signal decoding and data extraction. Specifically, the display signal passes through an acquisition card or other hardware module and is converted into a digital signal for further processing. Using image recognition or computer vision technology, the intelligent device processes the second display signal and can extract text content (such as temperature, pressure, speed, and other data) from the display signal using OCR (optical character recognition). If the signal contains other data forms (such as graphics or curves), algorithms such as image classification, edge detection, and pattern recognition are required for recognition. During the recognition process, the monitoring module 102 extracts key operating status information from the image or digital signal, such as the device's operating temperature, operating speed, and status flags. This data can be used to determine the device's current operating status. The identified operational status data is converted into a standard format, such as numeric data or JSON, for subsequent processing. This data can then be fed into the system's internal processing flow, displayed on a local display device, or used to trigger automated actions (such as alarms and parameter adjustments).

[0082] The technical effect of this embodiment is that, by identifying the second display signal, monitoring module 102 can obtain real-time equipment operating status data, including key parameters such as temperature, speed, and pressure. This effectively improves the accuracy and real-time nature of data acquisition during the production process, reduces manual intervention, and reduces the risk of operational errors. Furthermore, real-time acquisition of equipment operating status data provides a basis for subsequent automated control and decision-making, optimizing production processes and improving production efficiency and equipment utilization.

[0083] As an implementation method, Figure 3 As shown, the second display signal is identified to obtain the device operation status data, which is configured as follows:

[0084] Step S101: Decode the second display signal into a video stream, and display it through a second display device.

[0085] Step S102: Taking a screenshot of each frame of video displayed on the second display device to obtain a screenshot image.

[0086] Step S103: Identify the screenshot image to obtain the operating status data of the CNC production equipment.

[0087] Among them, this embodiment is to decode and convert the video data into a video stream after entering the chip, and then take frame screenshots for subsequent AI model analysis and OCR data conversion.

[0088] In step S101, a hardware decoder (e.g., an H264, HEVC, or MJPEG decoder chip) is used to decode the video data, converting the compressed video data into original uncompressed video frames (e.g., YUV or RGB format) for subsequent processing. The decoder extracts each frame of video data from the video stream and transmits it sequentially to subsequent processing modules. The decoded video signal of each frame becomes a video stream and is transmitted to the second display device 116 via a data transmission interface for display.

[0089] In step S102, each frame of video displayed on the second display device 116 is extracted within a certain time interval. For example, a set time interval (30 frames per second, or one frame every five frames) can be used to obtain continuous static frames. A dedicated hardware acceleration module (such as the frame extraction function in a video decoder) can be used to efficiently extract each frame of image, reducing processing delay. Software or hardware can also be used to extract specific frames from the video stream. Each frame in the video stream can be saved as an image and further analyzed. A specific image region or the entire image can be extracted from the video frame using an image processing library (such as OpenCV) or a specialized hardware interface (such as a GPU acceleration module). To ensure that the screenshot operation does not affect the playback of the real-time video stream, the screenshot operation is typically performed after the video is decoded. Each decoded frame data is temporarily stored and processed using a frame buffer. Within a set time interval, the software can select a specific frame for screenshot and save it as an image file (such as JPEG, PNG, etc.). The hardware image processing unit (GPU) can be used to accelerate frame extraction and conversion.

[0090] Among them, in step S103, the extracted screenshot image is passed to the AI module for processing. The AI module usually uses a deep learning algorithm (such as a convolutional neural network CNN) to extract useful features in the image. Through the trained AI model, the screenshot image can be classified, object detected, or other image analysis tasks can be performed. After the captured image is analyzed by AI, if the image contains text (such as numerical values, status information, etc.), OCR technology can be used to identify the text in the image and convert it into processable text data. The screenshot image is located and extracted by OCR software, and the extracted text is subjected to subsequent data processing, such as error correction, formatting, and comparison with the system database. AI analysis and OCR can be combined through a unified image processing platform to jointly process image data.

[0091] The working process of this embodiment is as follows: the compressed video data is decoded into original video frames through a hardware decoder; the decoded frames become a new video stream, which can be used for subsequent remote operations; static frames in the video stream are regularly extracted for screenshots (a certain number of frames per second or a specific time interval); the screenshot images are processed using AI algorithms to extract important information or perform object recognition; text recognition is performed on the screenshot images, and the extracted text is converted into data for subsequent analysis and decision-making.

[0092] The technical effect of this implementation is as follows: through video decoding, frame extraction, AI analysis, and OCR recognition, the function of efficiently extracting equipment operating status data from video streams is realized. First, the compressed video is decoded to ensure that the video data can be subsequently processed in an uncompressed format; then, through frame extraction technology, key frames are extracted at set time intervals; then, the screenshot image is analyzed using the AI module to extract equipment status information, and combined with OCR technology to identify numerical and text data in the image and convert it into a processable text format. Ultimately, this solution can realize automated monitoring of equipment operating status, reduce manual intervention, improve the accuracy and real-time performance of data collection, and provide efficient support for intelligent production management and automated decision-making.

[0093] As an implementation method, Figure 4 As shown, the monitoring and control system recognizes the screenshot image to obtain the equipment operation status data, which is configured as follows:

[0094] Step S121: Divide the screenshot image into multiple areas according to the coordinate areas.

[0095] Step S122: Identify each region and obtain the type of each region.

[0096] Step S123: Call the OCR text model or AI model for recognition according to the type of each area.

[0097] Step S124: Return and store the identified data in the requested key mark format.

[0098] Among them, the purpose of step S121 is to decompose the screen content and divide the entire screenshot into multiple different areas so that different recognition methods can be used for different areas later. The operation process is: load the screenshot image (usually RGB or grayscale image); divide the screen content into multiple areas through predefined templates or automatic detection methods, such as: numerical area (such as temperature, speed, pressure, time), status indication area (such as "running", "fault"), icon / curve area (may contain trend charts, warning symbols); output multiple area coordinates for the next step of recognition processing. Example: Assume that the device screen displays the following content: status information area: device status: running; numerical area: temperature: 180℃, speed: 1200rpm, output: 500 pieces; alarm information area: warning that the temperature is too high. This step will divide the screenshot into: status information area, numerical area (temperature, speed, output), alarm information area (warning message).

[0099] The goal of step S122 is to identify the content type of each region, thereby determining which recognition method to use. Operational process: Use machine learning models, rule matching, or template matching to identify region types: text regions (e.g., "Temperature: 180°C"), numerical regions (e.g., "1200 rpm"), and graphic / curve regions (e.g., device status indicator lights, trend charts). After region classification, output each region's type, for example: Region 1: Status information; Region 2: Numerical value; Region 3: Numerical value; Region 4: Alarm information.

[0100] In step S123, an OCR model (such as Tesseract OCR or PaddleOCR) is used to extract text content on the screen. The AI model is applied to graphic / icon areas, using deep learning (CNN, YOLO, etc.) to identify warning signs, operation icons, etc. Image processing and machine learning are used to extract information such as temperature curves and production trends.

[0101] The goal of step S124 is to format the recognition results and store them in a database or management system. The recognized CNC equipment operation data is stored in a key-value pair format (JSON, database table, etc.).

[0102] The technical effect of this implementation is: through screenshot analysis and OCR / AI recognition, the operating status data of CNC production equipment can be accurately extracted and centrally managed; first, the screenshots are divided according to the coordinate area to distinguish different information areas such as values, status, and alarms; then, text data is recognized through OCR, and the AI model is used to parse graphics or trend information; finally, the data is formatted and stored to support remote monitoring, intelligent early warning and production optimization, effectively solving the problem of inconsistent data standards and difficult collection of equipment from different brands, and improving production management efficiency.

[0103] As an implementation method, Figure 5 As shown, the monitoring module centrally manages CNC production equipment and is configured as follows:

[0104] Step S111: Analyze the operating status data.

[0105] Step S112: When the operating status data meets the preset normal conditions, maintain the current status.

[0106] Step S113: When the operating status data meets the preset abnormal conditions, a response operation is performed.

[0107] This embodiment utilizes an automated execution mechanism based on operational status data analysis. By monitoring operational status data in real time, it automatically executes relevant actions (such as alarms, adjustments, and activating backup plans) when preset conditions are met, thereby improving production efficiency, reducing failure risks, and achieving automated control. This embodiment uses real-time analysis of operational status data to determine whether preset conditions (such as temperature, pressure, and speed) are met, and automatically executes the corresponding actions when these conditions are met. After preprocessing, the operational status data enters the analysis module. The analysis module can analyze the data using a rules engine, machine learning algorithms, or deep learning models. Specifically, it compares key parameters (such as temperature, pressure, speed, and flow rate) with preset conditions to determine whether the device meets predetermined normal / abnormal conditions. Preset conditions are defined based on the device's operating characteristics, usage scenarios, and historical data. For example, temperature exceeding a certain threshold (such as overheating above 80°C), speed falling below the minimum operating range (such as below 500 RPM), and pressure exceeding a set safety range (such as above 10 MPa) can be flexibly adjusted based on historical data, device specifications, and user needs. Once the equipment's operating status data is processed by the analysis module, it will determine in real time whether the data meets the preset conditions. If the data meets the preset normal conditions, the current state of the equipment will be maintained; if the data meets the preset abnormal conditions, an automatic response operation will be executed. Depending on the conditions, the system can perform different operations, including but not limited to the following:

[0108] 1. Alarm operation: When the equipment is abnormal (such as temperature is too high, pressure is too low, etc.), an alarm notification (such as sound, SMS, email, etc.) is triggered to notify the operator or relevant personnel to intervene.

[0109] 2. Automatic adjustment operation: According to the equipment status (such as overtemperature), the equipment settings are automatically adjusted, such as starting the cooling system, reducing the working power, etc. When the main system fails, it automatically switches to the backup system to continue working to ensure uninterrupted production.

[0110] 3. Shutdown operation: When it is detected that the equipment is in an extremely dangerous state (such as overload, overheating, etc.), the system will automatically perform a shutdown operation to avoid equipment damage or production accidents.

[0111] The technical effect of this embodiment is that: through real-time analysis of operating status data, an automatic execution mechanism based on conditional triggering is established, which effectively improves production efficiency and reduces the risk of failure; by monitoring key parameters (such as temperature, pressure, speed, etc.) and comparing them with preset conditions, once the operating status of the equipment meets the preset abnormal conditions, relevant operations such as alarm, automatic adjustment or shutdown operations can be automatically executed. The automated control provided by this technical solution can respond to equipment abnormalities in a timely manner, reduce manual intervention and reaction time, and ensure the continuity and safety of the production process. In addition, by flexibly adjusting the preset conditions, the system can adapt to different equipment and working conditions, and has strong scalability and flexibility, thereby effectively ensuring the normal operation of the equipment, improving production efficiency and reducing operating costs.

[0112] As an implementation method, Figure 6 As shown, the monitoring module 102 is also connected to a keyboard 105 and a mouse 106, respectively. The monitoring module 102 is further configured as follows:

[0113] Step S201 : receiving keyboard signals output by the keyboard 105 and / or mouse signals output by the mouse 106 .

[0114] Step S202 , recording the movement trajectory of the keyboard signal and / or mouse signal during the execution process, and generating an automated operation script.

[0115] In step S201, input signals are received from the keyboard 105 and the mouse 106, primarily including keyboard 105 key input signals and mouse 106 movement, click, scroll, and other signals. These signals represent how the user interacts with the device and are used for subsequent operation recording and automated control. The keyboard 105 and mouse 106 are connected via an interface (such as USB, Bluetooth, etc.) to ensure that the device can receive signals from these input devices. Keyboard 105 key input (such as pressing, releasing, etc.) and mouse 106 actions (such as moving, clicking, etc.) are captured via a driver or input interface. The received raw signals are converted into digital signals that can be read by the device and then transmitted to the device's processing unit for further processing.

[0116] In step S202, the movement trajectories of the keyboard 105 and mouse 106 during the operation process are recorded, including keystrokes and every movement and click of the mouse 106. These operations are then converted into a reusable automated script. In the device's processing unit, a program is used to record every operation from the keyboard 105 and mouse 106. This includes operations caused by keyboard signals, such as key presses and releases, the order and duration of key presses, and mouse signals, such as the movement path, click locations, double-clicks, and right-clicks of the mouse 106. Each operation event is timestamped, recording the time at which each signal occurred, ensuring that the sequence and timing of the operations can be accurately replicated in the automated script. Based on the recorded keyboard 105 and mouse 106 operation trajectories, a script code is generated. This script contains all the operation steps entered by the user and can be automatically executed without manual operation. These scripts can be saved in common script formats (such as Python, Shell, AutoHotkey scripts, etc.) and called upon by the device when needed. The generated scripts can be optimized, such as removing unnecessary repetitive operations and reducing unnecessary delays. Users can also manually adjust the scripts or set different operating parameters to ensure that the execution effect in actual applications meets the requirements.

[0117] As an example, assume there is a device that requires adjustment of certain parameters using keyboard 105 and mouse 106. Typically, an operator would manually modify device settings (such as adjusting the temperature, activating certain functions, etc.) by clicking the mouse 106 and typing on the keyboard 105. This embodiment simulates these operations through an automated script, eliminating manual intervention and improving efficiency.

[0118] The workflow is as follows:

[0119] Step 1: Simulate operator actions. Assume that the operator performs the following operations on the device interface:

[0120] Operation 1: Click the mouse 106 to enter the device parameter setting interface.

[0121] Operation 2: Use the mouse 106 to drag the slider to adjust the temperature setting.

[0122] Operation 3: Enter a new value on keypad 105 to confirm the temperature setting.

[0123] Step 4: Click the "Confirm" button to complete the settings.

[0124] These actions are the actual operator's operations on the equipment, and they need to be simulated through automated control.

[0125] Step 2: Record the action track:

[0126] Use software to record all operator actions, including:

[0127] 1. Mouse 106 trajectory: records each movement of the mouse 106, click position, drag distance, etc.

[0128] 2. Keyboard 105 input: records each key input and the order in which it is pressed.

[0129] For example, when the mouse 106 is clicked to enter the setting interface, the system will record the screen coordinates of the mouse 106 (for example: click (x:300,y:450)) and the trajectory of the mouse 106. When the operator adjusts the temperature slider, the system will record the initial position of the slider, the drag distance, and the final position.

[0130] Step 3: Generate automation script:

[0131] Based on the recorded motion trajectory, the system will automatically generate a script, for example, using an automated scripting tool (such as Python's PyAutoGUI library).

[0132] Step 4: Processing and Correcting Errors:

[0133] Assume that the operator may make some mistakes during operation. For example, when adjusting the temperature, the operator may drag the slider too quickly, resulting in an inaccurate slider position, or enter an incorrect number. These operational errors will be automatically detected and the operator will be allowed to correct them. If the operator's mouse 106 movement is imprecise (for example, the slider is dragged to the wrong position), after the script is generated, the user can manually adjust the slider's exact position. If the entered temperature value is incorrect, the user can directly modify the value in the script to ensure that the parameters are accurate when the script is executed. The final corrected script will be saved and used for future automatic execution.

[0134] Step 5: Execute the automation script:

[0135] The modified script is stored in the device and automatically executed when needed. For example, when the device needs to be reconfigured, the system will automatically execute the script, simulating all the operator's mouse 106 and keyboard 105 movements to accurately adjust the device parameters and ensure the accuracy of the settings.

[0136] The technical effect of this embodiment is that: by receiving input signals from the keyboard 105 and the mouse 106 and recording the operation trajectories of these signals, automatic control and operation script generation are realized; by accurately capturing the keyboard 105 key operation and every movement, click and other behaviors of the mouse 106, the user's manual operation can be converted into a repeatable automated script, which significantly improves the operation efficiency and accuracy; the generated script can automatically execute complex operation processes without human intervention, thereby reducing human errors and improving production efficiency; this embodiment not only supports accurate recording of the operation process, but also has flexible script optimization and custom adjustment functions to ensure execution in different application scenarios.

[0137] As an implementation method, Figure 8 As shown, the monitoring module 102 is further connected to the storage module 107, the storage module 107 is further connected to the control device 101, and the monitoring module 102 is further configured as follows:

[0138] The configuration file in the storage module 107 is updated, and the updated configuration file is written into the control device 101 .

[0139] Among them, the monitoring module 102 first receives a request to update the configuration file through the system interface or external input (such as through the network, USB, or other interface). These configuration files may include device parameters, operating procedures, or other operating settings. The monitoring module 102 communicates with the storage module 107 and reads the current configuration file in the storage module 107. The storage module 107 can be a hard disk, USB flash drive, flash memory, memory card, etc., which is used to store the configuration data of the device. The monitoring module 102 modifies the configuration file in the storage module 107 according to the new configuration requirements or the updated data obtained from the external system. The update may be manually input or automatically downloaded (for example, the latest configuration file is obtained from the management system via the network). These updates usually involve adjustments to the content such as device operating parameters, control algorithms, and production processes. During the update process, the monitoring module 102 will perform verification operations, which may include checksums, file integrity checks, or other verification methods. The updated configuration file is written into the control device 101, and the control device 101 performs real-time operation control according to the configuration file. The writing process is usually carried out through a data bus, memory interface, or a dedicated communication protocol. The configuration file is successfully written into the control device 101 , and the numerical control device 101 begins to adjust the working mode or operating parameters of the device according to the new configuration to ensure that the device operates normally according to the new configuration.

[0140] The technical effect of this embodiment is that, through the connection between monitoring module 102 and storage module 107, dynamic updating and application of configuration files are achieved. By receiving an update request through external input or a system interface, the device can read and modify the configuration file in storage module 107. The updated configuration file is successfully written to control device 101, which then adjusts its operating mode and operating parameters based on the new configuration, thereby improving the automation level, operational efficiency, and stability of the device. This embodiment effectively simplifies the configuration management of production equipment, enhances system flexibility and responsiveness, reduces manual intervention, and improves production efficiency and facilitates equipment maintenance.

[0141] As an implementation method, Figure 8 As shown, the monitoring and control system 200 also includes a remote server 104, which is connected to the monitoring module 102. By receiving real-time data and display signals from the monitoring module 102, the remote server 104 can perform operations such as data analysis, fault warning, equipment control, and report generation, and provide decision support for production management. Through the remote server 104, managers can view the operating status, production progress, working parameters, etc. of the equipment in real time. The remote server 104 provides administrators with a comprehensive user interface, which can monitor the equipment status from anywhere through the network. The remote server 104 can also send control instructions to the monitoring module 102. For example, when an abnormality occurs in CNC production equipment, the administrator can remotely adjust the equipment settings, start and stop the equipment, or initiate certain automated processes, making the management of the production process more flexible and efficient.

[0142] As an example, Figure 10 As shown, the first display signal of the control device 101 is connected to the first display device 111 through the monitoring module 102, the second display signal of the control device 101 is connected to the control chip 110 in the monitoring module 102, the third signal of the control device 101 is connected to the first mouse 112 and the first keyboard 113 respectively through the USB HUB in the monitoring module 102, the third signal of the control device 101 is connected to the control chip 110 through the HID in the monitoring module 102, the control chip 110 is also connected to the second mouse 114, the second keyboard 115 and the second display device 116 respectively, the control chip 110 is also connected to the SD card through the SW module, and the control chip 110 is also connected to the remote server 104.

[0143] The working process of this embodiment is as follows:

[0144] 1. Video signal input and output process:

[0145] 1. Video signal input process: The control device 101 outputs video signals through the VGA, DVI or HDMI interface. These signals are transmitted to the monitoring module 102. The monitoring module 102 first performs bypass mirroring processing, that is, one path of the video signal is directly transmitted to the original first display device 111 of the device to ensure that the original display mode of the device is not affected. This process requires processing various resolutions and display ratios to ensure that the original display remains unchanged.

[0146] 2. Video signal conversion process: The other video signal is transmitted to the core processing module control chip 110 inside the monitoring module 102. After being converted by the control chip 110, this signal is converted into USB format video data. During this process, the control chip 110 will further call and process the data to meet subsequent needs.

[0147] 3. Video Decoding and New Video Stream Generation Process: Within the control chip 110, video data is decoded and converted into new standard video streams. These decoded video streams contain the device's current operating status and related information, and can be used for remote monitoring, device management, and other operations. These new video streams are then transmitted to the remote server 104 for real-time viewing and remote operation by administrators.

[0148] 4. Image frame extraction and AI analysis process: Through the video stream, the control chip 110 can also perform image frame extraction operations. The control chip 110 regularly extracts key frames from the video stream to generate images. These images can be further applied to AI analysis, OCR (optical character recognition) data conversion and other technologies. Through AI analysis, the system can identify key information in the image, such as production status, equipment abnormalities, etc., and automatically generate data reports or alarm notifications. OCR technology can extract text information in the image and convert it into editable data for subsequent data analysis and decision support.

[0149] 2. Keyboard and mouse working process:

[0150] 1. Keyboard and Mouse Signal Input Process: Signals from the first keyboard 113 and first mouse 112 connected to the control device 101 enter the monitoring module 102 through the USB interface. At this point, the signals from the first keyboard 113 and first mouse 112 are transmitted to the product's USB port without any processing, preserving the integrity of the original signals.

[0151] 2. Signal Forwarding and Preserving Original Mode: Once the signal enters monitoring module 102, control chip 110 forwards the signals from first keyboard 113 and first mouse 112 back to the device's first keyboard 113 and first mouse 112 interfaces via another USB port. This process is completely transparent, ensuring that the original keyboard and mouse operation mode of control device 101 is not affected. The user's operations on the original device remain unchanged, and the device responds the same as if it were directly connected to the original device.

[0152] 3. System Control and Forwarding Interruption Process: Although signals are forwarded, monitoring module 102 retains certain control authority. When the system deems it necessary (e.g., for remote operation or automated control), it can interrupt signal forwarding. This prevents operation signals from the original device's first keyboard 113 and first mouse 112 from being transmitted back to the device, thus avoiding operational conflicts or unnecessary misoperations caused by external interference. Furthermore, control can be performed using a second mouse 114 and second keyboard 115.

[0153] 4. Simulated Keyboard and Mouse Operations: In addition to forwarding signals, the monitoring module 102 can also simulate the operations of the original control device 101's first keyboard 113 and first mouse 112 under system control. In this case, the system generates corresponding keyboard and mouse signals through the simulator. These simulated signals are sent to the control device 101, executing the relevant operations on the control device 101. For example, operations such as mouse clicks and keyboard input can be simulated to control production equipment. Unlike previous forwarding behavior, when the system performs simulated operations, signal forwarding is temporarily interrupted to avoid conflicts between manual and automated operations.

[0154] 3. U disk working process:

[0155] 1. USB flash drive simulation connection process: The device's USB interface is connected to a designated USB port on monitoring module 102 via a cable. This USB port simulates the functionality of an external USB flash drive, allowing the device to exchange data with external devices. At this point, the device's USB port simulates the internal storage element as a standard USB flash drive, with the read and write capabilities of an external storage device.

[0156] 2. Storage Component Emulation: The internal storage component is configured to emulate the functionality of a USB flash drive, allowing it to read and write data as an external storage device. This storage component, typically a flash memory module, memory card, or other type of memory chip, resides within the device and acts as a "virtual USB flash drive." The device's external USB port recognizes this storage component as a regular USB flash drive, enabling data exchange with other devices.

[0157] 3. Bidirectional read and write control process: This "virtual USB drive" can be used to read and write data with external devices, and can also be used as an internal storage unit for read and write operations. This means that the device's internal storage resources can simultaneously function as external storage, ensuring access to internal data. However, to ensure data integrity and prevent conflicts, these two operations are mutually exclusive. In other words, external data reading and writing and internal storage operations cannot be performed simultaneously. The system uses built-in control circuitry to manage storage usage.

[0158] 4. Control Circuit and Storage Usage Switching Process: The control circuit switches the storage component's usage mode dynamically. When the storage component needs to be used as a USB flash drive to exchange data with an external device, the system switches the storage component's mode to external USB flash drive. At this point, the device allows external devices to read and write to the storage component. When internal storage operations are required, the system switches to internal storage mode to prevent external devices from interfering with internal data reading and writing. This switching process is implemented by the control circuit, ensuring that the two functions do not interfere with each other.

[0159] 5. Remote Data Transfer: Even if the device is located without direct network access, remote data download can still be achieved through the external USB flash drive emulation function. Administrators or technicians can download data to a USB flash drive in advance, then connect it to the device's designated USB port. Using the bidirectional read and write capabilities of the storage element, the data can be transferred to the device, allowing data updates or configurations to be performed even without a network connection.

[0160] 4. Network working process:

[0161] 1. Network infrastructure: The core of the entire system relies on the network infrastructure to ensure effective connectivity between devices and management systems. All devices (including monitoring and control systems, display devices, CNC production equipment, remote servers, etc.) are connected via a local area network (LAN) or the Internet to form an interconnected network environment.

[0162] 2. Centralized Management and Control: Through the network, all device status and operational information can be centrally managed. Through a dedicated management system, managers can access real-time device data, including operating status, operating parameters, and alarm information. This system not only monitors device operation but also enables scheduling and management of multiple devices through a centralized interface, improving operator efficiency and value while reducing tedious on-site operations.

[0163] 3. Remote Operation and Maintenance: Network connectivity enables remote operation and maintenance of equipment. Regardless of the equipment's location, maintenance personnel can access it remotely to diagnose faults, provide repair instructions, or even make necessary adjustments to remote CNC production equipment. Real-time data transmitted over the network allows maintenance personnel to immediately identify equipment anomalies and take appropriate action, significantly improving equipment reliability and maintenance efficiency.

[0164] 4. Remote automated operation and control: Through the network, monitoring and control systems can achieve automated control and remote operation. Equipment operators are no longer restricted by physical location and can remotely control functions such as starting, stopping, and adjusting parameters of CNC production equipment over the network. This provides strong support for distributed production and cross-regional operations management, making it easier to manage multiple production sites or remote equipment.

[0165] 5. Identify the work process:

[0166] 1. AI technology for image interface recognition: First, the device's current image or video stream is captured on the screen. Using AI-based image recognition algorithms, the captured images are analyzed and processed in real time. These images are transmitted to the backend AI processing unit. The AI model, based on pre-trained algorithms, determines whether predefined interfaces or unusual content appear in the image. The key to success lies in the AI model's accuracy and adaptability to device interface styles. By continuously optimizing and training the model, it can effectively identify normal interfaces and unusual situations across different devices.

[0167] 2. Specified screen recognition: AI technology can be used to identify specific areas or interfaces. This means that the system can not only determine whether the entire screen meets expectations, but also focus on certain key interfaces or areas. For example, the system can identify and determine whether screen content that should not be displayed appears (such as error prompts, device failure screens, blank screens, etc.). Once these abnormal screens are detected, the system can send an alarm in real time or automatically perform corresponding operations, such as switching display content, performing remote repairs, etc.

[0168] 3. Combining AI with OCR technology for data recognition: In the picture, especially the data with a specific location on the image, the system uses OCR (optical character recognition) technology to extract the text or numbers in the picture. These data may be equipment status, parameter values, alarm information, etc. The AI model combined with OCR technology can identify dynamic changes in the picture and extract the required data in real time, such as temperature values, operating time, fault codes, etc. These data are converted into structured information and can be used for further data analysis, report generation, or triggering subsequent control operations. In this way, the operation and status information of the equipment can be effectively captured and processed, providing a decision-making basis for subsequent automated control.

[0169] A second embodiment of the present application provides a monitoring and control method for numerical control production equipment, which is applied to a monitoring and control system connected to the numerical control production equipment. The method includes the following steps:

[0170] When the CNC production equipment displays the operating status data through the display device, the screen content of the display device is obtained, the frame image is obtained according to the screen content, and the operating status data of the CNC production equipment is collected according to the frame image to centrally manage the CNC production equipment.

[0171] It should be understood that the size of the sequence number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples given, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0172] The embodiment of the present application also provides an electronic device, which may be a control device. Figure 10 As shown, the electronic device 2 includes: at least one processor 20, a memory 21, and a computer program 22 stored in the memory 21 and capable of running on at least one processor 20. When the processor 20 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented, or when the processor 20 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0173] For example, a computer program may be divided into one or more modules / units, one or more of which are stored in a memory and executed by a processor to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in an electronic device.

[0174] Those skilled in the art will understand that Figure 10 These are merely examples of electronic devices and do not constitute a limitation of the electronic device. The electronic device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0175] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0176] The memory can be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory can also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a SmartMedia Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Furthermore, the memory can also include both the internal storage unit of the electronic device and the external storage device.

[0177] An embodiment of the present application further provides a readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0178] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, a mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0179] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process in the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0180] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0181] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] In the embodiments provided in this application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0184] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A monitoring and control system for numerical control production equipment, characterized in that: The monitoring and control system is connected to the numerical control production equipment, and the monitoring and control system is configured as follows: When the CNC production equipment displays operating status data through a display device, the screen content of the display device is obtained, a frame image is obtained based on the screen content, and the operating status data of the CNC production equipment is collected based on the frame image to centrally manage the CNC production equipment.

2. The monitoring and control system according to claim 1, characterized in that: The numerical control production equipment includes a control device and a first display device, and the monitoring and control system includes a monitoring module and a second display device, wherein the monitoring module is connected to the control device and the first display device respectively; when the numerical control production equipment displays operating status data through the display device, before obtaining the screen content of the display device, the monitoring module is configured to: Receive an original display signal output by the control device, obtain a first display signal and a second display signal that are identical to the original display signal based on the original display signal, send the first display signal to the first display device, and send the second display signal to the second display device.

3. The monitoring and control system according to claim 2, characterized in that: The monitoring module obtains a first display signal and a second display signal that are identical to the original display signal according to the original display signal, and is configured to: The original display signal is copied into a first display signal and a second display signal through a signal distributor; Alternatively, the original display signal is set as the first display signal, and the original signal is copied to obtain the second display signal.

4. The monitoring and control system according to claim 2, characterized in that: When the numerical control production equipment displays operating status data through a display device, the screen content of the display device is obtained, a frame image is obtained according to the screen content, and the operating status data of the numerical control production equipment is collected according to the frame image. The monitoring module is configured to: Decoding the second display signal into a video stream, and displaying the video stream through the second display device; Taking a screenshot of each frame of video displayed by the second display device to obtain a screenshot image; The screenshot image is identified to obtain the operating status data of the CNC production equipment.

5. The monitoring and control system according to claim 4, characterized in that: The monitoring module identifies the screenshot image to obtain the operating status data of the CNC production equipment, and is configured to: Dividing the screenshot image into multiple areas according to coordinate areas; Identify each area and obtain the type of each area; Call the OCR text model or AI model for recognition according to the type of each area; The identified data is returned and stored in the requested key format.

6. The monitoring and control system according to claim 2, wherein: The monitoring and control system centrally manages the CNC production equipment and is configured to: Analyzing the operating status data; When the operating status data meets the preset normal conditions, maintaining the current state; When the operating status data meets a preset abnormal condition, a response operation is performed.

7. The monitoring and control system according to claim 2, wherein: The monitoring module is also connected to a keyboard and a mouse respectively, and is further configured to: receiving a keyboard signal output by the keyboard and / or a mouse signal output by the mouse; Record the motion trajectory of the keyboard signal and / or the mouse signal during execution, and generate an automated operation script.

8. The monitoring and control system according to claim 2, wherein: The monitoring module is further connected to the storage module, and the storage module is further connected to the control device. The monitoring module is further configured to: The configuration file in the storage module is updated, and the updated configuration file is written into the control device.

9. A monitoring and control method for numerical control production equipment, applied to a monitoring and control system, characterized in that: The monitoring and control system is connected to the numerical control production equipment, and the method comprises the following steps: When the CNC production equipment displays operating status data through a display device, the screen content of the display device is obtained, a frame image is obtained based on the screen content, and the operating status data of the CNC production equipment is collected based on the frame image to centrally manage the CNC production equipment.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by the display control device, the control device executes the control method according to claim 9.

Images