A device running state monitoring method and device and electronic device

Abstract

The application provides a device running state monitoring method and device and electronic equipment, the method comprises the following steps: capturing monitoring data corresponding to a target device through a plurality of video sensors, the plurality of video sensors are arranged at a plurality of key monitoring points corresponding to the target device, and the monitoring data comprises real-time images and video streams captured by each video sensor; inputting the monitoring data into a pre-trained image recognition model for classification and recognition to obtain an image recognition result, the image recognition model is trained to be capable of recognizing specific dynamic images and specific static images; and determining a device state corresponding to the target device according to the image recognition result. The application effectively identifies dynamic images or static images corresponding to uninterrupted running devices through an image recognition model, thereby effectively improving the recognition accuracy and speed of the device running state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition, and in particular to a device running state monitoring method and device and electronic device. BACKGROUND

[0002] Existing instrument pattern recognition technology mainly relies on static image processing and pattern matching methods. Although these technologies are effective under certain conditions, they perform poorly in dynamic environments and non-standard pattern recognition scenarios, especially in applications that require real-time monitoring and recognition of dynamically changing patterns, such as unmanned industrial control systems. Existing technologies often cannot meet the 24-hour uninterrupted monitoring requirements and have limitations in recognition accuracy and processing speed. SUMMARY

[0003] Therefore, the present application aims to provide at least a device running state monitoring method and device and electronic device, which effectively recognize dynamic or static patterns corresponding to uninterrupted running devices through image recognition models, thereby effectively improving the recognition accuracy and speed of the device running state.

[0004] The present application mainly includes the following aspects:

[0005] In a first aspect, the present application provides a device running state monitoring method, which includes: capturing monitoring data corresponding to a target device through a plurality of video sensors, the plurality of video sensors being arranged at a plurality of key monitoring points around the target device, the monitoring data including real-time images and video streams captured by each video sensor; inputting the monitoring data into a pre-trained pattern recognition model for classification and recognition to obtain a pattern recognition result, the pattern recognition model being trained to recognize specific dynamic patterns and specific static patterns; and determining a device state corresponding to the target device according to the pattern recognition result.

[0006] In a possible implementation, the method further includes: before inputting the monitoring data into the pre-trained pattern recognition model, performing image optimization on the monitoring data to obtain optimized monitoring data, the image optimization including at least one of the following: image noise reduction, brightness adjustment; verifying the quality definition of the optimized monitoring data to determine whether the quality definition of the optimized monitoring data meets a preset definition standard; if the quality definition of the optimized monitoring data meets the preset definition standard, inputting the optimized monitoring data into the pre-trained pattern recognition model for classification and recognition; and if the quality definition of the optimized monitoring data does not meet the preset definition standard, returning to perform image optimization on the monitoring data again.

[0007] In a possible implementation, the method further includes: before inputting the video stream captured by each video sensor into the pre-trained graphic recognition model, parsing the video stream by a video stream parser to determine a parsing result; if the parsing result indicates that the parsing is successful, setting a parsing failure flag to 0 and inputting the parsed video stream into the pre-trained graphic recognition model for classification and recognition; if the parsing result indicates that the parsing fails, setting the parsing failure flag to +1, performing data log collection rollback, and re-inputting the video stream obtained after the data log collection rollback into the video stream parser for parsing.

[0008] In a possible implementation, the graphic recognition result includes at least one target static graphic and / or at least one target dynamic graphic, and before determining the running state corresponding to the target device according to the graphic recognition result, the method further includes: for each target static graphic and / or each target dynamic graphic, performing the following processing: determining whether the target static graphic and / or the target dynamic graphic meets a detection requirement; if the target static graphic and / or the target dynamic graphic meets the detection requirement, determining the running state corresponding to the target device according to the graphic recognition result; and if the target static graphic and / or the target dynamic graphic does not meet the detection requirement, returning to input the monitoring data into the pre-trained graphic recognition model for re-classification and recognition.

[0009] In a possible implementation, the graphic recognition result includes at least one target static graphic and / or at least one target dynamic graphic, and the device state includes a plurality of functional component states, and determining the device state corresponding to the target device according to the graphic recognition result includes: obtaining a graphic decision response data table corresponding to the target device, the graphic decision response data table recording a plurality of static graphics corresponding to the target device, a functional component state reflected by each static graphic, a plurality of dynamic graphics, and a functional component state reflected by each dynamic graphic; and for each target static graphic and / or each target dynamic graphic, determining a target functional component state corresponding to the target static graphic and / or the target dynamic graphic according to the graphic decision response data table.

[0010] In a possible implementation, the functional component state includes normal operation, pre-warning, and failure, and the method further includes: for each target dynamic graphic, performing the following processing: if a target functional component state corresponding to the target dynamic graphic is normal operation, no subsequent operation is performed; if the target functional component state corresponding to the target dynamic graphic is pre-warning, determining a target preset pre-warning level corresponding to the target dynamic graphic, and if the target preset pre-warning level reaches a pre-warning danger level, generating corresponding pre-warning information and sending the pre-warning information to a mailbox of a person in charge through an email; and if the target functional component state corresponding to the target dynamic graphic is failure, generating corresponding failure information and sending the failure information to the mailbox of the person in charge through the email.

[0011] In a second aspect, the embodiments of the present application further provide a device running state monitoring apparatus, the apparatus comprising: a monitoring module, configured to capture monitoring data corresponding to a target device through a plurality of video sensors arranged at a plurality of key monitoring points around the target device, the monitoring data comprising real-time images and video streams captured by each video sensor; a pattern recognition module, configured to input the monitoring data into a pre-trained pattern recognition model to perform classification and recognition, to obtain a pattern recognition result, the pattern recognition model being trained to be capable of recognizing specific dynamic patterns and specific static patterns; and a device state determination module, configured to determine a device state corresponding to the target device according to the pattern recognition result.

[0012] In a possible implementation, the apparatus further comprises: an optimization module, configured to perform image optimization on the monitoring data before inputting the monitoring data into the pre-trained pattern recognition model, to obtain optimized monitoring data, the image optimization comprising at least one of the following: image noise reduction, brightness adjustment; a definition verification module, configured to verify the definition of the optimized monitoring data, to determine whether the definition of the optimized monitoring data meets a preset definition standard; an optimization success processing module, configured to input the optimized monitoring data into the pre-trained pattern recognition model to perform classification and recognition, if the definition of the optimized monitoring data meets the preset definition standard; and an optimization failure processing module, configured to return to perform image optimization on the monitoring data again, if the definition of the optimized monitoring data does not meet the preset definition standard.

[0013] In a third aspect, the embodiments of the present application further provide an electronic device, comprising: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the device running state monitoring method in the first aspect or any possible implementation of the first aspect.

[0014] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, the computer readable storage medium storing a computer program, when the computer program is executed by the processor, the steps of the device running state monitoring method in the first aspect or any possible implementation of the first aspect are performed.

[0015] The method, device and electronic equipment for monitoring the running state of equipment provided by the embodiment of the application, the method comprises: capturing monitoring data corresponding to a target equipment through a plurality of video sensors, the plurality of video sensors are arranged at a plurality of key monitoring points corresponding to the target equipment, and the monitoring data comprises real-time images and video streams captured by each video sensor; inputting the monitoring data into a pre-trained graphic recognition model for classification and recognition to obtain a graphic recognition result, the graphic recognition model is trained to be capable of recognizing specific dynamic graphics and specific static graphics; and determining a device state corresponding to the target equipment according to the graphic recognition result. The embodiment of the application effectively recognizes dynamic graphics or static graphics in instrument images through an image recognition model, and effectively improves the recognition accuracy and speed of instrument graphics.

[0016] The application has the advantages that:

[0017] 1. Improve monitoring efficiency: by automatically recognizing and tracking dynamic graphics corresponding to the target equipment, the demand for manual monitoring is greatly reduced, and truly all-weather monitoring is realized.

[0018] 2. Reduce response time: automatically respond at the first time when detecting that the running state of the target equipment has a problem, significantly reducing the time for problem diagnosis and processing.

[0019] 3. Improve production safety: through real-time monitoring and rapid response, the safety risks and production accidents caused by monitoring blind spots are significantly reduced, and the monitoring blind spots mainly refer to the phenomenon that personnel are occasionally absent during manual monitoring, while the machine can monitor for 24 hours.

[0020] In order to make the above-mentioned purposes, characteristics and advantages of the application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the application, and therefore should not be regarded as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0022] Figure 1 A flowchart of a method for monitoring the running state of equipment provided by the embodiment of the application is shown;

[0023] Figure 2 A functional module diagram of a device for monitoring the running state of equipment provided by the embodiment of the application is shown;

[0024] Figure 3A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps that have no logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application.

[0026] In addition, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0027] The existing instrument pattern recognition technology mainly relies on static image processing and pattern matching methods. Although these technologies are effective under certain conditions, they perform poorly in dynamic environments and non-standardized pattern recognition scenarios. Especially in applications that require real-time monitoring and recognition of dynamically changing patterns, such as unmanned industrial control systems, existing technologies often cannot meet the demand for 24-hour uninterrupted monitoring, and there are limitations in recognition accuracy and processing speed.

[0028] Based on this, the embodiments of the present application provide a device running state monitoring method, device and electronic device, which effectively recognize the dynamic or static patterns corresponding to the uninterrupted running device through an image recognition model, effectively improve the recognition accuracy and recognition speed of the device running state, and specifically as follows:

[0029] Please refer to Figure 1 , Figure 1 A flowchart of a device running state monitoring method provided by an embodiment of the present application is shown. As Figure 1 shown, the method provided by the embodiments of the present application is applied to a central monitoring system in a workshop, and includes the following steps:

[0030] S100, capturing monitoring data corresponding to the target device through a plurality of video sensors.

[0031] The plurality of video sensors are arranged at a plurality of key monitoring points corresponding to the target device, and the monitoring data includes real-time images and video streams captured by each video sensor.

[0032] S200, inputting the monitoring data into a pre-trained graphic recognition model for classification and recognition to obtain a graphic recognition result.

[0033] The graphic recognition model is trained to be capable of recognizing specific dynamic graphics and specific static graphics.

[0034] S300, determining a device state corresponding to the target device according to the graphic recognition result.

[0035] In a specific implementation, in steps S100-S300, the method provided by the application can be applied to a workshop. Specifically, the workshop includes a plurality of industrial devices operated by no one. Taking a target device in the plurality of industrial devices as an example, a plurality of video sensors are arranged at a plurality of key monitoring points corresponding to the target device. The plurality of key monitoring points correspond to the plurality of video sensors one by one. The video sensor can be a high-resolution camera. The plurality of key monitoring points are located around the target device to ensure no dead angle coverage of the target device. The video sensors of different key monitoring points are used to focus on different functional components on the target device, such as device indicator lights and device displays.

[0036] The video sensor is connected to a central monitoring system. The central monitoring system runs a graphic recognition model YOLOv5 based on YOLOv5. The pre-trained graphic recognition model YOLOv5 is optimized for dynamic graphic recognition and indefinite point real-time monitoring. The plurality of video sensors arranged at the plurality of key monitoring points corresponding to the target device capture real-time images and video streams of the target device in real time and transmit them to the graphic recognition model YOLOv5 for graphic recognition. Based on the recognition result, the device state corresponding to the target device is further determined, which effectively improves the recognition accuracy and speed of the device state corresponding to the device under all-weather monitoring.

[0037] In a preferred embodiment, after the monitoring data corresponding to the target device is obtained, the method further includes:

[0038] The monitoring data is image-optimized to obtain optimized monitoring data. The image optimization includes, but is not limited to, at least one of the following: image noise reduction, brightness adjustment, verifying the quality definition of the optimized monitoring data, determining whether the quality definition of the optimized monitoring data meets a preset definition standard, if the quality definition of the optimized monitoring data meets the preset definition standard, inputting the optimized monitoring data into a pre-trained pattern recognition model for classification and recognition, and if the quality definition of the optimized monitoring data does not meet the preset definition standard, returning to re-optimize the monitoring data.

[0039] In this application, the collected monitoring data is image-optimized through noise reduction, brightness adjustment and other operations to improve the quality of the monitoring data to clear and gear-free, thereby improving the image recognition quality and the accuracy of the subsequent pattern recognition model output pattern recognition result.

[0040] In another preferred embodiment, before the optimized monitoring data is input into the pre-trained pattern recognition model for classification and recognition, the method further comprises:

[0041] The video stream is parsed by a video stream parser to determine a parsing result. If the parsing result indicates that the parsing is successful, the parsing failure flag is set to 0, and the parsed video stream is input into the pre-trained pattern recognition model for classification and recognition. If the parsing result indicates that the parsing fails, the parsing failure flag is set to +1, data log collection rollback is performed, and the video stream obtained after the data log collection rollback is re-input into the video stream parser for parsing.

[0042] Before the optimized monitoring data is input into the pre-trained pattern recognition model for classification and recognition, the video stream in the monitoring data is also parsed by a video stream parser. The video input source of the video stream parser is continuously input by a video sensor. If the video stream input into the video stream parser is interrupted, data log rollback occurs, and then the parsing failure flag is set to +1. The initial value of the parsing failure flag is 0. The video stream obtained after the data log collection rollback is re-input into the video stream parser until the video parsing is successful. Before the video parsing is successful, if the parsing failure flag reaches a preset parsing quantity threshold (for example, 5), it is determined that timeout detection occurs. The timeout detection time can be freely set by the customer (in ms). If the timeout detection is triggered, a detection result report corresponding to the timeout detection is generated and output (for example, a detection result report xlsx records that the video stream parsing passes or fails).

[0043] Further, in step S200, the graphic recognition model YOLOv5 performs rapid recognition and classification of dynamic and static graphics in the image through a deep learning algorithm. The pre-trained graphic recognition model YOLOv5 of the present application can recognize static objects and track and recognize dynamic graphics in a video stream, such as rotating instrument pointers and flashing warning signals.

[0044] In a preferred embodiment, the graphic recognition result includes at least one target static graphic and / or at least one target dynamic graphic, and before step S300 is performed, the method further includes:

[0045] For each target static graphic and / or each target dynamic graphic, the following processing is performed: determining whether the target static graphic and / or the target dynamic graphic meets the detection requirement, if the target static graphic and / or the target dynamic graphic meets the detection requirement, determining the running state of the target device according to the graphic recognition result, and if the target static graphic and / or the target dynamic graphic does not meet the detection requirement, returning to input the monitoring data into the pre-trained graphic recognition model for re-classification and recognition. Specifically, the detection requirement is mainly selected freely by the client according to its own needs, for example, whether a specified dynamic graphic or static graphic (such as an icon, a flower screen, a black screen, a blue screen, etc.) can appear in the monitoring range as a judgment condition for whether the target static graphic and / or the target dynamic graphic meets the detection requirement.

[0046] In a preferred embodiment, the device state includes a plurality of functional component states, and step S300 includes:

[0047] A graphic decision response data table corresponding to the target device is obtained, which records a plurality of static graphics corresponding to the target device, a functional component state reflected by each static graphic, a plurality of dynamic graphics, and a functional component state reflected by each dynamic graphic. For each target static graphic and / or each target dynamic graphic, the target functional component state corresponding to the target static graphic and / or the target dynamic graphic is determined according to the graphic decision response data table.

[0048] In a specific embodiment, the graphic recognition model YOLOv5 is trained to recognize specific dynamic graphics, such as color changes of indicator lights, changes of numbers on digital displays, etc., and the graphic recognition model YOLOv5 is trained to recognize specific static graphics, such as static icons on instrument panels, etc. In this application, different video sensors of the target device monitor different functional components of the target device. For example, taking dynamic graphics as an example, a certain video sensor is used to monitor the fault status indicator light corresponding to the target device, and the video sensor inputs the target video stream corresponding to the fault status indicator light collected after processing into the graphic recognition model YOLOv5. The graphic recognition model YOLOv5 classifies and recognizes the target video stream to determine the target dynamic graphic corresponding to the target video stream. Assuming that the target dynamic graphic is that the indicator light jumps from green to red, the target functional state reflected by the target dynamic graphic is that the device has failed, which can be understood. Different video sensors can ultimately obtain different functional component states.

[0049] In another preferred embodiment, the functional component state includes normal operation, pre-warning, and failure, and the method further includes:

[0050] For each target dynamic graphic, the following processing is performed: if the target functional component state corresponding to the target dynamic graphic is normal operation, no subsequent operation is performed, if the target functional component state corresponding to the target dynamic graphic is pre-warning, the target preset pre-warning level corresponding to the target dynamic graphic is determined, if the target preset pre-warning level reaches the pre-warning danger level, the corresponding pre-warning information is generated, and the pre-warning information is sent to the mailbox of the person in charge through email, if the target functional component state corresponding to the target dynamic graphic is failure, the corresponding failure information is generated, and the failure information is sent to the mailbox of the person in charge through email.

[0051] In this application, the operator can pre-set a corresponding pre-warning level for each dynamic graphic, and the pre-warning level can be divided into 0-10. Assuming that the target dynamic graphic is that the indicator light jumps from green to red, the target functional state reflected by the target dynamic graphic is that the device has failed, and the pre-warning level is 3. Assuming that the pre-warning danger level is 5, therefore, in this case, no corresponding email is sent to the operator.

[0052] In this application, the email content includes but is not limited to: pre-warning / failure type, corresponding video stream, log, and excel record.

[0053] Based on the same application concept, the embodiment of the present application also provides a device running state monitoring device corresponding to the device running state monitoring method provided by the above-mentioned embodiment. Since the principle of solving problems in the device of the embodiment of the present application is similar to the device running state monitoring method of the above-mentioned embodiment of the present application, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0054] Please refer to Figure 2 , Figure 2 The function module diagram of a device running state monitoring device provided by the embodiment of the present application is shown. As shown in Figure 2 , the device includes:

[0055] The monitoring module 400 is configured to capture monitoring data corresponding to the target device through a plurality of video sensors arranged at a plurality of key monitoring points around the target device, wherein the monitoring data includes real-time images and video streams captured by each video sensor.

[0056] The pattern recognition module 410 is configured to input the monitoring data into a pre-trained pattern recognition model for classification and recognition to obtain a pattern recognition result, wherein the pattern recognition model is trained to be capable of recognizing specific dynamic patterns and specific static patterns.

[0057] The device state determination module 420 is configured to determine the device state corresponding to the target device according to the pattern recognition result.

[0058] Preferably, the device further includes:

[0059] The optimization module is configured to perform image optimization on the monitoring data before inputting the monitoring data into the pre-trained pattern recognition model to obtain optimized monitoring data, wherein the image optimization includes at least one of the following: image noise reduction, brightness adjustment.

[0060] The definition verification module is configured to verify the definition of the quality of the optimized monitoring data to determine whether the definition of the quality of the optimized monitoring data meets a preset definition standard.

[0061] The optimization success processing module is configured to input the optimized monitoring data into the pre-trained pattern recognition model for classification and recognition if the definition of the quality of the optimized monitoring data meets the preset definition standard.

[0062] The optimization failure processing module is configured to return to perform image optimization on the monitoring data again if the definition of the quality of the optimized monitoring data does not meet the preset definition standard.

[0063] Based on the same application concept, please refer to Figure 3 , Figure 3A structural schematic diagram of an electronic device provided in an embodiment of the present application is shown. The electronic device 600 comprises a processor 610, a memory 620 and a bus 630, the memory 620 stores machine readable instructions executable by the processor 610, when the electronic device 600 is running, the processor 610 and the memory 620 communicate through the bus 630, and the machine readable instructions are executed by the processor 610 to perform the steps of the device running state monitoring method provided in any of the above embodiments.

[0064] Based on the same application concept, the present embodiment further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the device running state monitoring method provided in the above embodiments.

[0065] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0066] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment.

[0067] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0068] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0069] The above merely describes the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of monitoring the operating state of a device, characterized in that The method comprises: capturing monitoring data corresponding to the target device through a plurality of video sensors arranged at a plurality of key monitoring points around the target device, the monitoring data comprising real-time images and video streams captured by each video sensor, the plurality of video sensors corresponding to a plurality of functional components on the target device, the plurality of functional components at least comprising a device display; inputting the monitoring data into a pre-trained graphic recognition model for classification and recognition to obtain a graphic recognition result, the graphic recognition model being trained to recognize specific dynamic graphics and specific static graphics, the graphic recognition model being a YOLOv5 model pre-trained to recognize specific dynamic graphics and specific static graphics; determining a device state corresponding to the target device according to the graphic recognition result; wherein the graphic recognition model recognizes static graphics and dynamic graphics on the display in the monitoring data corresponding to the device display, the graphics on the display at least comprising data and icons, the graphic recognition result comprising at least one target static graphic and at least one target dynamic graphic on the display, and the device state comprising a plurality of functional component states; the step of determining the device state corresponding to the target device according to the graphic recognition result comprises: for each target static graphic and each target dynamic graphic, determining whether the target static graphic and the target dynamic graphic meet a detection requirement; if the target static graphic and the target dynamic graphic meet the detection requirement, obtaining a graphic decision response data table corresponding to the target device, the graphic decision response data table recording a plurality of static graphics corresponding to the target device, a functional component state reflected by each static graphic, a plurality of dynamic graphics, and a functional component state reflected by each dynamic graphic; for each target static graphic and each target dynamic graphic, determining a target functional component state corresponding to the target static graphic and the target dynamic graphic respectively according to the graphic decision response data table; if the target static graphic and the target dynamic graphic do not meet the detection requirement, returning to input the monitoring data into the pre-trained graphic recognition model for classification and recognition again.

2. The method of claim 1, wherein, The method further comprises: before inputting the monitoring data into the pre-trained graphic recognition model, performing image optimization on the monitoring data to obtain optimized monitoring data, the image optimization comprising at least one of the following: image noise reduction, brightness adjustment; verifying the image quality definition of the optimized monitoring data to determine whether the image quality definition of the optimized monitoring data meets a preset definition standard; if the image quality definition of the optimized monitoring data meets the preset definition standard, inputting the optimized monitoring data into the pre-trained graphic recognition model for classification and recognition; if the image quality definition of the optimized monitoring data does not meet the preset definition standard, returning to perform image optimization on the monitoring data again.

3. The method of claim 1, wherein, The method further comprises: before inputting the video stream captured by each video sensor into the pre-trained graphic recognition model, analyzing the video stream through a video stream analyzer to determine an analysis result; If the analysis result indicates that the analysis is successful, the analysis failure flag is set to 0, and the analyzed video stream is input into a pre-trained graphic recognition model for classification and recognition. If the analysis result indicates that the analysis fails, the analysis failure flag is set to +1, data log collection rollback is performed, and the video stream obtained after the data log collection rollback is re-input into the video stream parser for analysis.

4. The method of claim 1, wherein, The function component state includes normal operation, pre-warning, and failure; The method further includes: For each target dynamic graphic, the following processing is performed: If the target function component state corresponding to the target dynamic graphic is normal operation, no subsequent operation is performed; If the target function component state corresponding to the target dynamic graphic is pre-warning, a target preset pre-warning level corresponding to the target dynamic graphic is determined, if the target preset pre-warning level reaches a pre-warning danger level, corresponding pre-warning information is generated, and the pre-warning information is sent to the mailbox of the person in charge through email; If the target function component state corresponding to the target dynamic graphic is failure, corresponding failure information is generated, and the failure information is sent to the mailbox of the person in charge through email.

5. A device operation state monitoring apparatus characterized by comprising: The device includes: A monitoring module configured to capture monitoring data corresponding to a target device through a plurality of video sensors arranged at a plurality of key monitoring points around the target device, the monitoring data including real-time images and video streams captured by each video sensor, the plurality of video sensors corresponding to a plurality of function components on the target device, and the plurality of function components including at least a device display; A graphic recognition module configured to input the monitoring data into a pre-trained graphic recognition model for classification and recognition to obtain a graphic recognition result, the graphic recognition model being trained to be capable of recognizing specific dynamic graphics and specific static graphics, and the graphic recognition model being a YOLOv5 model pre-trained to recognize specific dynamic graphics and specific static graphics; A device state determination module configured to determine a device state corresponding to the target device according to the graphic recognition result; The graphic recognition model recognizes static graphics and dynamic graphics on the display in the monitoring data corresponding to the device display, the graphics on the display include at least data and icons, the graphic recognition result includes at least one target static graphic and at least one target dynamic graphic, and the device state includes a plurality of function component states; The device state determination module is further configured to: For each target static graphic and each target dynamic graphic, determine whether the target static graphic and the target dynamic graphic meet a detection requirement; If the target static graphic and the target dynamic graphic meet the detection requirement, obtain a graphic decision response data table corresponding to the target device, the graphic decision response data table recording a plurality of static graphics corresponding to the target device, function component states reflected by each static graphic, a plurality of dynamic graphics, and function component states reflected by each dynamic graphic; For each target static graphic and each target dynamic graphic, determine a target function component state corresponding to the target static graphic and the target dynamic graphic, respectively, according to the graphic decision response data table. If the target static image and the target dynamic image do not meet the detection requirement, the monitoring data is returned to a pre-trained image recognition model for re-classification.

6. The apparatus of claim 5, wherein, The device further comprises: An optimization module, configured to perform image optimization on the monitoring data before inputting the monitoring data into the pre-trained image recognition model, to obtain optimized monitoring data, wherein the image optimization comprises at least one of the following: image noise reduction, brightness adjustment; A definition verification module, configured to verify the definition of the optimized monitoring data, to determine whether the definition of the optimized monitoring data meets a preset definition standard; An optimization success processing module, configured to, if the definition of the optimized monitoring data meets the preset definition standard, input the optimized monitoring data into the pre-trained image recognition model for classification. An optimization failure processing module, configured to, if the definition of the optimized monitoring data does not meet the preset definition standard, return to perform image optimization on the monitoring data again.

7. An electronic device, comprising: comprise: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the device running state monitoring method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the device running state monitoring method according to any one of claims 1 to 4.

Images