Equipment state monitoring method and device, storage medium and monitoring equipment
By identifying the scene and associated targets in the monitoring image, obtaining deviation values and using warning strategies, the problem of low intelligence in existing monitoring devices is solved, and high-precision equipment failure monitoring and alarming is achieved.
Patent Information
- Application Number
- CN202510245980.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-04
- Publication Date
- 2025-07-25
AI Technical Summary
The existing monitoring methods and monitoring equipment are relatively low in intelligence, making it difficult to automatically determine whether the equipment has abnormalities with high accuracy, and lacks a fault warning mechanism.
By obtaining monitoring images, identifying scene information, targets to be monitored and their associated targets, obtaining deviation values of normal and real-time states, and using deviation scores and preset warning strategies for monitoring and alarming.
It improves the intelligence of equipment abnormality or fault status monitoring, and realizes high-precision fault judgment and safety alarm.
Smart Images
Figure CN120375563A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of video surveillance, and particularly relates to a device status monitoring method, device, storage medium, and monitoring device. Background Art
[0002] Currently, in scenarios such as industrial assembly lines, production parks, and equipment machine rooms, a large number of monitoring devices are required to monitor the working status of equipment.
[0003] There are various existing automated monitoring methods applied to monitoring devices in the market. For example, motion monitoring: that is, by monitoring the motion of the equipment in real time to determine whether there is an abnormal motion state or stagnation; temperature and color change monitoring: using an infrared camera to monitor the temperature change of the equipment, or using a standard camera to capture color changes, and abnormal temperature or color changes may indicate equipment failures; defect monitoring: using a high-resolution camera to obtain images of the appearance or part of the equipment, and by comparing with a standard image to determine whether the equipment has abnormalities.
[0004] However, existing real-time equipment monitoring devices can often only obtain images of the target scenario, and it is difficult to automatically and accurately determine whether the target is abnormal in an intelligent and automated manner, and there is also a lack of an equipment failure warning mechanism.
[0005] Therefore, it is necessary to improve the existing methods and devices for monitoring equipment status. Summary of the Invention
[0006] The purpose of the embodiments of the present application is to provide a device status monitoring method, aiming to solve the problems that the existing monitoring methods and monitoring devices have a low degree of intelligence and it is difficult to accurately determine faults.
[0007] The embodiments of the present application are implemented as follows. A device status monitoring method is provided, and the method includes: Obtain a monitoring image, and identify the scene information, the target to be monitored, and the associated target associated with the target to be monitored in the monitoring image; Obtain the normal working state of the target to be monitored and the normal associated state of the associated target; Perform an image comparison between the real-time state of the target to be monitored and the normal working state to obtain a first deviation value; compare the real-time state of the associated target with the normal associated state of the associated target to obtain a second deviation value; Based on the first deviation value and the second deviation value, obtain the deviation score of the target to be monitored in the monitoring screen; Obtain a preset warning policy table, in which several warning policies are preset; based on the correspondence between the deviation score and the warning policy, monitor and alarm the target to be monitored.
[0008] Another object of the embodiments of the present application is to provide a device status monitoring device, which includes: An image recognition unit, configured to obtain a monitoring image, and recognize scene information, a target to be monitored, and an associated target associated with the target to be monitored in the monitoring image; A working state acquisition unit, configured to obtain the normal working state of the target to be monitored and the normal associated state of the associated target; A deviation value acquisition unit, configured to perform an image comparison between the real-time state of the target to be monitored and the normal working state to obtain a first deviation value; compare the real-time state of the associated target with the normal associated state of the associated target to obtain a second deviation value; A deviation score acquisition unit, configured to obtain a deviation score of the target to be monitored in the monitoring screen based on the first deviation value and the second deviation value; A monitoring and alarm unit, configured to obtain a preset warning policy table, in which several warning policies are preset; based on the correspondence between the deviation score and the warning policy, monitor and alarm the target to be monitored.
[0009] Another object of the embodiments of the present application is to provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor is caused to execute the steps of the device status monitoring method described above.
[0010] Another object of the embodiments of the present application is to provide a monitoring device, including a camera, a memory, and a processor; the camera is configured to obtain a monitoring image; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is caused to execute the steps of the device status monitoring method described above.
[0011] An advantage of the device status monitoring method provided by the embodiments of the present application is that the present application improves the intelligence level of the existing device anomaly or fault status monitoring method or device, has a high accuracy in judging faults or anomalies, and can efficiently process the working state of monitoring production equipment and provide safety alarms. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 It is an application environment diagram of the device status monitoring method provided by the embodiments of the present application; Figure 2Flowchart of the device status monitoring method provided by the embodiment of the present application; Figure 3 Block diagram of the structure of the device status monitoring device provided by the embodiment of the present application; Figure 4 Internal structure block diagram of a computer device in one embodiment. Detailed implementation manners
[0013] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0014] It can be understood that the terms "first", "second", etc. used in the present application may be used herein to describe various elements and technical terms, but unless otherwise specified, these elements and technical terms are not limited by these terms. These terms are only used to distinguish one unit or module from another unit or module.
[0015] Figure 1 Application environment diagram of the device status monitoring method provided by the embodiment of the present application, as Figure 1 shown. In this application environment, it includes a monitoring device 110 and a computer device 120.
[0016] The computer device 120 may be an independent physical server or terminal, or may be a server cluster composed of multiple physical servers, and may be a cloud server or a desktop computer, a personal computer, etc. that provide basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN.
[0017] The monitoring device 110 may be a smart phone, a smart camera, an infrared camera, etc. that includes an imaging device, but is not limited thereto. The monitoring device 110 and the computer device 120 may be connected through a network, and the present application does not limit this here.
[0018] As Figure 2 shown, in one embodiment, a device status monitoring method is proposed. In this embodiment, it is mainly illustrated by applying this method to the above Figure 1 monitoring device 110 and computer device 120. A device status monitoring method may specifically include the following steps: Step S10, obtain a monitoring image, and identify scene information, a target to be monitored, and an associated target associated with the target to be monitored in the monitoring image.
[0019] In this embodiment, image semantic segmentation technology can be used to segment and process images. Among them, the scene information may refer to the scene where the target to be monitored is located. For example, the transformer room where the transformer is located, or outdoor scenes such as mountain forests and residential communities in the city, which are set as the environmental layer of the working background. The target to be monitored may refer to the monitoring subject of the monitoring system. For example, a transformer device in a certain environment. The associated target refers to a fixed or moving target object that can have an associated impact on the target to be monitored, and can include multiple types. For example, objects such as cables, the cooling system around the transformer, dead branches and leaves in the environment, artificial objects, and flammable objects that appear in the detected image. These objects can interact with the target to be monitored, thereby affecting or characterizing the working state of the target to be monitored.
[0020] Step S20: Obtain the normal working state of the target to be monitored and the normal associated state of the associated target.
[0021] In this embodiment, the system presets parameters such as the position, temperature, and shape of the target to be monitored and the associated target in the normal state, which can be used for abnormal comparison.
[0022] Step S30: Compare the real-time state of the target to be monitored with the normal working state to obtain a first deviation value; compare the real-time state of the associated target with the normal associated state of the associated target to obtain a second deviation value.
[0023] Step S40: Based on the first deviation value and the second deviation value, obtain the deviation score of the target to be monitored in the monitoring screen.
[0024] Step S50: Obtain a preset warning policy table, which presets several warning policies; based on the corresponding relationship between the deviation score and the warning policy, monitor and alarm the target to be monitored.
[0025] In this embodiment, different levels of automatic alarm policies can be executed based on different final deviation scores, and the policy library can be preset according to the actual situation. For example, when the deviation score increases due to the direct contact between combustibles and the transformer, a potential danger warning alarm is issued. When alarming, the system can simultaneously output the reason for the danger. The alarm can notify the duty personnel for processing by means of the network or the like.
[0026] In the embodiment of the present application, the above device state detection method can be applied and deployed inside a monitoring device in an industrial production scenario. The monitoring device is used to monitor the state of the target to be monitored in the monitored scene.
[0027] In the embodiments of the present application, by acquiring monitoring images and identifying the scene information, the target to be monitored, and its associated targets therein, the system can comprehensively understand the state of the device and its surrounding environment based on visual data. This provides a multi-dimensional information source for condition monitoring and avoids the one-sidedness that may be caused by traditional methods relying only on a single data source. Moreover, traditional methods often focus on a single state of the target to be monitored, while this method can capture a more comprehensive device operation situation by considering other targets associated with the target to be monitored, enhancing the accuracy and sensitivity of monitoring. The multi-level and multi-dimensional monitoring method provided by this solution can identify the device fault trend at an early stage, improving the accuracy and reliability of the early warning system.
[0028] The researchers of the present application found that in a certain accident, long-term leakage of a distribution box caused local high temperature, but the relevant infrared video alarm system did not trigger an abnormal alarm, and finally a fire was caused due to short-circuit overheating. In the above accident, the reason why the infrared video monitoring system did not trigger an abnormal alarm was that its monitoring method only focused on monitoring the target to be monitored itself and ignored the physical connection between entities. Since the outer shell of the distribution box itself had good heat dissipation, the temperature of its body did not show abnormalities, while the associated components such as bolts and fixing brackets connected to the distribution box generated high temperature due to directly conducting large current. Therefore, the video monitoring system of the present application can not only judge whether the target to be monitored itself has abnormalities, but also automatically analyze the state of other object targets physically associated with the target to be monitored, thus analyzing and predicting more comprehensively and accurately.
[0029] In a preferred embodiment, the method for acquiring the monitoring image, identifying the scene information, the target to be monitored, and the associated targets associated with the target to be monitored in the monitoring image is as follows: Perform denoising, enhancement, and normalization preprocessing on the monitoring image to obtain a preprocessed image; based on an image segmentation model, obtain the background area representing the scene information and the meaningful area representing the object in the preprocessed image, and identify and segment several candidate targets from the meaningful area; based on the monitoring requirements of the user, obtain the target to be monitored from the candidate targets, and classify the other candidate targets except the target to be monitored as potential targets.
[0030] In the embodiment of the present application, the system first acquires a monitoring image, and then processes the image through an image segmentation model and an image recognition model. The above-mentioned segmentation and recognition tasks can be performed based on relatively mature algorithms for image processing, such as convolutional networks, YOLO, Faster R-CNN, etc. Among them, the monitoring requirements of the user refer to the monitoring instructions obtained by the user for specific devices or apparatuses in the environment. The image recognition model can also obtain the basic attribute information of several potential targets obtained by recognition, such as their approximate material type, category, item name, etc., for example, to determine whether it is a combustible material, etc., so as to facilitate the judgment and analysis of subsequent steps.
[0031] In a preferred embodiment, the method further includes: dividing the associated target into a physical associated target, an environmental associated target, and a functional associated target based on the attribute characteristics of the associated target; the physical associated target has a contact relationship with the target to be monitored all the time; the environmental associated target has a non-contact relationship with the target to be monitored; the functional associated target is an object that comes into contact interaction and / or disconnection interaction with the target to be monitored during normal operation, resulting in a position change.
[0032] In the embodiment of the present application, based on the association relationship between target entities, the associated target is refined, classified, managed, and monitored. The physical associated target refers to an entity object that directly contacts the target to be monitored. For example, several cables connected to a certain transformer, the fixing component bolts and frame components used to fix a certain transformer, and there is a physical association between these objects. The environmental associated target refers to different entities that do not contact the target to be monitored under normal conditions in the working environment of the target to be monitored, but may also affect each other. For example, a certain transformer and the branches of surrounding trees, flammable artificial objects, etc. in the environment where the transformer is located. These branches, flammable objects, etc. contacting the transformer or occurring natural conditions, etc. will also affect the working state of the transformer and need to be monitored. The functional association means that objects, components, etc. that interact with the target to be monitored during normal operation. For example, the target to be monitored is a certain high-voltage circuit breaker. This circuit breaker will close or disconnect several lines in the power grid or change the position of some air switches in a mechanical movement manner based on the working conditions. For such devices, since their spatial positions of several objects in the environment will inevitably change during normal operation, causing relative displacement of different components, for such functional associated targets, there is no need to determine the relative position of their components to reduce false alarms caused by changes in the monitoring screen.
[0033] In this embodiment, by further precisely subdividing the components in the environment that may be associated with the target to be monitored, different image analysis objects are obtained, so as to improve the monitoring accuracy and sensitivity of system anomalies, greatly improve the pertinence of the monitoring system, and at the same time reduce the false alarm probability.
[0034] In a preferred embodiment, the method for classifying the associated objects into physical associated objects, environmental associated objects, and functional associated objects based on the attribute characteristics of the associated objects is as follows: Based on an image segmentation model, obtain the target boundary lines of all the alternative objects; Differentially distinguish the target boundary lines of different alternative objects, and superimpose the target boundary lines of the alternative objects on the to-be-detected image to obtain a to-be-labeled image, and display the to-be-labeled image on the user layer interface; Highlight the target boundary line of the to-be-monitored object in the to-be-labeled image, and sequentially highlight the target boundary lines of each associated object; Obtain the attribute characteristic classification results of the physical objects represented by the target boundary lines of each associated object by the user, and obtain the attribute characteristic classification results of all the associated objects.
[0035] In this embodiment, the segmentation model segments the image based on the boundary line, so the associated object can be further refined based on the boundary line data for attribute classification. The advantage of this method is that it does not need to consider the three-dimensional coordinate relationship of the objects in the three-dimensional entity space, which is convenient for simplifying the processing difficulty. Since the image analysis accuracy of the three-dimensional relationship between images is currently limited without using a depth camera, it can be displayed on the user layer display interface in the form of a target boundary line, which is convenient for the user to directly distinguish and classify the attribute characteristics of each target, thereby improving the classification accuracy. The user can classify the attribute characteristics of the physical objects represented by the sequentially highlighted target boundary lines according to the actual situation, that is, label each associated object as one of the physical associated object, environmental associated object, and functional associated object, so as to quickly obtain the attribute characteristics of each associated object.
[0036] In a preferred embodiment, the method for obtaining the first deviation value is as follows: the monitoring image includes a visible light image and an infrared image; based on the visible light image, obtain the position offset and shape offset of the to-be-monitored object; based on the infrared image, obtain the temperature offset of the to-be-monitored object; based on the temperature offset, the position offset, and the shape offset, obtain the first deviation value.
[0037] In the embodiments of the present application, the offset and the change amount refer to the difference between the real-time state of an object and the preset normal state. The position offset can refer to the numerical value of the position offset and change of the target object as a whole in the environment, characterizing the abnormal displacement of the object. The morphological offset can characterize the degree of change in the shape of the object itself. By comparing the real-time monitoring image with the standard image, it can be determined whether the monitored target has abnormal states such as damage or missing appearance. The temperature offset refers to the difference between the real-time temperature of the object and the standard temperature. For example, whether the transformer body under the infrared image has overheating or overcooling of the temperature, and then it can be judged whether its working state is abnormal. By comprehensively analyzing the numerical values of the above temperature offset, the position offset, and the morphological offset, for example, multiplying the above three offsets based on their respective weights, the magnitude of the first deviation value is obtained. At this time, the first deviation value can measure whether the monitored target body has abnormalities.
[0038] In a preferred embodiment, the method for the second deviation value is as follows: The monitoring image includes a visible light image and an infrared image; based on the position change amount and the temperature change amount of the physically associated target, the contact score K 1 is obtained; based on the temperature change amount of the environmentally associated target and the relative distance change amount between the environmentally associated target and the monitored target, the environment score K 2 is obtained; based on the temperature change amount of the functionally associated target, the function score K 3 is obtained; based on the contact score K 1, the environment score K 2, and the function score K 3, the second deviation value is obtained.
[0039] In the embodiments of the present application, in order to further reduce the influence of irrelevant factors on the alarm system, and further conduct a detailed analysis and judgment on the target to be pre-warned for different target types, so as to indirectly judge whether the monitored target body has or may have abnormalities and exclude the influence of irrelevant factors. It can be understood that both the first deviation value and the second deviation value can be displayed on the user interface in real time, and independent warning thresholds and warning strategies can be set for the user to refer to and process.
[0040] In the embodiments of the present application, for the physically associated target, since it is in direct contact with the monitored target, it is necessary to monitor whether its position has abnormal changes and whether the temperature change is abnormal. For example, when the above factors are all normal, the contact score K1 can be set to 1, and in case of an abnormal situation, the value is a number greater than one based on the degree of abnormality. For environmentally related targets, since they are often movable objects such as plant branches, etc., it is necessary to simultaneously monitor the relative distance between them and the target to be monitored and their body temperature. For example, affected by weather factors such as strong winds, billboards, canvases, etc. near the transformer may move onto the transformer, the relative distance becomes 0, and there is a potential risk of short - circuit fire. At this time, due to the abnormal change in the position of the related target, the environmental score K 2 value increases significantly and exceeds the threshold, and the system issues an alarm. For functionally related targets, since their positions change frequently during normal operation, it is only necessary to monitor their temperatures and there is no need to pay attention to their position changes to reduce false alarms.
[0041] In a preferred embodiment, the contact score K 1, environmental score K 2 and functional score K 3 all have a value range of [1, +∞); the second deviation value W satisfies: W = K 1 * K 2 * K 3.
[0042] In the embodiment of the present application, a specific definition form of the second deviation value is given. That is, when the monitored environment is normal, the contact score K 1, environmental score K 2 and functional score K 3 can all take the value of 1, and the value increases as the abnormal situation becomes more serious. In order to simultaneously reflect the influence of all risk factors in the environment, their relationship of multiplying values is adopted, which is more direct and effective.
[0043] In a preferred embodiment, based on the first deviation value and the second deviation value, the method for obtaining the deviation score of the target to be monitored in the monitoring screen is: Obtain the integral of the first deviation value with respect to time at time t: ; where, P targ represents the integral of the first deviation value with respect to time; T 1 and T 2 respectively represent the initial moment and the end moment of the monitoring window; Δ P targ ( t ) represents the deviation degree of the target to be monitored at time t; Obtain the integral of the second deviation value with respect to time at time t: ; Wherein, P asso represents the integral of the second deviation value with respect to time; Δ P asso ( t ) represents the deviation degree of the associated target at time t; Obtain the abnormal score: ; Wherein, P ( t ) represents the abnormal score at time t, μ 1 represents the influence weight of the target to be monitored, μ 2 represents the influence weight of the associated target..
[0044] In the embodiments of the present application, by performing time integration on the deviation degrees of the target to be monitored and the associated target, a deviation score including a time dimension can be obtained. This method can capture the long-term change trend of the device state and provide a more accurate device health assessment. The weights can be determined based on the actual situation.
[0045] At the same time, the deviation score after time integration can consider the overall state of the device over a period of time, avoiding the misleading of the instantaneous deviation degree, so that while the system provides early warnings, it reduces the possibility of false alarms caused by occasional data peaks. For example, several birds flying across the screen of an outdoor monitoring system for power equipment will not cause a short-term abnormal peak in the data of the monitoring system and trigger a high-level abnormal alarm. For the duration of the monitoring window, that is, T 2 - T The value of 1 can be determined according to the actual situation.
[0046] The deviation score obtained in this embodiment provides more dynamic and intelligent support for subsequent warning strategies, thereby helping to achieve early warning and maintenance of the device.
[0047] As Figure 3 shown, in one embodiment, a device state monitoring device is provided. The device state monitoring device can be integrated into the above-mentioned computer device 120, and specifically can include an image recognition unit 510, a working state acquisition unit 520, a deviation value acquisition unit 530, a deviation score acquisition unit 540, and a monitoring and warning unit 550.
[0048] Among them, the image recognition unit 510 is used to obtain a monitoring image and identify the scene information, the target to be monitored, and the associated target associated with the target to be monitored in the monitoring image; The working state acquisition unit 520 is used to obtain the normal working state of the target to be monitored and the normal associated state of the associated target; A deviation value acquisition unit 530 is configured to perform an image comparison between the real-time state of the target to be monitored and the normal operating state to obtain a first deviation value; and compare the real-time state of the associated target with the normal associated state of the associated target to obtain a second deviation value. A deviation score acquisition unit 540 is configured to obtain a deviation score of the target to be monitored in the monitoring screen based on the first deviation value and the second deviation value. A monitoring and warning unit 550 is configured to obtain a preset warning policy table, where several warning policies are preset in the warning policy table; and perform monitoring and warning on the target to be monitored based on the corresponding relationship between the deviation score and the warning policy.
[0049] In the embodiments of the present application, for the explanation and description of the above device status monitoring device, reference can be made to the explanation and description of the corresponding method above. For the description of the above device status monitoring method, please refer to the above, and details are not described herein again.
[0050] In the embodiments of the present application, by acquiring monitoring images and identifying the scene information, the target to be monitored, and its associated target therein, the system can comprehensively understand the status of the device and its surrounding environment based on visual data. This provides a multi-dimensional information source for status monitoring and avoids the one-sidedness that may be caused by traditional methods relying only on a single data source. Moreover, traditional methods often focus on a single state of the target to be monitored, while this method can capture a more comprehensive device operation situation by considering other targets associated with the target to be monitored, enhancing the accuracy and sensitivity of monitoring. The multi-level and multi-dimensional monitoring method provided by this solution can identify the device failure trend at an early stage and improve the accuracy and reliability of the early warning system.
[0051] Figure 4 The internal structure diagram of a computer device in an embodiment is shown. The computer device may specifically be Figure 1 the computer device 120 in Figure 4 As shown, the computer device includes a processor, a memory, a network interface, an input device, and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the device status monitoring method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the device status monitoring method. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.
[0052] Those skilled in the art can understand that Figure 4 the structure shown in Figure 4 is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.
[0053] In one embodiment, the device status monitoring device provided by the present application can be implemented in the form of a computer program, and the computer program can run on a computer device as shown in Figure 4 . Each program module constituting the device status monitoring device can be stored in the memory of the computer device. For example, Figure 4 the image recognition unit 510, the working status acquisition unit 520, the deviation value acquisition unit 530, the deviation score acquisition unit 540, and the monitoring and warning unit 550 shown in Figure 3 . The computer program constituted by each program module enables the processor to execute the steps in the device status monitoring method of each embodiment of the present application described in this specification. Figure 3 For example, the computer device shown in can execute step S10 through the image recognition unit 510 in the device status monitoring device shown in Figure 4 . The computer device can execute step S20 through the working status acquisition unit 520. And so on.
[0054] For example, Figure 4 the computer device shown in Figure 4 can execute step S10 through the image recognition unit 510 in the device status monitoring device shown in Figure 4 . The computer device can execute step S20 through the working status acquisition unit 520. And so on. Figure 3 the computer device can execute step S20 through the working status acquisition unit 520. And so on.
[0055] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor is enabled to execute the steps of the device status monitoring method as described above.
[0056] In the embodiments of the present application, for the description of the above device status monitoring method, please refer to the above, and details are not described herein again.
[0057] In the embodiments of the present application, based on the program running according to the method stored in the storage medium of the embodiments of the present application, by acquiring the monitoring image and identifying the scene information, the target to be monitored and its associated targets therein, the system can comprehensively understand the state of the device and its surrounding environment based on the visual data. This provides a multi-dimensional information source for status monitoring and avoids the one-sidedness that may be caused by traditional methods relying only on a single data source. Moreover, traditional methods often focus on a single state of the target to be monitored, while the present method can capture a more comprehensive device operation situation by considering other targets associated with the target to be monitored, enhancing the accuracy and sensitivity of the monitoring. The multi-level and multi-dimensional monitoring method provided by the present solution can identify the device fault trend at an early stage, improving the accuracy and reliability of the early warning system.
[0058] In one embodiment, a monitoring device is provided, including a camera, a memory, and a processor; the camera is used to acquire monitoring images; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is caused to execute the steps of the device state monitoring method as described above.
[0059] In the embodiments of the present application, the device may be a device including computer software or programs, running on hardware containing processing devices, and when the software system is running, it executes its corresponding method. For the description of the above device state monitoring method, please refer to the above text and will not be elaborated here.
[0060] In the embodiments of the present application, by acquiring monitoring images and identifying the scene information, the target to be monitored, and its associated targets therein, the system can comprehensively understand the state of the device and its surrounding environment based on visual data. This provides a multi-dimensional information source for state monitoring and avoids the one-sidedness that may be caused by traditional methods relying only on a single data source. Moreover, traditional methods often focus on a single state of the target to be monitored, while this method can capture a more comprehensive device operation situation by considering other targets associated with the target to be monitored, enhancing the accuracy and sensitivity of monitoring. The multi-level and multi-dimensional monitoring method provided by this solution can identify the device fault trend at an early stage and improve the accuracy and reliability of the early warning system.
[0061] It should be understood that although the steps in the flowcharts of the embodiments of the present application are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.
[0062] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.
[0063] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.
[0064] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it cannot be understood as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.
Claims
1. A method for monitoring the state of a device, characterized in that, The method includes: Obtain a monitoring image, and identify the scene information, the target to be monitored, and the associated target associated with the target to be monitored in the monitoring image; Obtain the normal working state of the target to be monitored and the normal associated state of the associated target; Perform an image comparison between the real-time state of the target to be monitored and the normal working state to obtain a first deviation value; compare the real-time state of the associated target with the normal associated state of the associated target to obtain a second deviation value; Based on the first deviation value and the second deviation value, obtain the deviation score of the target to be monitored in the monitoring screen; Obtain a preset warning policy table, and several warning policies are preset in the warning policy table; based on the corresponding relationship between the deviation score and the warning policy, monitor and alarm the target to be monitored.
2. The device status monitoring method according to claim 1, wherein The method for obtaining a monitoring image and identifying the scene information, the target to be monitored, and the associated target associated with the target to be monitored in the monitoring image is: Perform denoising, enhancement, and normalization preprocessing on the monitoring image to obtain a preprocessed image; Based on an image segmentation model, obtain the background area representing the scene information and the meaningful area representing the object in the preprocessed image, and identify and segment several alternative targets from the meaningful area; Based on the monitoring requirements of the user, obtain the target to be monitored from the alternative targets, and classify the other alternative targets except the target to be monitored as associated targets.
3. The method for monitoring the state of a device according to claim 2, wherein, The method further includes: Based on the attribute characteristics of the associated target, classify the associated target into a physical associated target, an environmental associated target, and a functional associated target; The physical associated target and the target to be monitored are in a state of constant contact; The environmental associated target and the target to be monitored are in a non-contact relationship; The functional associated target is an object with which the target to be monitored makes contact interaction and / or disconnection interaction during normal operation, resulting in a position change.
4. A method for monitoring the state of a device according to claim 3, characterized in that, The method for classifying the associated target into a physical associated target, an environmental associated target, and a functional associated target based on the attribute characteristics of the associated target is: Based on the image segmentation model, obtain the target boundary lines of all the alternative targets; Differentially distinguish the target boundary lines of different alternative targets, and superimpose the target boundary lines of the alternative targets on the to-be-detected image to obtain a to-be-labeled image, and display the to-be-labeled image on the user layer interface; Highlight the target boundary line of the target to be monitored in the to-be-labeled image, and sequentially highlight the target boundary lines of each associated target; Obtain the attribute characteristic classification result of the physical object represented by the target boundary line of each associated target by the user to obtain the attribute characteristic classification result of all the associated targets.
5. A method for monitoring the state of a device according to claim 1, characterized in that, The method for obtaining the first deviation value is: The monitoring image includes a visible light image and an infrared image; Based on the visible light image, obtain the position offset and the morphological offset of the target to be monitored; Based on the infrared image, obtain the temperature offset of the target to be monitored; Based on the temperature offset, the position offset, and the morphological offset, obtain the first deviation value.
6. The method for monitoring the state of a device according to claim 3, characterized in that, The method for the second deviation value is: The monitoring images include visible light images and infrared images; Obtain a contact fraction based on the amount of change in the position and the amount of change in the temperature of the physical association target K 1; An environmental score is obtained based on the temperature change amount of the environmentally related target and the relative distance change amount between the environmentally related target and the target to be monitored K 2; Obtain a function score based on the temperature change amount of the functional association target K 3;; Based on the contact fraction K 1. Environmental score K 2 and functional score K 3, the second deviation value is obtained, and the second deviation value W satisfies: W = K 1 * K 2 * K 3 7. A method for monitoring the state of a device according to claim 1, characterized in that, The method for obtaining the deviation score of the target to be monitored in the monitoring screen based on the first deviation value and the second deviation value is as follows: Obtain the integral of the first deviation value with respect to time at time t: Among them, P targ represents the integral of the first deviation value with respect to time; T 1 and T 2 respectively represent the initial time and the end time of the monitoring window; Δ P targ ( t ) represents the degree of deviation of the target to be monitored at time t; Obtain the integral of the second deviation value with respect to time at time t: ； Among them, P asso represents the integral of the second deviation value with respect to time; Δ P asso ( t ) represents the degree of deviation of the associated target at time t; Obtain the deviation score: Among them, P ( t ) represents the deviation score at time t, μ 1 represents the influence weight of the target to be monitored, μ 2 represents the influence weight of the associated target.
8. A device status monitoring device, characterized in that, The device status monitoring device includes: An image recognition unit, configured to obtain monitoring images, and recognize scene information, a target to be monitored, and an associated target associated with the target to be monitored in the monitoring images; A working state acquisition unit, configured to acquire the normal working state of the target to be monitored and the normal associated state of the associated target; A deviation value acquisition unit, configured to perform an image comparison between the real-time state of the target to be monitored and the normal working state to obtain a first deviation value; and compare the real-time state of the associated target with the normal associated state of the associated target to obtain a second deviation value; A deviation score acquisition unit, configured to obtain the deviation score of the target to be monitored in the monitoring screen based on the first deviation value and the second deviation value; A monitoring and warning unit, configured to obtain a preset warning policy table, where a plurality of warning policies are preset in the warning policy table; and perform monitoring and warning on the target to be monitored based on the correspondence between the deviation score and the warning policy.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the device status monitoring method according to any one of claims 1 to 7.
10. A monitoring device, characterized in that, It includes a camera, a memory, and a processor; the camera is configured to obtain monitoring images; the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the device status monitoring method according to any one of claims 1 to 7.