Equipment exception detection method and device, storage medium and electronic equipment

Abstract

The invention discloses an equipment exception detection method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a region image of a target region and a region reference image of the target region; determining a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity; under the condition that the first similarity is smaller than a first preset threshold value, identifying an image area corresponding to the target equipment from the area image to obtain an equipment image, and identifying an image area corresponding to the target equipment from the area reference image to obtain an equipment reference image; determining a feature similarity between the equipment image and the equipment reference image through a second difference detection model to obtain a second similarity; and determining a detection result of the target equipment according to the second similarity. Through application of the method and the device, the problem of low detection accuracy when anomaly detection is performed on the equipment in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular, to a method, device, storage medium, and electronic device for detecting device anomalies. Background Art

[0002] In the current field of intelligent operation and maintenance, especially in the maintenance and management of data centers and communication machine rooms, device anomaly detection is a key link to ensure system stability and security. With the popularization of high-definition visual monitoring technology, a large amount of video and image data makes it possible to monitor the device status in real time.

[0003] Currently, traditional detection methods mostly rely on simple image processing techniques based on template matching or threshold setting. These methods have poor effects when facing complex and changeable machine room environments (such as light changes, occlusion, a wide variety of device types, etc.). When there are slight changes in the device appearance or the environmental conditions are unstable, template matching is prone to false alarms, while threshold setting is difficult to adapt to the differences in different devices and scenarios, resulting in the problem of low detection accuracy for device anomaly detection.

[0004] In view of the above problems in the related art, no effective solution has been proposed yet. Summary of the Invention

[0005] The main purpose of the present application is to provide a method, device, storage medium, and electronic device for detecting device anomalies, so as to solve the problem of low detection accuracy in the related art when detecting device anomalies.

[0006] To achieve the above object, according to one aspect of the present application, a method for detecting device anomalies is provided. The method includes: obtaining a regional image of a target area and a regional reference image of the target area, where at least one device is set in the target area; determining a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity, where the first difference detection model is trained in a self-supervised manner; in the case where the first similarity is less than a first preset threshold, identifying an image area corresponding to the target device from the regional image to obtain a device image, and identifying an image area corresponding to the target device from the regional reference image to obtain a device reference image; determining a feature similarity between the device image and the device reference image through a second difference detection model to obtain a second similarity, where the second difference detection model is trained in a self-supervised manner; and determining a detection result of the target device according to the second similarity, where the detection result is used to represent whether the target device has an anomaly.

[0007] Further, the method for detecting device anomalies further includes: detecting the image quality of the regional image and the regional reference image through a quality detection model to obtain a quality detection result; and when it is determined that the quality of the regional image and the regional reference image is qualified according to the quality detection result, determining the feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity.

[0008] Further, the method for detecting device anomalies further includes: identifying the regional image and the regional reference image through a target recognition model to obtain the device information corresponding to the regional image and the regional reference image respectively, where the device information at least includes the coordinates of the image regions corresponding to each device; determining any device in the regional image as the target device, and determining the device image of the target device according to the device information corresponding to the regional image; determining each device image region in the regional reference image according to the device information corresponding to the regional reference image, and determining the device reference image of the target device from the device image regions in the regional reference image according to the area intersection ratio between the device image of the target device and each device image region.

[0009] Further, the method for detecting device anomalies further includes: judging whether the number of devices in the regional image and the regional reference image is the same according to the device information corresponding to the regional image and the regional reference image respectively; and when the number of devices is the same, determining any device in the regional image as the target device.

[0010] Further, the method for detecting device anomalies further includes: obtaining a first training sample set, where the training samples in the first training sample set include the first sample regional image of the sample region and the second sample regional image, the second sample regional image is obtained by performing image enhancement on the first sample regional image, and the sample region includes at least one sample device; training an initial difference detection model through the first training sample set to obtain a first difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0011] Further, the method for detecting device anomalies further includes: training an initial difference detection model through the first training sample set to obtain a third difference detection model; and performing post-quantization processing on the third difference detection model to obtain a first difference detection model.

[0012] Further, the method for detecting device anomalies further includes: obtaining a second training sample set, where the training samples in the second training sample set include a first sample device image and a second sample device image of a sample device, and the second sample device image is obtained by performing image enhancement on the first sample device image; training an initial difference detection model with the second training sample set to obtain a second difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0013] Further, the method for detecting device anomalies further includes: the first difference detection model is deployed on a first server, and the second difference detection model is deployed on a second server, where the distance between the first server and the target area is less than the distance between the second server and the target area.

[0014] To achieve the above object, according to another aspect of the present application, a device detection apparatus is provided. The apparatus includes: an acquisition module, configured to acquire a regional image of a target area and a regional reference image of the target area, where at least one device is provided in the target area; a first determination module, configured to determine a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity, where the first difference detection model is trained in a self-supervised manner; a processing module, configured to, when the first similarity is less than a first preset threshold, identify an image area corresponding to a target device from the regional image to obtain a device image, and identify an image area corresponding to the target device from the regional reference image to obtain a device reference image; a second determination module, configured to determine a feature similarity between the device image and the device reference image through a second difference detection model to obtain a second similarity, where the second difference detection model is trained in a self-supervised manner; a third determination module, configured to determine a detection result of the target device according to the second similarity, where the detection result is used to indicate whether the target device has an anomaly.

[0015] Further, the first determination module further includes: a detection sub-module, configured to detect the image quality of the regional image and the regional reference image through a quality detection model to obtain a quality detection result; a first determination sub-module, configured to, when it is determined according to the quality detection result that the image quality of the regional image and the regional reference image is qualified, determine a feature similarity between the regional image and the regional reference image through the first difference detection model to obtain a first similarity.

[0016] Further, the processing module further includes: a first processing sub-module, configured to identify the region image and the region reference image through a target recognition model to obtain the device information corresponding to the region image and the region reference image respectively, where the device information at least includes the coordinates of the image regions corresponding to each device; a second determination sub-module, configured to determine any device in the region image as the target device, and determine the device image of the target device according to the device information corresponding to the region image; a third determination sub-module, configured to determine each device image region in the region reference image according to the device information corresponding to the region reference image, and determine the device reference image of the target device from the device image regions in the region reference image according to the area intersection ratio between the device image of the target device and each device image region.

[0017] Further, the second determination sub-module further includes: a judgment unit, configured to judge whether the number of devices in the region image and the region reference image is the same according to the device information corresponding to the region image and the region reference image respectively; a determination unit, configured to determine any device in the region image as the target device when the number of devices is the same.

[0018] Further, the device detection device further includes: a first acquisition module, configured to acquire a first training sample set, where the training samples in the first training sample set include a first sample region image and a second sample region image of a sample region, the second sample region image is obtained by performing image enhancement on the first sample region image, and the sample region includes at least one sample device; a first training module, configured to train an initial difference detection model through the first training sample set to obtain a first difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0019] Further, the first training module further includes: a training sub-module, configured to train the initial difference detection model through the first training sample set to obtain a third difference detection model; a second processing sub-module, configured to perform post-quantization processing on the third difference detection model to obtain the first difference detection model.

[0020] Further, the device detection device further includes: a second acquisition module, configured to acquire a second training sample set, where the training samples in the second training sample set include a first sample device image and a second sample device image of a sample device, the second sample device image is obtained by performing image enhancement on the first sample device image; a second training module, configured to train the initial difference detection model through the second training sample set to obtain a second difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0021] Further, the first difference detection model is deployed on the first server, and the second difference detection model is deployed on the second server, where the distance between the first server and the target area is less than the distance between the second server and the target area.

[0022] To achieve the above object, according to another aspect of the present application, there is provided a computer-readable storage medium, the computer-readable storage medium including an executable program stored therein, wherein when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the above-mentioned method for detecting device anomalies.

[0023] To achieve the above object, according to another aspect of the present application, there is provided an electronic device, the electronic device including a memory storing an executable program; and a processor for running the program, wherein when the program runs, it executes the above-mentioned method for detecting device anomalies.

[0024] To achieve the above object, according to another aspect of the present application, there is provided a computer program product including computer instructions, and when the computer instructions are executed by a processor, the steps of the above-mentioned method for detecting device anomalies are implemented.

[0025] In the embodiments of the present application, by designing that the first difference detection model and the second difference detection model are trained in a self-supervised manner, the model can learn rich feature representations from a large amount of unlabeled data, thereby improving the generalization ability of the model. By using the first difference detection model and the second difference detection model to determine the feature similarity between images, the accuracy of the determined similarity is improved, so that when determining the detection result based on the similarity, the accuracy of the determined detection result can be effectively improved. In addition, by first using the first detection model to detect the difference in regional images and then detecting the difference in device images when there are differences in the regional images, the device detection efficiency can be effectively improved.

[0026] It can be seen that the method provided by the embodiments of the present application achieves the purpose of using a model trained by self-supervision to detect device anomalies, solves the problem of low detection accuracy in the related art when detecting device anomalies, and further achieves the effect of improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings forming a part of this application are used to provide a further understanding of this application. The illustrative embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0028] Figure 1 is a hardware structure block diagram of a computer terminal provided according to an embodiment of the present application;

[0029] Figure 2It is the flowchart of the method for detecting device anomalies provided by the embodiments of the present application Figure 1 ;

[0030] Figure 3 It is a schematic diagram of post-quantization processing of the model provided by the embodiments of the present application;

[0031] Figure 4 It is a framework diagram of the initial difference detection model provided by the embodiments of the present application;

[0032] Figure 5 It is the flowchart of the method for detecting device anomalies provided by the embodiments of the present application Figure 2 ;

[0033] Figure 6 It is a schematic diagram of the device for detecting device anomalies provided by the embodiments of the present application;

[0034] Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0035] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0037] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Moreover, for the processing of relevant data such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with relevant laws, regulations, and standards, adopt necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, there are interfaces set between this system and relevant users or institutions to provide corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0038] Embodiment 1

[0039] According to an embodiment of the present application, an embodiment of a method for detecting device anomalies is further provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0040] The method embodiment provided by the first embodiment of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for detecting device anomalies is shown. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b,..., 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than those Figure 1 shown, or have a different configuration from that Figure 1 shown.

[0041] It should be noted that one or more of the above-mentioned processors 102 and / or other data processing circuits can generally be referred to as "data processing circuits" herein. The data processing circuit can be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit can be a single independent processing module, or be incorporated in whole or in part into any one of other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is a kind of processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0042] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage devices corresponding to the device anomaly detection method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned device anomaly detection method. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 can further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0043] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0044] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0045] Under the above operating environment, the present application provides a device anomaly detection method as Figure 2 shown. Figure 2 is a flow chart of the device anomaly detection method provided according to the embodiments of the present application Figure 1 .

[0046] Step S201: Obtain the regional image of the target area and the regional reference image of the target area, where at least one device is set in the target area.

[0047] Optionally, devices such as electronic devices, application systems, servers, etc. can be used as the execution subject of this application. In this embodiment, the target processing system is used as the execution subject to execute the above-mentioned device anomaly detection method.

[0048] Optionally, the target area can refer to areas such as computer rooms and data centers, and the devices included in the target area can be server racks, air-conditioning equipment, power equipment, etc.

[0049] Optionally, the image of the target area collected at the first moment can be determined as the regional image. The first moment can be the current moment, and the image of the target area collected at the second moment can be determined as the regional reference area image. The second moment can be a historical moment.

[0050] Optionally, the target processing system can obtain the current image (i.e., the regional image) and the image in the historical normal state (i.e., the regional reference image) of the target area regularly or in real time through a camera or other image acquisition devices. For example, the target processing system can pull the real-time stream video data of each camera in the computer room (i.e., the target area), and save the snapshot image of the target area after decoding.

[0051] Step S202: Determine the feature similarity between the regional image and the regional reference image through the first difference detection model to obtain the first similarity, where the first difference detection model is trained in a self-supervised manner.

[0052] Optionally, the first difference detection model is a model trained based on self-supervised learning, which is used to preliminarily judge the overall difference between the regional image and the regional reference image. The target processing system can input the regional image and the regional reference image into the first difference detection model respectively, and perform feature extraction on the regional image and the regional reference image through the first difference detection model to obtain the image features corresponding to the regional image and the regional reference image respectively.

[0053] After obtaining the image features, the target processing system can calculate the feature similarity between the image features corresponding to the regional image and the regional reference image respectively, so as to obtain the first similarity. For example, the cosine similarity calculation method is used to obtain the feature similarity between the two.

[0054] It should be noted that the first difference detection model is trained in a self-supervised manner. The model can learn rich feature representations from a large amount of unlabeled data, thereby improving the generalization ability of the model. This avoids the problem that due to the difficulty in exhausting the differences in device images, it is difficult for a supervised training model to recognize the differences not covered in the training set, which may affect the accuracy of the detection results.

[0055] Step S203: When the first similarity is less than the first preset threshold, identify the image region corresponding to the target device from the regional image to obtain the device image, and identify the image region corresponding to the target device from the regional reference image to obtain the device reference image.

[0056] Optionally, if the first similarity is less than the first preset threshold, it is determined that there may be significant differences between the two images, and the device in the target region may have changed. Further attention needs to be paid to the situation of the device itself. Therefore, in this case, the target processing system can extract the local image containing the target device (i.e., the device image) from the current regional image, and extract the local image of the same device (i.e., the device reference image) from the historical regional reference image, so as to prepare for the next detailed device status detection.

[0057] Optionally, the target device can be any device in the target region.

[0058] Optionally, if the first similarity is greater than or equal to the first preset similarity, it is determined that the device in the target region has not changed. In this case, a detection result indicating that there is no abnormality in the device in the target region is directly generated.

[0059] Step S204: Determine the feature similarity between the device image and the device reference image through the second difference detection model to obtain the second similarity, where the second difference detection model is trained in a self-supervised manner.

[0060] Optionally, the model parameters of the first difference detection model can be the same as or different from the model parameters of the second difference detection model. The second difference detection model is also trained based on self-supervised learning, but it mainly focuses on the detailed differences at the device level.

[0061] Optionally, the target processing system can input the device image and the device reference image into the second difference detection model respectively, and the second difference detection model extracts the image features corresponding to the device image and the device reference image respectively.

[0062] After obtaining the image features, the target processing system can calculate the feature similarity between the device image and the device reference image respectively, so as to obtain the second similarity. For example, the feature similarity between the two is obtained by using the cosine similarity calculation method.

[0063] Step S205, determine the detection result of the target device according to the second similarity, where the detection result is used to characterize whether the target device has an abnormality.

[0064] Optionally, if the second similarity is less than the second preset threshold, it is determined that the device state of the target device has changed before and after, for example, the shape of the device has changed, the switches and buttons on the device have changed, objects have been added or removed on the device, etc. In this case, the target device may have a fault or an abnormal situation, and the target processing system can determine that the detection result characterizes that the target device has an abnormality. On the contrary, if the second similarity is greater than or equal to the second preset threshold, it is determined that the device state of the target device has not changed before and after. In this case, the target processing system can determine that the detection result characterizes that the target device has no abnormality. Among them, the first preset threshold and the second preset threshold may be the same or different.

[0065] Optionally, when the second similarity is less than the second preset threshold, the target processing system can also use the differential algorithm for the device image and the device reference image to visualize the pixel points of the different parts of the two images, and then mark the pixel points of this part in the two images. For example, the different pixel points are surrounded by a marking frame, and the marked images are used as the detection result together.

[0066] Optionally, after obtaining the detection result, the detection result can be stored in a preset storage area and sent to the staff for the staff to process in time.

[0067] In the embodiments of the present application, the first difference detection model and the second difference detection model are designed and trained in a self-supervised manner. The model can learn rich feature representations from a large amount of unlabeled data, thereby improving the generalization ability of the model. By using the first difference detection model and the second difference detection model to determine the feature similarity between images, the accuracy of the determined similarity is improved. Therefore, when determining the detection result based on the similarity, the accuracy of the determined detection result can be effectively improved. In addition, by first using the first detection model to detect the difference in the regional image and then detecting the difference in the device image when there is a difference in the regional image, the device detection efficiency can be effectively improved.

[0068] It can be seen that the method provided by the embodiments of the present application achieves the purpose of using the model obtained by self-supervised training for device anomaly detection, solves the problem of low detection accuracy in the related art when performing anomaly detection on devices, and further achieves the effect of improving detection accuracy.

[0069] Optionally, in the device anomaly detection method provided by the embodiments of the present application, the feature similarity between the regional image and the regional reference image is determined by the first difference detection model to obtain the first similarity, including: detecting the image quality of the regional image and the regional reference image through the quality detection model to obtain the quality detection result; when it is determined that the quality of the regional image and the regional reference image is qualified according to the quality detection result, determining the feature similarity between the regional image and the regional reference image through the first difference detection model to obtain the first similarity.

[0070] Optionally, the quality detection model is used to evaluate the image quality of the regional image and the regional reference image. The image quality may refer to the image clarity. The target processing system may input the regional image and the regional reference image into the quality detection model to obtain the quality detection result, where the quality detection result includes a first value and a second value. The first value refers to the probability that the quality of the regional image is qualified, and the second value refers to the probability that the quality of the regional reference image is qualified.

[0071] If both the first value and the second value are greater than or equal to the preset probability value (e.g., 50%), it is determined that the quality of the regional image and the regional reference image is qualified. In this case, the feature similarity between the regional image and the regional reference image is determined by the first difference detection model to obtain the first similarity.

[0072] If there is a value less than the preset probability value (e.g., 50%) in the first value and the second value, it is determined that the quality of the regional image and the regional reference image is unqualified. In this case, determining the first similarity is prohibited.

[0073] Optionally, the target processing system can train the quality detection model in the following manner: constructing a third training sample set, where the training samples in the third training sample set are the sample regional images of the sample regions, and the true labels of the training samples represent whether the quality of the sample regional images is qualified; training the initial quality detection model through the third training sample set to obtain the trained initial quality detection model. The trained initial quality detection model is determined as the quality detection model, or the trained initial quality detection model is subjected to post-training quantization (PTQ) processing to obtain the quality detection model.

[0074] Optionally, Figure 3 is a schematic diagram of the model post-training quantization process provided by the embodiments of the present application, such as Figure 3As shown, during the post - quantization process, the target processing system can set a quantization threshold for each processing layer in the trained initial quality detection model. This threshold determines the accuracy and range of the quantized model parameters. Different thresholds will affect the performance of the model, such as accuracy and recall rate. For example, a threshold is set to act on the feature map of the forward output to ensure that the overall output after quantization is between this value and the feature map of the forward output. The target processing system can construct a third calibration data set with the same data form as the third training sample set, and perform performance tests on the model under different groups of quantization thresholds (a group of quantization thresholds includes the quantization thresholds corresponding to all processing layers in the model) through the third calibration data set, so as to find a group of quantization thresholds that can not only ensure the operation efficiency of the model but also reach a relatively high level in terms of accuracy and recall rate. Then, this group of quantization thresholds is applied to the trained initial quality detection model to obtain the quality detection model.

[0075] In an optional embodiment, the quality detection model is deployed on the first server. The first server can be understood as an edge - side server or also called a local server, which is relatively close to the target area. This deployment method makes full use of the instant response ability and local computing ability of the edge - side device, reduces the dependence on the cloud, and improves the overall efficiency and response speed of the system. Since the computing power of the edge - side is limited, therefore, by performing post - quantization processing on the trained initial quality monitoring model, the resource consumption of the edge - side can be effectively reduced.

[0076] It should be noted that by first using the quality detection model to perform conditional screening on the image, misjudgment caused by image quality problems can be avoided, the invalid calculations of the first difference detection model can be reduced, and thus the efficiency and accuracy of determining the detection result can be improved.

[0077] Optionally, in the device anomaly detection method provided in the embodiments of the present application, identifying the image area corresponding to the target device from the regional image to obtain the device image, and identifying the image area corresponding to the target device from the regional reference image to obtain the device reference image includes: identifying the regional image and the regional reference image through the target recognition model to obtain the device information corresponding to the regional image and the regional reference image respectively, where the device information at least includes the coordinates of the image areas corresponding to each device; determining any device in the regional image as the target device, and determining the device image of the target device according to the device information corresponding to the regional image; determining each device image area in the regional reference image according to the device information corresponding to the regional reference image, and determining the device reference image of the target device from the device image areas in the regional reference image according to the area intersection ratio between the device image of the target device and each device image area.

[0078] Optionally, the target recognition model is used for target detection. The target processing system can input the regional image and the regional reference image into the target recognition model to identify each device in the regional image and the regional reference image through the target recognition model, and obtain device information. The device information includes at least the coordinates (x1, y1, x2, y2) of the image region (such as a rectangular region) of each device object, and may also include the number of devices and the device type.

[0079] Optionally, the target processing system can determine any device in the regional image as the target device, and then determine the device image of the target device from the regional image according to the coordinate information in the device information.

[0080] Optionally, the target processing system can determine each device image region in the regional reference image according to the coordinate information in the device information corresponding to the regional reference image. Then, for each device image region, calculate the intersection over union (IoU) between the device image of the target device in the regional image and the device image region. Thus, the device image region with the largest intersection over union and exceeding the preset intersection over union (such as 90%) is determined as the device reference image.

[0081] Optionally, the target processing system can obtain the device image and the device reference image of the target device through image cropping.

[0082] In an optional embodiment, a hybrid structure of convolution and transformer can be used as the target recognition model. The target processing system can construct a fourth training sample set. The training samples in the fourth training sample set are sample regional images, and the true labels of the training samples in the fourth training sample set are the sample regional images with device detection box information after annotation. The initial recognition model is trained through the fourth training sample set to obtain a trained initial recognition model. The trained initial recognition model is determined as the target recognition model, or the trained initial recognition model is post-quantized to obtain the target recognition model. The post-quantization process of the trained initial recognition model can refer to the post-quantization process of the trained initial quality detection model, so it will not be elaborated here.

[0083] In an optional embodiment, the target recognition model is deployed on a second server. The second server can be understood as a cloud server or a remote server, which is relatively far from the target area.

[0084] It should be noted that by introducing the target recognition model and the device reference image matching mechanism, the accuracy and reliability of device anomaly detection are enhanced. First, the target recognition model ensures that the system can accurately detect and locate each device in the target area, providing an accurate image area for subsequent difference detection. Second, image matching is performed through the calculation method of the intersection over union ratio, ensuring that even when the device position is slightly adjusted or the environment changes, a matching reference image can be found, thereby improving the accuracy of image matching and further improving the accuracy of device anomaly detection.

[0085] Optionally, in the device anomaly detection method provided in the embodiments of the present application, the device information further includes the number of devices. Among them, determining any device in the regional image as the target device includes: judging whether the number of devices in the regional image and the regional reference image is the same according to the device information corresponding to the regional image and the regional reference image respectively; when the number of devices is the same, determining any device in the regional image as the target device.

[0086] Optionally, the target processing system can compare whether the number of devices in the device information corresponding to the regional image and the regional reference image is the same. If it is the same, any device in the regional image is determined as the target device to determine the device image and the device reference image. If it is different, a detection result indicating that there is no anomaly in the devices in the target area is directly output, and the device missing or device addition is recorded in the detection result.

[0087] It should be noted that through the above method, the number of invalid image comparison times can be effectively reduced, the consumption of computing resources can be reduced, and the efficiency of device anomaly detection can be improved.

[0088] Optionally, in the device anomaly detection method provided in the embodiments of the present application, the first difference detection model is obtained through the following method: obtaining a first training sample set, where the training samples in the first training sample set include the first sample regional image and the second sample regional image of the sample area, the second sample regional image is obtained by performing image enhancement on the first sample regional image, and the sample area includes at least one sample device; training an initial difference detection model through the first training sample set to obtain the first difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0089] Optionally, the first sample regional image is obtained by performing image acquisition on the sample area, the second sample regional image is obtained by performing image enhancement on the first sample regional image, and the image enhancement includes, but is not limited to, Gaussian blur, HSV transformation, brightness and contrast adjustment, etc., to simulate image changes under different environmental conditions. The second sample regional image is equivalent to the positive sample of the first sample regional image.

[0090] Optionally, after obtaining the first training sample set, the target processing system may train the initial difference detection model with the first training sample set to obtain the first difference detection model. During the training process, the initial difference detection model adopts contrastive learning for representation learning. The initial difference detection model includes an image encoder and a momentum image encoder.

[0091] In an optional embodiment, during the training process, the image encoder processes the first sample region image and the second sample region image to obtain the first image feature corresponding to the first sample region image and the second image feature corresponding to the second sample region image. The momentum image encoder processes the first sample region image and the second sample region image to obtain the third image feature corresponding to the first sample region image and the fourth image feature corresponding to the second sample region image. And the loss function value can be calculated through the following loss function:

[0092]

[0093] where, L Con represents the loss function value, exp() represents the natural exponential function, q represents the image feature output by the image encoder (the first image feature or the second image feature), k + represents the image feature output by the momentum image encoder (the third image feature or the fourth image feature). Among them, when q represents the first image feature, k + represents the fourth image feature. When q represents the second image feature, k + represents the third image feature. k i represents the i-th image in the first training sample set input to the momentum image encoder, k represents the total number of images in the first training sample set input to the momentum image encoder, and t represents the temperature parameter.

[0094] For example, a framework diagram of an optional initial difference detection model is as Figure 4 shown. Figure 4 In the picture 1 in it, it is equivalent to the above-mentioned first sample region image, picture 2 is equivalent to the above-mentioned second sample region image, q1 is equivalent to the above-mentioned first image feature, q2 is equivalent to the above-mentioned second image feature, k1 is equivalent to the above-mentioned third image feature, and k2 is equivalent to the above-mentioned fourth image feature. Multiply q1 and k2, and multiply q2 and k1 to calculate the loss function value.

[0095] In another alternative embodiment, during the training process, the first sample region image and the second sample region image are processed by an image encoder to obtain the first image feature corresponding to the first sample region image and the second image feature corresponding to the second sample region image. The images in the training samples other than the training sample to which the first sample region image belongs are determined as the third sample region images, and the third sample region images are processed by a momentum image encoder to obtain the fifth image feature corresponding to the third sample region images. The third sample region images are equivalent to the negative samples of the first sample region images. And the loss function value can be calculated by the following loss function:

[0096]

[0097] where sim() represents similarity calculation, q′ represents the feature representation of the anchor sample (i.e., the first image feature), k′ + represents the feature representation of the positive sample (i.e., the second image feature), k j represents the image feature of the negative sample (i.e., the fifth image feature), and F represents the number of negative samples selected.

[0098] In an alternative embodiment, the initial difference detection model trained by the first training sample set is determined as the first difference detection model.

[0099] In an alternative embodiment, the initial difference detection model trained by the first training sample set is determined as the third difference detection model, and post-quantization processing is performed on the third difference detection model to obtain the first difference detection model.

[0100] Optionally, the momentum image encoder itself does not perform gradient calculation, but is updated according to the updated weights of the image momentum encoder.

[0101] It should be noted that through the above method, effective self-supervised training of the initial difference detection model can be realized, thereby improving the model performance of the obtained first difference detection model.

[0102] Optionally, in the device anomaly detection method provided in the embodiments of the present application, training the initial difference detection model by the first training sample set to obtain the first difference detection model includes: training the initial difference detection model by the first training sample set to obtain the third difference detection model; performing post-quantization processing on the third difference detection model to obtain the first difference detection model.

[0103] Optionally, during the post-quantization process, the target processing system can set a quantization threshold for each processing layer in the third difference detection model. This threshold determines the precision and range after the quantization of the model parameters, and different thresholds will affect the performance of the model, such as accuracy and recall rate. For example, a threshold is set for the feature map of the forward output to ensure that the overall output after quantization is between this value and the feature map of the forward output. The target processing system can construct a first calibration data set with the same data form as the first training sample set, and perform performance tests on the model under different quantization threshold groups (a quantization threshold group includes the quantization thresholds corresponding to all processing layers in the model) through the first calibration data set, so as to find a quantization threshold group that can not only ensure the running efficiency of the model but also reach a relatively high level in terms of accuracy and recall rate. Then, apply this quantization threshold group to the third difference detection model to obtain the first difference detection model.

[0104] It should be noted that by performing post-quantization processing on the trained initial difference detection model, the resource consumption during the model operation can be effectively reduced, and the efficiency of determining the detection result can be improved.

[0105] Optionally, in the device anomaly detection method provided in the embodiments of the present application, the second difference detection model is obtained through the following steps: obtaining a second training sample set, where the training samples in the second training sample set include the first sample device image and the second sample device image of the sample device, and the second sample device image is obtained by performing image enhancement on the first sample device image; training the initial difference detection model through the second training sample set to obtain the second difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0106] Optionally, the first sample device image is obtained by performing image acquisition on the sample device. For example, it can be obtained by performing image cropping on the first sample area image. The second sample area image is obtained by performing image enhancement on the first sample area image. The image enhancement includes, but is not limited to, Gaussian blur, HSV transformation, brightness and contrast adjustment, etc., to simulate image changes under different environmental conditions. For example, the second sample device image can be obtained by performing image cropping on the second sample area. The second sample device image is equivalent to the positive sample of the first sample device image.

[0107] Optionally, after obtaining the second training sample set, the target processing system can train the initial difference detection model through the second training sample set to obtain the second difference detection model. During the training process, the initial difference detection model uses the method of contrastive learning for representation learning. The initial difference detection model includes an image encoder and a momentum image encoder.

[0108] Optionally, the initial difference detection model can be trained with the second training sample set in a manner similar to that of training with the first training sample set, so it will not be elaborated here.

[0109] In an optional embodiment, the initial difference detection model trained with the second training sample set is determined as the second difference detection model.

[0110] In an optional embodiment, the initial difference detection model trained with the second training sample set is determined as the fourth difference detection model, and post-quantization processing is performed on the fourth difference detection model to obtain the second difference detection model. For example, a second calibration data set with the same data form as the second training sample set is constructed, and the performance of the model under different quantization threshold groups (one quantization threshold group includes the quantization thresholds corresponding to all processing layers in the model) is tested with the second calibration data set to determine the second difference detection model.

[0111] It should be noted that through the above method, effective self-supervised training of the initial difference detection model can be achieved, thereby improving the model performance of the obtained second difference detection model.

[0112] Optionally, in the device anomaly detection method provided in the embodiments of the present application, the first difference detection model is deployed on the first server, and the second difference detection model is deployed on the second server, where the distance between the first server and the target area is less than the distance between the second server and the target area.

[0113] Optionally, the first server is an edge server, which can refer to an edge computing node, such as an intelligent camera or an edge server in a computer room. They are physically close to the "target area" and can quickly respond to and process image data from this area.

[0114] Optionally, the second server is a cloud server, which is physically far from the "target area". The purpose of this deployment strategy is to utilize the powerful computing power of the cloud server to perform in-depth analysis on the preliminarily screened device images and device reference images in order to determine the detection results.

[0115] It should be noted that by deploying the first difference detection model on the first server close to the target area, the data transmission delay and bandwidth consumption can be significantly reduced, because a large amount of preliminary screening work is completed at the edge, and only the images considered to be possibly abnormal will be sent to the second server for further analysis. This cloud-edge collaborative working mode not only improves the overall response speed of the system but also effectively distributes computing resources to ensure efficient and accurate anomaly detection capabilities even in the case of a large amount of real-time data streams.

[0116] In an optional embodiment,Figure 5 is the process of the device anomaly detection method provided by the embodiments of the present application Figure 2 , according to Figure 5 An optional application process of this embodiment will be described. As Figure 5 shown, the edge side pulls the real-time stream video data of each camera at the site of the current computer room (i.e., the target area), decodes it and saves the computer room snapshot image. Then, according to the input requirements of the quality detection model and the first difference detection model, image preprocessing is performed on the snapshot images at two different times before and after (i.e., the area image and the area reference image). After that, quality judgment and image difference judgment are carried out. If the quality passes and the two pictures are judged to be different, the original image pair data is uploaded to the cloud; otherwise, the current process ends. After that, after receiving the image pair data uploaded by the edge side, the cloud makes the input pictures meet the input requirements of the subsequent target recognition model through the image preprocessing module; the target recognition model is used to perform device detection on the two pictures respectively, and the device type, device quantity and their corresponding detection frame coordinates (x1, y1, x2, y2) in the corresponding images are obtained. According to the device quantity obtained in the previous step, if the device quantity in the two pictures changes, it is determined that a device is missing or added; if the device quantity does not change and the device quantity is not 0, the next step is entered, otherwise there is no output. After that, according to the category of the devices and their corresponding positions in the figure, the intersection over union (IoU) method is used to find the devices in the other picture with an IoU exceeding 90% for each device, and they are determined as the corresponding device reference images. Then, each pair of matched devices of the same type is cropped by image cropping, and after being processed by the image preprocessing module, it is sent to the second difference detection model. After the two pictures pass through the second difference detection model, the cosine similarity calculation method is used to obtain the feature similarity between the two after obtaining the corresponding features. If the feature similarity is greater than the preset similarity threshold, the next step is entered, otherwise there is no output. Then, the differential algorithm is used to visualize the pixel points with differences in the previous two device pictures to achieve difference indication. After that, image rendering is performed, and the returned alarm situation is visualized in the form of a coordinate box and stored in the platform data storage module. At the same time, the alarm situation is submitted for manual review.

[0117] It can be seen that the method provided by the embodiments of the present application achieves the purpose of using the model obtained by self-supervised training for device anomaly detection, and solves the problem of low detection accuracy in the related art when detecting device anomalies. Furthermore, the effect of improving the detection accuracy is achieved.

[0118] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0119] Embodiment 2

[0120] The embodiment of the present application also provides a device detection device. It should be noted that the device detection device in the embodiment of the present application can be used to execute the device anomaly detection method provided in the embodiment of the present application. The device detection device provided in the embodiment of the present application will be introduced below.

[0121] According to the embodiment of the present application, there is also provided a device for implementing the above-mentioned device anomaly detection method. Figure 6 is a schematic diagram of the device detection device for device anomaly provided according to the embodiment of the present application, as Figure 6 shown, the device includes:

[0122] An acquisition module 601, configured to acquire a region image of a target region and a region reference image of the target region, wherein at least one device is arranged in the target region;

[0123] A first determination module 602, configured to determine the feature similarity between the region image and the region reference image through a first difference detection model to obtain a first similarity, wherein the first difference detection model is trained in a self-supervised manner;

[0124] A processing module 603, configured to, when the first similarity is less than a first preset threshold, identify an image region corresponding to the target device from the region image to obtain a device image, and identify an image region corresponding to the target device from the region reference image to obtain a device reference image;

[0125] A second determination module 604, configured to determine the feature similarity between the device image and the device reference image through a second difference detection model to obtain a second similarity, wherein the second difference detection model is trained in a self-supervised manner;

[0126] A third determination module 605, configured to determine the detection result of the target device according to the second similarity, wherein the detection result is used to characterize whether the target device has an anomaly.

[0127] In the embodiment of the present application, the first difference detection model and the second difference detection model are designed and trained in a self-supervised manner. The model can learn rich feature representations from a large amount of unlabeled data, thereby improving the generalization ability of the model. By using the first difference detection model and the second difference detection model to determine the feature similarity between images, the accuracy of the determined similarity is improved. Therefore, when determining the detection result based on the similarity, the accuracy of the determined detection result can be effectively improved. In addition, by first using the first detection model to detect the regional image difference, and when there is a difference in the regional image, then performing the device image difference detection, the device detection efficiency can be effectively improved.

[0128] It can be seen that the method provided by the embodiment of the present application achieves the purpose of using the model trained by self-supervision for device anomaly detection, and solves the problem of low detection accuracy in the related art when performing anomaly detection on devices. Furthermore, the effect of improving the detection accuracy is achieved.

[0129] Optionally, in the device detection device provided by the embodiment of the present application, the first determination module further includes: a detection sub-module, configured to detect the image quality of the regional image and the regional reference image through a quality detection model to obtain a quality detection result; a first determination sub-module, configured to determine the feature similarity between the regional image and the regional reference image through the first difference detection model to obtain a first similarity when it is determined that the quality of the regional image and the regional reference image is qualified according to the quality detection result.

[0130] Optionally, in the device detection device provided by the embodiment of the present application, the processing module further includes: a first processing sub-module, configured to identify the regional image and the regional reference image through a target recognition model to obtain the device information corresponding to the regional image and the regional reference image respectively, where the device information at least includes the coordinates of the image regions corresponding to each device; a second determination sub-module, configured to determine any device in the regional image as the target device, and determine the device image of the target device according to the device information corresponding to the regional image; a third determination sub-module, configured to determine each device image region in the regional reference image according to the device information corresponding to the regional reference image, and determine the device reference image of the target device from the device image regions in the regional reference image according to the area intersection ratio between the device image of the target device and each device image region.

[0131] Optionally, in the device detection device provided by the embodiment of the present application, the second determination sub-module further includes: a judgment unit, configured to judge whether the number of devices in the regional image and the regional reference image is the same according to the device information corresponding to the regional image and the regional reference image respectively; a determination unit, configured to determine any device in the regional image as the target device when the number of devices is the same.

[0132] Optionally, in the device detection apparatus provided in the embodiments of the present application, the device detection apparatus further includes: a first acquisition module, configured to acquire a first training sample set, where the training samples in the first training sample set include a first sample region image and a second sample region image of a sample region, the second sample region image is obtained by performing image enhancement on the first sample region image, and the sample region includes at least one sample device; a first training module, configured to train an initial difference detection model through the first training sample set to obtain a first difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0133] Optionally, in the device detection apparatus provided in the embodiments of the present application, the first training module further includes: a training sub-module, configured to train an initial difference detection model through the first training sample set to obtain a third difference detection model; a second processing sub-module, configured to perform post-quantization processing on the third difference detection model to obtain a first difference detection model.

[0134] Optionally, in the device detection apparatus provided in the embodiments of the present application, the device detection apparatus further includes: a second acquisition module, configured to acquire a second training sample set, where the training samples in the second training sample set include a first sample device image and a second sample device image of a sample device, the second sample device image is obtained by performing image enhancement on the first sample device image; a second training module, configured to train an initial difference detection model through the second training sample set to obtain a second difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0135] Optionally, in the device detection apparatus provided in the embodiments of the present application, the first difference detection model is deployed on a first server, and the second difference detection model is deployed on a second server, where the distance between the first server and the target region is less than the distance between the second server and the target region.

[0136] It should be noted here that the above acquisition module 601, first determination module 602, processing module 603, second determination module 604, and third determination module 605 correspond to steps S201 to S205 in Embodiment 1. The five modules have the same implementation examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules or units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n), and the above modules may also be part of the apparatus and can run in the computer terminal 10 provided in Embodiment 1.

[0137] Embodiment 3

[0138] An embodiment of the present application may provide an electronic device. Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present application. As Figure 7 shown, the electronic device may include: one or more ( Figure 7 only one is shown in the figure) processors 1002, a memory 1004, a storage controller, and a peripheral interface, where the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0139] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above methods. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely disposed relative to the processor, and these remote memories may be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0140] The processor may call the information and application programs stored in the memory through a transmission device to perform the following steps: obtaining a regional image of a target area and a regional reference image of the target area, where at least one device is provided in the target area; determining a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity, where the first difference detection model is trained in a self-supervised manner; in the case where the first similarity is less than a first preset threshold, identifying an image area corresponding to the target device from the regional image to obtain a device image, and identifying an image area corresponding to the target device from the regional reference image to obtain a device reference image; determining a feature similarity between the device image and the device reference image through a second difference detection model to obtain a second similarity, where the second difference detection model is trained in a self-supervised manner; determining a detection result of the target device according to the second similarity, where the detection result is used to characterize whether the target device is abnormal.

[0141] The processor may also call the information and application programs stored in the memory through a transmission device to perform the following steps: detecting the image quality of the regional image and the regional reference image through a quality detection model to obtain a quality detection result; in the case where it is determined according to the quality detection result that the quality of the regional image and the regional reference image is qualified, determining a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity.

[0142] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: identify the regional image and the regional reference image through the target recognition model to obtain the device information corresponding to the regional image and the regional reference image respectively, where the device information at least includes the coordinates of the image regions corresponding to each device; determine any device in the regional image as the target device, and determine the device image of the target device according to the device information corresponding to the regional image; determine each device image region in the regional reference image according to the device information corresponding to the regional reference image, and determine the device reference image of the target device from the device image regions in the regional reference image according to the area intersection ratio between the device image of the target device and each device image region.

[0143] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: judge whether the number of devices in the regional image and the regional reference image is the same according to the device information corresponding to the regional image and the regional reference image respectively; in the case where the number of devices is the same, determine any device in the regional image as the target device.

[0144] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain the first training sample set, where the training samples in the first training sample set include the first sample regional image and the second sample regional image of the sample region, the second sample regional image is obtained by performing image enhancement on the first sample regional image, and the sample region includes at least one sample device; train the initial difference detection model through the first training sample set to obtain the first difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0145] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: train the initial difference detection model through the first training sample set to obtain the third difference detection model; perform post-quantization processing on the third difference detection model to obtain the first difference detection model.

[0146] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: obtaining a second training sample set, where the training samples in the second training sample set include the first sample device image and the second sample device image of the sample device, and the second sample device image is obtained by performing image enhancement on the first sample device image; training an initial difference detection model with the second training sample set to obtain a second difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

[0147] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: the first difference detection model is deployed on the first server, and the second difference detection model is deployed on the second server, where the distance between the first server and the target area is less than the distance between the second server and the target area.

[0148] Those of ordinary skill in the art can understand that Figure 7 The structure shown is only schematic, and the electronic device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and terminal devices such as Mobile Internet Devices (MIDs), PADs, etc. Figure 7 It does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components (such as a network interface, a display device, etc.) than those shown Figure 7 in the figure, or have a different configuration from that shown Figure 7 in the figure.

[0149] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware of the terminal device through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0150] Embodiment 4

[0151] An embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the above storage medium can be used to save the program code executed by the method for detecting device anomalies provided in the first embodiment above.

[0152] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0153] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for detecting method steps of device anomalies.

[0154] The serial numbers of the above embodiments of the present application are only for description and do not represent the superiority or inferiority of the embodiments.

[0155] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0156] In the several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0157] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, the functional units in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0159] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs.

[0160] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of this application.

Claims

1. A method for detecting device anomalies, characterized in that, Including: Obtaining a regional image of a target region and a regional reference image of the target region, wherein at least one device is arranged in the target region; Determining a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity, wherein the first difference detection model is trained in a self-supervised manner; When the first similarity is less than a first preset threshold, identifying an image region corresponding to a target device from the regional image to obtain a device image, and identifying an image region corresponding to the target device from the regional reference image to obtain a device reference image; Determining a feature similarity between the device image and the device reference image through a second difference detection model to obtain a second similarity, wherein the second difference detection model is trained in a self-supervised manner; Determining a detection result of the target device according to the second similarity, wherein the detection result is used to represent whether the target device is abnormal.

2. The method according to claim 1, wherein, Determining a feature similarity between the regional image and the regional reference image through a first difference detection model to obtain a first similarity, including: Detecting the image quality of the regional image and the regional reference image through a quality detection model to obtain a quality detection result; When it is determined according to the quality detection result that the quality of the regional image and the regional reference image is qualified, determining a feature similarity between the regional image and the regional reference image through the first difference detection model to obtain the first similarity.

3. The method according to claim 1, characterized in that Identifying an image region corresponding to a target device from the regional image to obtain a device image, and identifying an image region corresponding to the target device from the regional reference image to obtain a device reference image, including: Identifying the regional image and the regional reference image through a target recognition model to obtain device information corresponding to the regional image and the regional reference image respectively, wherein the device information at least includes coordinates of image regions corresponding to each device; Determining any device in the regional image as the target device, and determining a device image of the target device according to the device information corresponding to the regional image; Determining each device image region in the regional reference image according to the device information corresponding to the regional reference image, and determining a device reference image of the target device from the device image regions in the regional reference image according to an area intersection ratio between the device image of the target device and each device image region.

4. The method according to claim 3, wherein The device information further includes the number of devices, wherein determining any device in the regional image as the target device includes: Judging whether the number of devices in the regional image is the same as that in the regional reference image according to the device information corresponding to the regional image and the regional reference image respectively; When the number of devices is the same, determining any device in the regional image as the target device.

5. The method according to claim 1, wherein The first difference detection model is obtained through the following method: Obtain a first training sample set, where the training samples in the first training sample set include a first sample region image and a second sample region image of a sample region, and the second sample region image is obtained by performing image enhancement on the first sample region image, and at least one sample device is included in the sample region; Train an initial difference detection model with the first training sample set to obtain the first difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

6. The method according to claim 5, characterized in that, Training the initial difference detection model with the first training sample set to obtain the first difference detection model includes: Train the initial difference detection model with the first training sample set to obtain a third difference detection model; Perform post-quantization processing on the third difference detection model to obtain the first difference detection model.

7. The method according to claim 1, characterized in that, The second difference detection model is obtained by the following method: Obtain a second training sample set, where the training samples in the second training sample set include a first sample device image and a second sample device image of a sample device, and the second sample device image is obtained by performing image enhancement on the first sample device image; Train an initial difference detection model with the second training sample set to obtain the second difference detection model, where the initial difference detection model includes an image encoder and a momentum image encoder, and the initial difference detection model performs self-supervised learning based on the image encoder and the momentum image encoder.

8. The method according to claim 1, characterized in that, The first difference detection model is deployed on a first server, and the second difference detection model is deployed on a second server, where the distance between the first server and the target region is less than the distance between the second server and the target region.

9. A device detection apparatus, characterized in that, Includes: An acquisition module, configured to acquire a region image of a target region and a region reference image of the target region, where at least one device is arranged in the target region; A first determination module, configured to determine the feature similarity between the region image and the region reference image through the first difference detection model to obtain a first similarity, where the first difference detection model is trained in a self-supervised manner; A processing module, configured to, when the first similarity is less than a first preset threshold, identify an image region corresponding to a target device from the region image to obtain a device image, and identify an image region corresponding to the target device from the region reference image to obtain a device reference image; A second determination module, configured to determine the feature similarity between the device image and the device reference image through the second difference detection model to obtain a second similarity, where the second difference detection model is trained in a self-supervised manner; A third determination module, configured to determine a detection result of the target device according to the second similarity, where the detection result is used to characterize whether the target device has an abnormality.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the detection method for device anomalies described in any one of claims 1 to 8.

11. An electronic device, characterized in that, Comprising: a memory storing an executable program; a processor for running the program, wherein when the program runs, it executes the detection method for device anomalies described in any one of claims 1 to 8.

12. A computer program product, comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, the steps of the detection method for device anomalies described in any one of claims 1 to 8 are implemented.

Images