Equipment exception monitoring method and device, equipment and storage medium

Abstract

The invention discloses an equipment abnormity monitoring method and device, equipment and a storage medium. The method comprises the following steps: acquiring a to-be-detected image of to-be-detected equipment; inputting the to-be-detected image into a trained monitoring model, and outputting an abnormal result of the to-be-detected equipment; wherein the monitoring model comprises a backbone network and a plurality of independent task feature towers, when the monitoring model is trained, after sample images are subjected to shared representation learning through the backbone network, extracted features are input into the task feature towers respectively, and the task feature towers process and output feature recognition results of abnormal types which are responsible for the task feature towers respectively; and during training, according to the real probability and the prediction probability of the sample image belonging to each abnormal type and the weight coefficient of each abnormal type, constructing a target loss function of the monitoring model. By utilizing the method disclosed by the invention, the full-automatic, comprehensive and efficient abnormal type monitoring of the to-be-tested equipment can be realized, the dependence on manpower is reduced, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics monitoring, and in particular to a method, device, equipment and storage medium for monitoring equipment abnormality. Background Art

[0002] In the existing logistics monitoring field, equipment inspections mainly rely on manual execution, which is not only labor-intensive but also consumes a lot of human resources, resulting in low efficiency and high costs. Especially for the operational monitoring of vehicle-mounted equipment, its maintenance strategy requires team leaders to conduct regular inspections, and fleet managers need to conduct regular comprehensive inspections. This periodic manual inspection mode significantly increases labor costs and management burdens. In addition, when equipment failures need to be reported for maintenance, the response time of this process is often too long, which will have a negative impact on service quality and customer satisfaction; and the existing monitoring model can only detect one type of equipment anomalies, and cannot monitor multiple types of equipment anomalies at the same time, further resulting in low monitoring efficiency.

[0003] Therefore, there is an urgent need for a method that can automatically, comprehensively and efficiently identify anomalies of various logistics equipment. Summary of the invention

[0004] The present invention provides a device abnormality monitoring method, device, equipment and storage medium, which are used to solve the technical problems that the existing device abnormality monitoring method relies on manual inspection, has low efficiency, long response time and cannot comprehensively monitor various types of device abnormalities.

[0005] In order to solve the above technical problems, in a first aspect, the present invention provides a device abnormality monitoring method, the method comprising:

[0006] Acquire the image to be tested of the device to be tested;

[0007] Inputting the image to be detected into a trained monitoring model, and outputting the abnormal result of the device to be tested;

[0008] Among them, the monitoring model includes a backbone network and multiple independent task feature towers. When the monitoring model is trained, the sample image is shared by the backbone network for representation learning, and the extracted features are respectively input into each of the task feature towers. The task feature towers process and output the feature recognition results of the abnormality type they are responsible for. During training, the target loss function of the monitoring model is constructed based on the true probability and predicted probability that the sample image belongs to each of the abnormality types, as well as the weight coefficient of each of the abnormality types.

[0009] Optionally, constructing a target loss function of the monitoring model according to the true probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types, includes:

[0010]

[0011] Among them, Loss represents the target loss function, n represents the number of abnormal types, k i Represents the weight coefficient of the i-th abnormal type, y i represents the true probability that the sample image has the i-th abnormal type, It indicates the predicted probability that the sample image has the i-th abnormal type.

[0012] Optionally, after the sample image is learned through the shared representation of the backbone network, the extracted features are respectively input into each of the task feature towers, including:

[0013] The backbone network includes a shared layer constructed by a convolutional layer and a pooling layer, and the shared layer extracts common features that characterize the abnormality type and inputs them into each of the task feature towers respectively.

[0014] Optionally, the task feature tower processes and outputs feature recognition results of the anomaly types that each task feature tower is responsible for, including:

[0015] Each of the task feature towers includes a fully connected layer for processing feature identification of the abnormal type it is responsible for. The fully connected layer identifies the features of the abnormal type it is responsible for from the general features and outputs the feature identification result.

[0016] Optionally, the weight coefficient calculation formula of the i-th abnormal category is as follows:

[0017]

[0018] Wherein, m represents the total number of sample images, n i Indicates the number of sample images corresponding to the i-th abnormal type.

[0019] Optionally, obtaining the image to be tested of the device to be tested includes:

[0020] Acquire an initial image to be detected of the device to be tested;

[0021] The size of the initial image to be detected is adjusted to a preset size, and denoising is performed to obtain the image to be detected.

[0022] Optionally, the outputting the abnormal result of the device under test includes:

[0023] If the abnormal result of the device under test is that an abnormality exists, all abnormality types are output.

[0024] In a second aspect, the present invention provides a device for monitoring equipment abnormality, including a collection module and a monitoring module;

[0025] The acquisition module is used to obtain the image to be tested of the device to be tested;

[0026] The monitoring module is used to input the image to be detected into the trained monitoring model and output the abnormal result of the device to be tested;

[0027] Among them, the monitoring model includes a backbone network and multiple independent task feature towers. When the monitoring model is trained, the sample image is shared by the backbone network for representation learning, and the extracted features are respectively input into each of the task feature towers. The task feature towers process and output the feature recognition results of the abnormality type they are responsible for. During training, the target loss function of the monitoring model is constructed based on the true probability and predicted probability that the sample image belongs to each of the abnormality types, as well as the weight coefficient of each of the abnormality types.

[0028] In a third aspect, the present invention provides a device for monitoring device abnormality, including a memory and a processor, wherein:

[0029] The memory is used to store computer programs;

[0030] The processor is used to read the program in the memory and execute the steps of a device abnormality monitoring method provided in the first aspect above.

[0031] In a fourth aspect, the present invention provides a computer-readable storage medium having a readable computer program stored thereon, which, when executed by a processor, implements the steps of a device abnormality monitoring method provided in the first aspect above.

[0032] Compared with the prior art, the device abnormality monitoring method, apparatus, device and storage medium provided by the present invention have the following beneficial effects:

[0033] The method obtains an image to be detected of the device to be tested; inputs the image to be detected into a trained monitoring model, and outputs the abnormal result of the device to be tested; wherein the monitoring model includes a backbone network and multiple independent task feature towers, when the monitoring model is trained, after the sample image is shared and represented by the backbone network, the extracted features are respectively input into each of the task feature towers, and the task feature towers process and output the feature recognition results of the abnormal types they are responsible for; during training, the target loss function of the monitoring model is constructed according to the real probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types. Through this method, the inspection task of the device to be tested can be automatically completed, which greatly reduces the dependence on manpower, thereby reducing the abnormal monitoring cost of the device to be tested; at the same time, the monitoring model provided by the method can identify various abnormal types of the device to be tested, and there is no need to train a separate monitoring model for each abnormal type, and the target loss function of the monitoring model can fuse the loss functions corresponding to each abnormal type, and realize the model training of multiple abnormal type recognition tasks in one back propagation process, and the monitoring model provided by the method can realize the abnormal monitoring of the device to be tested comprehensively and efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only part of the embodiments of the present invention, rather than all of the embodiments. For ordinary technicians in this field, without paying creative work, other drawings obtained based on these drawings all fall within the scope of protection of this application.

[0035] Figure 1 is a flow chart of a device abnormality monitoring method provided by an embodiment of the present invention;

[0036] Figure 2 is a structural diagram of a monitoring model provided by an embodiment of the present invention;

[0037] Figure 3 It is a structural schematic diagram of a device abnormality monitoring device provided by an embodiment of the present invention;

[0038] Figure 4 It is a structural schematic diagram of a device abnormality monitoring device provided by an embodiment of the present invention;

[0039] Figure 5 It is a structural schematic diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] In order to make the description of the disclosed content more detailed and complete, the following is an illustrative description of the implementation mode and specific examples of the present invention; however, this is not the only form of implementing or applying the specific embodiments of the present invention. The implementation mode covers the features of multiple specific embodiments and the method steps and their sequence for constructing and operating these specific embodiments. However, other specific embodiments can also be used to achieve the same or equal functions and step sequences. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] In the description of the embodiments of the present invention, "multiple" means two or more than two, and other quantifiers are similar and should be understood. The preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention. In addition, the embodiments of the present application and the features therein may be combined with each other without conflict.

[0043] Example 1

[0044] like Figure 1 The flowchart of a device abnormality monitoring method provided in an embodiment of the present invention includes the following steps.

[0045] Step S101, obtaining an image to be tested of the device to be tested;

[0046] The device under test may be an image acquisition device. In order to monitor the abnormality of the device under test, the image captured by the device under test is read from the device under test to obtain the image to be detected. Exemplarily, the device under test may be a camera, a driving recorder, etc., which may be determined according to actual conditions, and the embodiments of the present invention do not specifically limit this.

[0047] As an optional implementation, in step S101, obtaining the image to be tested of the device to be tested includes:

[0048] Acquire an initial image to be detected of the device to be tested;

[0049] The size of the initial image to be detected is adjusted to a preset size, and denoising is performed to obtain the image to be detected.

[0050] Exemplarily, an image collected during the operation of the device to be tested is used as an initial detection image. Before the initial detection image is input into the monitoring model, the size of the initial detection image is adjusted to a preset size, for example, the size of the initial detection image is adjusted to 224*224PPI (Pixels Per Inch); and the initial detection image is denoised using a filter such as a Gaussian blur filter or a median filter, and the image obtained after processing is used as the image to be detected.

[0051] Step S102, input the image to be detected into the trained monitoring model, and output the abnormal result of the device to be tested; wherein, the monitoring model includes a backbone network and multiple independent task feature towers. When the monitoring model is trained, the sample image is shared by the backbone network for representation learning, and the extracted features are respectively input into each of the task feature towers, and the task feature towers process and output the feature recognition results of the abnormal type they are responsible for; during training, the target loss function of the monitoring model is constructed according to the true probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types.

[0052] Exemplarily, the structure of the constructed monitoring model includes a backbone network and multiple independent task feature towers, and the output end of the backbone network is respectively connected to the input end of each task feature tower, wherein the number of task feature towers corresponds to the number of possible abnormality types that may exist in the device under test, and each task feature tower is used to implement an abnormality type identification task. The number of the task feature towers can be determined based on actual conditions, and the embodiment of the present invention does not specifically limit this.

[0053] During the training process of the monitoring model, the sample image is input into the monitoring model. The sample image is first shared by the backbone network for representation learning. After the extracted features are output from the backbone network, they are input into each independent task feature tower respectively. Each task feature tower processes and outputs the feature recognition result of the abnormal type recognition task it is responsible for. According to the feature recognition result, the abnormal result is output. During the training process of the sample image input into the monitoring model, the monitoring model determines the true probability of the sample image belonging to each abnormal type based on the pre-labeling of the sample image and the abnormal type. The predicted probability of each abnormal type is obtained by combining the model to predict the sample image belonging to each abnormal type, and the weight coefficient of each abnormal type is used to construct the target loss function, so as to adjust the parameters of the monitoring model until a trained monitoring model is obtained.

[0054] The image to be tested is input into the trained monitoring model, and the abnormal results related to the device to be tested and each abnormal type are directly output.

[0055] As an optional implementation manner, the outputting the abnormal result of the device under test includes:

[0056] If the abnormal result of the device under test is that an abnormality exists, all abnormality types are output.

[0057] Exemplarily, taking the camera as an example, the abnormality types may be a black screen, a flowery screen, dirt, and an infrared abnormality. If the monitoring model detects that the device under test has at least one of these abnormality types, the output abnormality result is that the device under test is abnormal, and the corresponding abnormality type is output; if the monitoring device detects that the device under test does not have any abnormality type, the output abnormality result is that the device under test is normal.

[0058] In this embodiment, the method obtains the image to be detected of the device to be tested; the image to be detected is input into the trained monitoring model, and the abnormal result of the device to be tested is output; wherein the monitoring model includes a backbone network and multiple independent task feature towers, when the monitoring model is trained, the sample image is shared by the backbone network, and the extracted features are respectively input into each of the task feature towers, and the task feature towers process and output the feature recognition results of the abnormal types they are responsible for; during the training, the target loss function of the monitoring model is constructed according to the real probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types. Through this method, the inspection task of the device to be tested can be automatically completed, which greatly reduces the dependence on manpower, and thus reduces the abnormal monitoring cost of the device to be tested; at the same time, the monitoring model provided by the method can identify various abnormal types of the device to be tested, and there is no need to train a separate monitoring model for each abnormal type, and the target loss function of the monitoring model can fuse the loss functions corresponding to each abnormal type, and realize the model training of multiple abnormal type recognition tasks in one back propagation process, and the monitoring model provided by the method can realize the abnormal monitoring of the device to be tested comprehensively and efficiently.

[0059] As an optional implementation manner, the target loss function of the monitoring model is constructed according to the true probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types, including:

[0060]

[0061] Among them, Loss represents the target loss function, n represents the number of abnormal types, k i Represents the weight coefficient of the i-th abnormal type, y i represents the true probability that the sample image has the i-th abnormal type, It indicates the predicted probability that the sample image has the i-th abnormal type.

[0062] For example, in the objective loss function, we can set Among them, 1 represents that the sample image belongs to the abnormal type, and 0 represents that it does not belong to it. According to the logarithmic value of the true probability and predicted probability of whether the sample image belongs to the abnormal type, the loss function corresponding to each abnormal type in the monitoring model training process is determined, and the weight coefficient corresponding to each abnormal type is used. As mentioned above, when the abnormal types include four types: black screen, flower screen, dirt and infrared abnormality, n = 4, then the target loss function is:

[0063]

[0064] Among them, n i is the total number of training samples of the ith category; i = {1, 2, 3, 4} represents four abnormal scene types: black screen, flowery screen, dirty, and infrared abnormality.

[0065] In this embodiment, the target loss function uses logarithmic probability, which can be saturated when the output is close to the true label, and will increase rapidly when the prediction is wrong, thereby providing a stronger learning signal to the model; at the same time, the logarithmic probability can ensure that even very small probability values ​​can be accurately represented, thereby avoiding numerical instability when calculating losses. The designed target loss function allows the monitoring model to take into account the detection training of multiple types of anomalies, allowing the model to converge quickly and effectively in calculating multiple tasks, achieving the expected effect of the model.

[0066] As an optional implementation, the weight coefficient calculation formula of the i-th abnormal category is as follows:

[0067]

[0068] Wherein, m represents the total number of sample images, n i Indicates the number of sample images corresponding to the i-th abnormal type.

[0069] Exemplarily, the sample image includes a label related to the abnormality type, and the weight coefficient corresponding to each abnormality type is determined according to the number of sample images corresponding to each abnormality type in the sample image training set and the total number of sample images in the sample image training set.

[0070] In the embodiment of the present invention, by setting the weight coefficient of each abnormal type, the problem of uneven distribution of data samples when identifying different abnormal types, that is, the so-called "data imbalance" problem, is solved. By adding a weight adjustment mechanism to the target loss function, the importance of various abnormal identification tasks can be better weighed, ensuring that even when there is a significant difference in the number of various abnormal sample images, the monitoring model can still perform comprehensive and accurate learning, thereby maintaining a high level of performance in various scenarios.

[0071] As an optional implementation, after the sample image is shared and represented by the backbone network, the extracted features are respectively input into each of the task feature towers, including:

[0072] The backbone network includes a shared layer constructed by a convolutional layer and a pooling layer, and the shared layer extracts common features that characterize the abnormality type and inputs them into each of the task feature towers respectively.

[0073] Exemplarily, the backbone network is a deep neural network composed of a series of convolutional layers, pooling layers, activation functions, etc. For example, the backbone network can be ResNet18 (Residual Network-18, 18-layer residual network). The first part of the backbone network is a shared layer including convolutional layers and pooling layers. The shared layer processes the image input to the monitoring model and extracts common features that can be used for all abnormality types. The common features can be one or more combinations of color features, texture features, shape features, and spatial relationship features. The extracted common features are respectively input into the task feature tower.

[0074] As an optional implementation, the task feature tower processes and outputs the feature recognition results of the abnormality type that each task feature tower is responsible for, including:

[0075] Each of the task feature towers includes a fully connected layer for processing feature identification of the abnormal type it is responsible for. The fully connected layer identifies the features of the abnormal type it is responsible for from the general features and outputs the feature identification result.

[0076] Exemplarily, after learning the shared representation of the backbone network, based on the abnormality type recognition task that the monitoring model needs to perform, the network is split into independent branches, each of which is responsible for a specific abnormality type recognition task. Each branch is called a "tower" as a task feature tower. Each task feature tower includes a fully connected layer set according to the abnormality type recognition task that each task feature tower is responsible for, and the fully connected layer recognizes the features of the responsible abnormality type based on the input common features, and outputs the feature recognition result of the abnormality type. For example, Figure 2The structural diagram of the monitoring model shown in the figure, the monitoring model needs to perform four types of abnormality identification tasks including black screen, flower screen, dirt and infrared abnormality. The backbone network is connected to four task feature towers, namely task feature tower 1, task feature tower 2, task feature tower 3 and task feature tower 4. Among them, task feature tower 1 is used to detect whether the general feature has a black screen feature, task feature tower 2 is used to detect whether the general feature has a flower screen feature, task feature tower 3 is used to detect whether the general feature has a dirt feature, and task feature tower 4 is used to detect whether the general feature has an infrared abnormality feature. After the four task feature towers identify the abnormality types they are responsible for, they output their own feature recognition results respectively. According to the feature recognition results, the abnormal results are output.

[0077] In an embodiment of the present invention, common features are extracted through a backbone network, and then input into a task feature tower to perform recognition tasks of different abnormality types. This can achieve data sharing of image features, reduce the model's video memory usage and computing cost, reduce the response time of the monitoring model, and improve the efficiency of the monitoring model.

[0078] In order to verify the monitoring efficiency of the method in the embodiment of the present invention, the accuracy, time consumption and video memory of the independent model in the prior art in identifying the four abnormal types of black screen, flowery screen, dirt and infrared abnormality on the device under test are compared, and the effect comparison table shown in Table 1 is obtained:

[0079] Table 1

[0080]

[0081] As can be seen from Table 1, compared with the independent model of the prior art, the monitoring model disclosed in this solution shortens the detection time from 228 milliseconds to 108 milliseconds, which is a reduction of about 53%; at the same time, the video memory consumption is reduced from 3490 megabytes (M) to 1379 megabytes (M), which is a reduction of about 60%. Although the monitoring model of this solution has a slight decline in calculation accuracy, from the total accuracy of the independent model of 95.6% to 95.2%, a decrease of about 0.4%, overall, this application solution performs well in improving efficiency and saving resources, at the cost of a slight compromise in accuracy.

[0082] Compared with the prior art, the device abnormality monitoring method provided by the present invention has the following beneficial effects:

[0083] The method obtains an image to be detected of the device to be tested; inputs the image to be detected into a trained monitoring model, and outputs the abnormal result of the device to be tested; wherein the monitoring model includes a backbone network and multiple independent task feature towers, when the monitoring model is trained, after the sample image is shared and represented by the backbone network, the extracted features are respectively input into each of the task feature towers, and the task feature towers process and output the feature recognition results of the abnormal types they are responsible for; during training, the target loss function of the monitoring model is constructed according to the real probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types. Through this method, the inspection task of the device to be tested can be automatically completed, which greatly reduces the dependence on manpower, thereby reducing the abnormal monitoring cost of the device to be tested; at the same time, the monitoring model provided by the method can identify various abnormal types of the device to be tested, and there is no need to train a separate monitoring model for each abnormal type, and the target loss function of the monitoring model can fuse the loss functions corresponding to each abnormal type, and realize the model training of multiple abnormal type recognition tasks in one back propagation process, and the monitoring model provided by the method can realize the abnormal monitoring of the device to be tested comprehensively and efficiently.

[0084] Example 2

[0085] Based on the above-mentioned device abnormality monitoring method, an embodiment of the present invention provides a device abnormality monitoring device, such as Figure 3 As shown, the device abnormality monitoring device includes a collection module and a monitoring module;

[0086] The acquisition module 310 is used to obtain the image to be tested of the device to be tested;

[0087] The monitoring module 320 is used to input the image to be detected into the trained monitoring model and output the abnormal result of the device to be tested;

[0088] Among them, the monitoring model includes a backbone network and multiple independent task feature towers. When the monitoring model is trained, the sample image is shared by the backbone network for representation learning, and the extracted features are respectively input into each of the task feature towers. The task feature towers process and output the feature recognition results of the abnormality type they are responsible for. During training, the target loss function of the monitoring model is constructed based on the true probability and predicted probability that the sample image belongs to each of the abnormality types, as well as the weight coefficient of each of the abnormality types.

[0089] For other details about how the modules in the above-mentioned equipment abnormality monitoring device implement the above-mentioned technical solution, please refer to the description of the equipment abnormality monitoring method provided in the above-mentioned invention embodiment, which will not be repeated here.

[0090] Based on the above equipment abnormality monitoring method, such as Figure 4As shown, an embodiment of the present invention further provides a structural schematic diagram of a device abnormality monitoring device, the identification device includes a processor 41 and a memory 42 coupled to the processor 41. The memory 42 stores a computer program, and when the computer program is executed by the processor 41, the processor 41 executes the steps of the device abnormality monitoring method in the above embodiment.

[0091] For other details about how the processor 41 in the above-mentioned device abnormality monitoring device implements the above-mentioned technical solution, please refer to the description of the device abnormality monitoring method provided in the above-mentioned invention embodiment, which will not be repeated here.

[0092] Among them, the processor 41 can also be called a CPU (Central Processing Unit), and the processor 41 may be an integrated circuit chip with signal processing capabilities; the processor 41 can also be a general-purpose processor, DSP (Digital Signal Process), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, among which the general-purpose processor can be a microprocessor or the processor 41 can also be any conventional processor, etc.

[0093] like Figure 5 As shown, the embodiment of the present invention also provides a schematic diagram of the structure of a computer-readable storage medium, on which a readable computer program 51 is stored; wherein the computer program 51 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, magnetic disk or optical disk, ROM (Read-Only Memory), RAM (Random Access Memory) and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0094] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0095] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0097] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0098] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0099] The technical solution provided by the present application is introduced in detail above. The principles and implementation methods of the present application are explained by using specific examples in the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0101] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0104] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for monitoring equipment abnormality, characterized in that: include: Acquire the image to be tested of the device to be tested; Inputting the image to be detected into a trained monitoring model, and outputting the abnormal result of the device to be tested; The monitoring model includes a backbone network and multiple independent task feature towers. When the monitoring model is trained, after the sample image is shared and represented by the backbone network, the extracted features are respectively input into each of the task feature towers, and the task feature towers process and output the feature recognition results of the abnormal type they are responsible for. During training, the target loss function of the monitoring model is constructed based on the true probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types.

2. The device abnormality monitoring method according to claim 1, characterized in that: The target loss function of the monitoring model is constructed based on the true probability and predicted probability that the sample image belongs to each of the abnormal types, and the weight coefficient of each of the abnormal types, including: Among them, Loss represents the target loss function, n represents the number of abnormal types, k i Represents the weight coefficient of the i-th abnormal type, y i represents the true probability that the sample image has the i-th abnormal type, It indicates the predicted probability that the sample image has the i-th abnormal type.

3. The device abnormality monitoring method according to claim 1, characterized in that: After the sample image is learned through the backbone network shared representation, the extracted features are respectively input into each of the task feature towers, including: The backbone network includes a shared layer constructed by a convolutional layer and a pooling layer, and the shared layer extracts common features that characterize the abnormality type and inputs them into each of the task feature towers respectively.

4. The device abnormality monitoring method according to claim 3, characterized in that: The task feature tower processes and outputs the feature recognition results of the anomaly type that each task is responsible for, including: Each of the task feature towers includes a fully connected layer for processing feature identification of the abnormal type it is responsible for. The fully connected layer identifies the features of the abnormal type it is responsible for from the general features and outputs the feature identification result.

5. The device abnormality monitoring method according to claim 2, characterized in that: The weight coefficient calculation formula of the i-th abnormal category is as follows: Wherein, m represents the total number of sample images, n i Indicates the number of sample images corresponding to the i-th abnormal type.

6. The device abnormality monitoring method according to any one of claims 1 to 5, characterized in that: The step of obtaining the image to be detected of the device to be tested comprises: Acquire an initial image to be detected of the device to be tested; The size of the initial image to be detected is adjusted to a preset size, and denoising is performed to obtain the image to be detected.

7. The device abnormality monitoring method according to claim 1, characterized in that: The outputting the abnormal result of the device under test includes: If the abnormal result of the device under test is that an abnormality exists, all abnormality types are output.

8. A device for monitoring equipment abnormality, characterized in that: Including acquisition module and monitoring module; The acquisition module is used to obtain the image to be tested of the device to be tested; The monitoring module is used to input the image to be detected into the trained monitoring model and output the abnormal result of the device to be tested; Among them, the monitoring model includes a backbone network and multiple independent task feature towers. When the monitoring model is trained, the sample image is shared by the backbone network for representation learning, and the extracted features are respectively input into each of the task feature towers. The task feature towers process and output the feature recognition results of the abnormality type they are responsible for. During training, the target loss function of the monitoring model is constructed based on the true probability and predicted probability that the sample image belongs to each of the abnormality types, as well as the weight coefficient of each of the abnormality types.

9. A device abnormality monitoring device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store computer programs; The processor is used to read the computer program in the memory and execute the steps of the device abnormality monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A readable computer program is stored thereon, and when the program is executed by a processor, the steps of the device abnormality monitoring method as described in any one of claims 1 to 7 are implemented.

Images