A fault type determination method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a fault type determination method and device, electronic equipment, readable storage medium and computer program product, and relates to the field of computer vision. The specific implementation scheme is: in the process of feature extraction of a to-be-detected image, adjusting the region weight of different image regions to obtain image features of the to-be-detected image, the to-be-detected image being an image including a target object; using the image features, determining a target fault type corresponding to the target object. This scheme can obtain the image features of the to-be-detected image including the target object, and use the image features to determine the target fault type corresponding to the target object. Thus, the automatic detection of the fault type of the target object is realized, and the detection efficiency of the fault type is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to computer vision and image processing technologies, which can be used in scenarios such as image processing. Background Technology

[0002] With the rapid development of internet and computer technologies, an increasing number of terminal devices are being manufactured to perform various functions. Simultaneously, a large amount of operational support equipment for these devices is also being manufactured to provide maintenance and support, ensuring their normal operation. To ensure the reliability of this operational support equipment, regular fault detection is necessary. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, readable storage medium, and computer program product for determining fault types, thereby improving the detection efficiency of fault detection.

[0004] According to one aspect of this disclosure, a method for determining a fault type is provided, the method comprising the following steps:

[0005] During the feature extraction process of the image to be detected, the region weights of different image regions are adjusted to obtain the image features of the image to be detected, which is an image that includes the target object.

[0006] By utilizing image features, the target fault type corresponding to the target object can be determined.

[0007] According to a second aspect of this disclosure, a fault type determination apparatus is provided, the apparatus comprising:

[0008] The image feature extraction unit is used to adjust the region weights of different image regions during the feature extraction process of the image to be detected, so as to obtain the image features of the image to be detected, which is an image including the target object.

[0009] The target fault type determination unit is used to determine the target fault type corresponding to the target object by utilizing image features.

[0010] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0011] At least one processor; and

[0012] The memory is communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods in any embodiment of this disclosure.

[0014] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods of any embodiment of this disclosure.

[0015] According to another aspect of this disclosure, a computer program product is provided, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the methods in any embodiment of this disclosure.

[0016] The technology disclosed herein can obtain image features of an image to be detected, including the target object, and use these image features to determine the target fault type corresponding to the target object. This achieves automated detection of the fault type of the target object, thereby improving the efficiency of fault type detection.

[0017] In addition, since the fault types of the target object are detected automatically, there is no need for maintenance personnel to manually detect faults in the target object, which ensures the safety of maintenance personnel and reduces the labor cost of fault detection.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0020] Figure 1 A flowchart of a method for determining a fault type is provided for embodiments of this disclosure;

[0021] Figure 2 A flowchart illustrating a method for obtaining image features provided in an embodiment of this disclosure;

[0022] Figure 3 A flowchart illustrating a method for determining a target fault type provided in an embodiment of this disclosure;

[0023] Figure 4 A flowchart illustrating a probability determination method provided in an embodiment of this disclosure;

[0024] Figure 5 A flowchart illustrating another probability determination method provided in the embodiments of this disclosure;

[0025] Figure 6 This is a schematic diagram illustrating a fault type determination process provided in an embodiment of this disclosure;

[0026] Figure 7 A schematic diagram of a fault type determination device provided for embodiments of this disclosure;

[0027] Figure 8 This is a schematic diagram of an electronic device provided for an embodiment of the present disclosure. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] To ensure the reliability of operational support equipment, regular fault detection is necessary during operation. Taking batteries, which ensure the normal operation of server and other terminal devices during power outages, as an example, the internal chemical reactions of the battery may become unstable over time. This can lead to problems such as broken vent valves, acid creep on the terminals, and copper plate corrosion, severely impacting the reliability of the battery's power supply. Therefore, fault detection of the battery is essential.

[0030] However, the current method for regularly detecting faults in operational support equipment is generally through manual fault detection by maintenance personnel. This method is not only inefficient, but also carries the risk of safety hazards arising from equipment malfunctions during the detection process.

[0031] To address the aforementioned issues, this disclosure provides a method for determining fault types; please refer to the following for details. Figure 1 This is a flowchart illustrating a method for determining a fault type, as provided in embodiments of this disclosure. The method may include the following steps:

[0032] Step S101: During the feature extraction process of the image to be detected, the region weights of different image regions are adjusted to obtain the image features of the image to be detected, which is an image including the target object.

[0033] Step S102: Use image features to determine the target fault type corresponding to the target object.

[0034] The fault type determination method provided in the embodiments of this disclosure can obtain image features of an image to be detected, including a target object, and use these image features to determine the target fault type corresponding to the target object. This achieves automated detection of the fault type of the target object, thereby improving the efficiency of fault type detection.

[0035] In addition, since the fault types of the target object are detected automatically, there is no need for maintenance personnel to manually detect faults in the target object, which not only ensures the safety of maintenance personnel, but also reduces the labor cost of fault detection.

[0036] In the embodiments of this disclosure, the so-called image to be detected is an image acquired targeting a target object. For example, an image targeting a target object acquired by a drone, or an image targeting a target object acquired by an image acquisition device preset in a designated area.

[0037] Region weight refers to the weight of visual perception of different image regions during the image feature extraction process.

[0038] The target equipment generally refers to the operational support equipment for terminal devices, including but not limited to batteries, solar panels, and lithium batteries. Among them, batteries are those that can be recharged through reversible chemical reactions, mainly referring to lead-acid batteries.

[0039] It should be noted that the embodiments of this disclosure do not specifically limit the operation support equipment. The operation support equipment can also be other equipment besides the battery, such as power station, power supply line, etc.

[0040] The following section uses a battery as an example to explain in detail the implementation steps of the fault type determination method. It should be noted that the implementation steps for determining the fault type of other support equipment besides the battery are the same as those for determining the fault type when the target object is a battery.

[0041] Before feature extraction from the image to be detected, image preprocessing can be performed to enrich the image information, reduce the pixel count, and decrease the workload for subsequent image processing. Image preprocessing methods include, but are not limited to, image rotation, image cropping, image incrementing, and image pixel reduction.

[0042] The term "fault type" refers to a pre-defined category of potential faults that may occur in a target object. Taking a storage battery as an example, battery fault types generally include, but are not limited to, broken vent valve, acid creep in the terminals, and copper plate corrosion. Furthermore, different target objects may correspond to different fault types.

[0043] In the embodiments of this disclosure, the specific implementation method for adjusting the region weights of different image regions to obtain the image features of the image to be detected is as follows: Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for obtaining image features according to an embodiment of the present disclosure. The method for obtaining image features is as follows:

[0044] Step S201: Perform the first feature extraction on the image to be detected, and reduce the region weight of other image regions during the first feature extraction process to obtain the first feature map of the image to be detected. Other image regions are the image regions in the image to be detected other than the target image region. The target image region is the image region in the image to be detected corresponding to the target object's region to be detected.

[0045] Step S202: Perform a second feature extraction on the first feature map, and increase the regional weight of the target image region during the second feature extraction process to obtain the second feature map of the image to be detected.

[0046] Step S203: Determine the second feature map as an image feature.

[0047] In the embodiments of this disclosure, by reducing the region weights of other image regions during the first feature extraction process and increasing the region weights of the target image region during the second feature extraction process, the second feature map can be focused on the target image region. Therefore, when the second feature map is determined as an image feature, the features of the region to be detected can be highlighted in this image feature. Based on this image feature, the target fault type corresponding to the target object can be accurately determined.

[0048] The area to be tested refers to a pre-defined area for the target object. For example, it could be a pre-defined area based on the prior knowledge of maintenance personnel. Specifically, taking a battery as the target object, battery fault types generally include, but are not limited to, a broken vent valve, acid creep in the terminals, and corrosion of the copper plates. In this case, the pre-defined area to be tested for the battery includes, but is not limited to, the area where the vent valve is located, the area where the terminals are located, and the area where the copper plates are located.

[0049] Alternatively, in the embodiments of this disclosure, image features can also be obtained in the following manner: First, the image to be detected is input into a trained region extraction model to obtain the target image region. The region extraction model is a pre-trained model based on image samples and corresponding labeled image regions. Then, feature extraction is performed on the target image region to obtain the target image features of the target image region. Finally, the target image features are determined as image features.

[0050] Obtaining target image features of the target image region as image features can make the image features more targeted and have fewer interfering features, thereby improving the accuracy of the identified target fault type.

[0051] In the embodiments of this disclosure, the specific implementation process of performing a first feature extraction on the image to be detected and reducing the regional weights of other image regions during the first feature extraction process is as follows: performing a first convolutional pooling process on the image to be detected and adjusting the weight value of the first convolutional kernel during the first convolutional pooling. The first convolutional kernel is a convolutional kernel used to reduce the regional weights of other image regions.

[0052] In the first convolutional pooling process, by adjusting the weight value of the first convolutional kernel, the first convolutional kernel can reduce the regional weights of other image regions, so that the obtained first feature map can highlight the features of the region to be detected.

[0053] In the embodiments of this disclosure, the specific implementation process of performing a second feature extraction on the first feature map and increasing the regional weight of the target image region during the second feature extraction process is as follows: performing a second convolutional pooling process on the first feature map and adjusting the weight value of the second convolution kernel during the second convolutional pooling. The second convolution kernel is a convolution kernel used to increase the regional weight of the target image region.

[0054] In the second convolutional pooling process, by adjusting the weight value of the second convolutional kernel, the second convolutional kernel can increase the regional weight of other image regions, thereby enabling the obtained second feature map to further highlight the features of the region to be detected.

[0055] In the embodiments of this disclosure, the specific steps for determining the target fault type corresponding to the target object are as follows: Figure 3 As shown, Figure 3 This is a flowchart illustrating a method for determining a target fault type according to an embodiment of this disclosure. The steps for determining the target fault type are as follows:

[0056] Step S301: Using the first fault classification model, obtain the first probability of the candidate fault type corresponding to the target object based on the image features.

[0057] Step S302: Using the second fault classification model, obtain the second probability of the candidate fault type corresponding to the target object based on the image features.

[0058] Step S303: Using the first probability and the second probability, determine the target probability of the candidate fault type corresponding to the target object.

[0059] Step S304: Determine the target fault type from the candidate fault types based on the target probability, wherein the fault classification model is a model pre-trained based on the probability of image feature samples and corresponding annotations.

[0060] For different image features, a first probability and a second probability are obtained respectively. Based on these probabilities, the target probability of the candidate fault type corresponding to the target object is determined. Based on the target probability, the target fault type is determined from the candidate fault types. This approach balances the contradiction between underfitting and overfitting in the fault classification model, thereby improving the accuracy of the target fault type.

[0061] It should be noted that the embodiments of this disclosure may also determine the target fault type using only one of the first fault classification model or the second fault classification model.

[0062] Specifically, taking the first fault classification model as an example, the steps to determine the target fault type based on the first fault classification model are as follows: First, using the first fault classification model, obtain the first probability of the candidate fault type corresponding to the target object based on the image features; determine the first probability as the target probability; and determine the target fault type from the candidate fault types according to the target probability.

[0063] In the embodiments of this disclosure, the fault classification model is generally a Softmax multi-classification model constructed based on the S-regression function (Softmax logical regression, Softmax).

[0064] In embodiments of this disclosure, the steps for determining the first probability are as follows: Figure 4 As shown, Figure 4 This is a flowchart illustrating a probability determination method provided in an embodiment of this disclosure. The steps of the probability determination method are as follows:

[0065] Step S401: Perform feature extraction on the second feature map at multiple preset scales to obtain the third feature map corresponding to the image to be detected at multiple scales.

[0066] Step S402: Fuse multiple third feature maps to obtain the fourth feature map of the image to be detected.

[0067] Step S403: Input the fourth feature map into the first fault classification model to obtain the first probability.

[0068] After obtaining the third feature maps corresponding to the image to be detected at multiple scales, these third feature maps are then fused to obtain the fourth feature map of the image to be detected. This allows for the compression of feature dimensions and reduces the computational load of subsequent data processing, while ensuring that the fourth feature map provides a good feature representation of the image to be detected. The above process achieves the effect of improving the detection efficiency while ensuring the accuracy of fault detection.

[0069] The method for extracting features from the second feature map at multiple preset scales and fusing multiple third feature maps is as follows: Multi-scale feature extraction and feature fusion are achieved using a residual series network (Inception-ResNet, Inception). Here, the multiple preset scales are 1*1, 3*3, and 5*5.

[0070] Taking the target object as a storage battery as an example, the fourth feature map is input into the first fault classification model, and the first probability obtained may be as follows: the probability of the fault type being exhaust valve breakage is 0.34, the probability of the fault type being terminal acid creep is 0.36, the probability of the fault type being copper plate corrosion is 0.2, and the probability of the fault type being no fault is 0.1.

[0071] In embodiments of this disclosure, the steps for determining the second probability are as follows: Figure 5 As shown, Figure 5 This is a flowchart of another probability determination method provided in an embodiment of this disclosure. The steps of this probability determination method are as follows:

[0072] Step S501: Perform feature extraction on the fourth feature map at multiple preset scales to obtain the fifth feature map corresponding to the image to be detected at multiple scales.

[0073] Step S502: Fuse multiple fifth feature maps to obtain the sixth feature map of the image to be detected.

[0074] Step S503: Input the sixth feature map into the second fault classification model to obtain the second probability.

[0075] After obtaining the fifth feature maps corresponding to the image to be detected at multiple scales, these fifth feature maps are then fused to obtain the sixth feature map. This allows for further compression of the feature dimension, reducing the computational load of subsequent data processing, while ensuring that the sixth feature map provides a good feature representation of the image to be detected. The above process achieves the effect of further improving the detection efficiency while ensuring the accuracy of fault detection.

[0076] The implementation method for extracting features from the fourth feature map at multiple preset scales and fusing multiple fifth feature maps is as follows: Multi-scale feature extraction and feature fusion are achieved based on a residual series network (Inception-ResNet). Here, the multiple preset scales are 1*1, 3*3, and 5*5.

[0077] Taking the target object as a storage battery as an example, the fifth feature map is input into the second fault classification model, and the second probability obtained may be: the probability of the fault type being exhaust valve breakage is 0.5, the probability of the fault type being post acid creep is 0.3, the probability of the fault type being copper plate corrosion is 0.18, and the probability of the fault type being no fault is 0.02.

[0078] In embodiments of this disclosure, the step of determining the target probability using a first probability and a second probability includes adding the first probability and the second probability to obtain the target probability. In this way, the target probability has a better probability differentiation effect on the probability of the candidate fault type corresponding to the target object, thereby improving the accuracy of the target fault type.

[0079] Specifically, for the first probability, the probability of the fault type being exhaust valve breakage is 0.34, and the probability of the fault type being electrode post acid creep is 0.36. Since the probabilities of exhaust valve breakage and electrode post acid creep are quite close, using electrode post acid creep as the target fault type may lead to an incorrect target fault type.

[0080] Adding the first probability to the second probability yields the following target probabilities: the probability of the fault type being exhaust valve breakage is 0.84, the probability of the fault type being electrode column acid creep is 0.66, the probability of the fault type being copper plate corrosion is 0.38, and the probability of the fault type being no fault occurs is 0.12. Therefore, based on the target probabilities, the target fault type determined from the candidate fault types is exhaust valve breakage.

[0081] The fault type determination method provided in the embodiments of this disclosure, in which the region weights are adjusted by adjusting the weight values ​​of the convolution kernels, the Inception network is used to achieve multi-scale feature extraction and feature fusion, and the Softmax multi-classification model is used to determine the target probability, the specific implementation process is as follows: Figure 6 As shown. Figure 6 This is a schematic diagram illustrating a fault type determination process provided in an embodiment of this disclosure. The residual series network 1 is an Inception network used for "extracting features from the second feature map at multiple preset scales to obtain third feature maps corresponding to the image to be detected at multiple scales; fusing multiple third feature maps to obtain a fourth feature map of the image to be detected"; the residual series network 2 is an Inception network used for "extracting features from the fourth feature map at multiple preset scales to obtain fifth feature maps corresponding to the image to be detected at multiple scales; fusing multiple fifth feature maps to obtain a sixth feature map of the image to be detected"; the Softmax multi-classification model 1 is a first fault classification model; and the Softmax multi-classification model 2 is a second fault classification model. Figure 6The process for determining the fault type shown is as follows:

[0082] First, perform image preprocessing operations such as image rotation, image cropping, image incrementing, and image pixel reduction on the image to be processed to obtain the grayscale image corresponding to the image to be processed.

[0083] Second, the grayscale image undergoes a first convolutional pooling process to obtain a first feature map. In the first convolutional pooling, the weight values ​​of the first convolutional kernel need to be adjusted to reduce the regional weights of other image regions outside the image region where the battery is located in the image to be detected, so that the first feature map highlights the image features of the image region where the battery is located.

[0084] Third, a second convolutional pooling process is performed on the first feature map to obtain the second feature map. In the second convolutional pooling, the weight values ​​of the second convolution kernel need to be adjusted to increase the regional weight of the image region where the battery is located in the image to be detected, so that the second feature map further highlights the image features of the image region where the battery is located.

[0085] Fourth, the second feature map is input into the residual series network 1 to obtain the fourth feature map, and the fourth feature map is input into the residual series network 2 and the Softmax multi-classification model 1 respectively.

[0086] Fifth, after obtaining the fourth feature map, the Softmax multi-classification model 1 will output the first probability for that fourth feature map. For example, the probability of the fault type being exhaust valve breakage is 0.34, the probability of the fault type being acid creep on the electrode column is 0.36, the probability of the fault type being copper plate corrosion is 0.2, and the probability of the fault type being no fault is 0.1.

[0087] Sixth, after obtaining the fourth feature map, the residual series network 2 will output the sixth feature map based on the fourth feature map and input the sixth feature map into the Softmax multi-classification model 2.

[0088] Seventh, after obtaining the sixth feature map, the Softmax multi-classification model 2 will output a second probability for that sixth feature map. For example, the probability of the fault type being exhaust valve breakage is 0.5, the probability of the fault type being electrode column acid creep is 0.3, the probability of the fault type being copper plate corrosion is 0.18, and the probability of the fault type being no fault is 0.02.

[0089] Eighth, after obtaining the first probability and the second probability, add the first probability and the second probability together to obtain the target probability. For example: the probability of the fault type being exhaust valve breakage is 0.84, the probability of the fault type being electrode column acid creep is 0.66, the probability of the fault type being copper plate corrosion is 0.38, and the probability of the fault type being no fault is 0.12.

[0090] Ninth, select the target probability with the highest probability value, and determine the candidate fault type corresponding to the target probability with the highest probability value as the target fault type. That is, determine the target fault type as exhaust valve failure.

[0091] Tenth, output and display the target fault type.

[0092] Because a large number of batteries are often required to provide operational support for servers and other terminals, and these batteries can release harmful gases and leak current when energized, posing safety hazards, there is a strong need for automated fault detection for batteries. Therefore, the target object in the fault type determination method provided in the embodiments of this disclosure is often a battery. The steps for determining the target fault type corresponding to the target object are as follows: first, determine the battery fault type corresponding to the battery; then, determine the battery fault type as the target fault type.

[0093] like Figure 7 As shown, embodiments of this disclosure provide a fault type determination apparatus, the apparatus comprising:

[0094] The image feature extraction unit 701 is used to adjust the region weights of different image regions during the feature extraction process of the image to be detected, so as to obtain the image features of the image to be detected, wherein the image to be detected is an image including the target object.

[0095] The target fault type determination unit 702 is used to determine the target fault type corresponding to the target object by utilizing image features.

[0096] In one embodiment, the image feature extraction unit 701 may further include:

[0097] The first feature extraction subunit is used to perform the first feature extraction on the image to be detected, and reduce the region weight of other image regions during the first feature extraction process to obtain the first feature map of the image to be detected. Other image regions are image regions in the image to be detected other than the target image region. The target image region is the image region in the image to be detected corresponding to the target object's region to be detected in the image to be detected.

[0098] The second feature extraction subunit is used to perform a second feature extraction on the first feature map, and to add the region weight of the target image region during the second feature extraction process to obtain the second feature map of the image to be detected;

[0099] The image feature determination subunit is used to determine the second feature map as an image feature.

[0100] In one implementation, the first feature extraction subunit may further include:

[0101] The first convolutional pooling processing subunit is used to perform the first convolutional pooling processing on the image to be detected, and to adjust the weight value of the first convolutional kernel in the first convolutional pooling. The first convolutional kernel is a convolutional kernel used to reduce the regional weights of other image regions.

[0102] In one implementation, the second feature extraction subunit may further include:

[0103] The second convolutional pooling processing subunit is used to perform a second convolutional pooling process on the first feature map, and adjust the weight value of the second convolutional kernel in the second convolutional pooling. The second convolutional kernel is a convolutional kernel used to increase the regional weight of the target image region.

[0104] In one embodiment, the target fault type determination unit 702 may further include:

[0105] The first probability acquisition subunit is used to obtain the first probability of the candidate fault type corresponding to the target object based on the image features using the first fault classification model.

[0106] The second probability acquisition subunit is used to obtain the second probability of the candidate fault type corresponding to the target object based on the image features using the second fault classification model.

[0107] The target probability acquisition subunit is used to determine the target probability of the candidate fault type corresponding to the target object using the first probability and the second probability;

[0108] The target fault type determination subunit is used to determine the target fault type from the candidate fault types based on the target probability.

[0109] The fault classification model is a model pre-trained based on the probability of image feature samples and corresponding annotations.

[0110] In one implementation, the first probability acquisition subunit may further include:

[0111] The third feature map acquisition subunit is used to extract features from the second feature map at multiple preset scales to obtain the third feature map corresponding to the image to be detected at multiple scales respectively.

[0112] The fourth feature map acquisition subunit is used to fuse multiple third feature maps to obtain the fourth feature map of the image to be detected;

[0113] The first fault classification model subunit is used to input the fourth feature map into the first fault classification model to obtain the first probability.

[0114] In one implementation, the second probability-obtaining subunit may further include:

[0115] The fifth feature map acquisition subunit is used to extract features from the fourth feature map at multiple preset scales to obtain the fifth feature map of the image to be detected at multiple scales respectively.

[0116] The sixth feature map acquisition subunit is used to fuse multiple fifth feature maps to obtain the sixth feature map of the image to be detected;

[0117] The second fault classification model subunit is used to input the sixth feature map into the second fault classification model to obtain the second probability.

[0118] In one implementation, the target fault type determination subunit may further include:

[0119] The target probability calculation subunit is used to add the first probability and the second probability to obtain the target probability.

[0120] In one embodiment, when the target object includes a battery, the target fault type determination unit 702 may further include:

[0121] The battery fault type determination subunit is used to determine the corresponding battery fault type.

[0122] The fault type determination subunit is used to determine the battery fault type as the target fault type.

[0123] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0124] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0125] Figure 8 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0126] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0127] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the fault type determination method. For example, in some embodiments, the fault type determination method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the fault type determination method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the fault type determination method by any other suitable means (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable fault-determining device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0133] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0134] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0135] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0136] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining a fault type, comprising: In the process of feature extraction of the image to be detected, the region weights of different image regions are adjusted to obtain the image features of the image to be detected, wherein the image to be detected is an image including a target object; the process includes: performing a first feature extraction on the image to be detected, and reducing the region weights of other image regions during the first feature extraction to obtain a first feature map of the image to be detected; wherein the target object is the operation support equipment of the terminal device; wherein the other image regions are image regions in the image to be detected other than the target image region, and the target image region is the image region in the image to be detected corresponding to the target object's detection region; performing a second feature extraction on the first feature map, and increasing the region weights of the target image region during the second feature extraction to obtain a second feature map of the image to be detected; and determining the second feature map as the image feature; The image features are processed using a first residual series network to obtain a fourth feature map; The fourth feature map is input into the first fault classification model to obtain the first probability of the candidate fault type; The fourth feature map is processed using the second residual series network to obtain the sixth feature map; The sixth feature map is input into the second fault classification model to obtain the second probability of the candidate fault type; Using the first probability and the second probability, determine the target probability of the target object corresponding to the candidate fault type; Based on the target probability, a target fault type is determined from the candidate fault types; if the target object includes a battery, the target fault type includes a battery fault type; the battery fault type includes at least one of the following: exhaust valve breakage, terminal acid creep, and copper plate corrosion.

2. The method according to claim 1, wherein, The first feature extraction of the image to be detected, and the reduction of the region weights of other image regions during the first feature extraction process, includes: The image to be detected is subjected to a first convolutional pooling process, and the weight value of the first convolutional kernel is adjusted in the first convolutional pooling. The first convolutional kernel is a convolutional kernel used to reduce the weight of the region in the other image regions.

3. The method according to claim 1 or 2, wherein, The second feature extraction of the first feature map, and the addition of the region weight of the target image region during the second feature extraction process, includes: The first feature map is subjected to a second convolutional pooling process, and the weight value of the second convolutional kernel is adjusted in the second convolutional pooling. The second convolutional kernel is a convolutional kernel used to increase the weight of the target image region.

4. The method according to claim 1 or 2, wherein, The first fault classification model and the second fault classification model are models pre-trained based on the probability of image feature samples and corresponding annotations.

5. The method according to claim 1 or 2, wherein, The steps for determining the fourth feature map include: The second feature map is subjected to feature extraction at multiple preset scales to obtain the third feature map of the image to be detected at multiple scales respectively. The third feature maps are fused together to obtain the fourth feature map of the image to be detected.

6. The method according to claim 1 or 2, wherein, The steps for determining the sixth feature map include: The fourth feature map is subjected to feature extraction at multiple preset scales to obtain the fifth feature map of the image to be detected at multiple scales respectively. The sixth feature map of the image to be detected is obtained by fusing multiple fifth feature maps.

7. The method according to claim 1, wherein, Determining the target probability of the target object corresponding to the candidate fault type using the first probability and the second probability includes: The first probability is added to the second probability to obtain the target probability.

8. A fault type determination device, comprising: An image feature extraction unit is used to adjust the region weights of different image regions during the feature extraction process of the image to be detected, thereby obtaining the image features of the image to be detected, wherein the image to be detected is an image including a target object; it includes: a first feature extraction subunit, used to perform a first feature extraction on the image to be detected, and reduce the region weights of other image regions during the first feature extraction process to obtain a first feature map of the image to be detected, wherein the target object is the operation support equipment of the terminal device; the other image regions are image regions in the image to be detected other than the target image region, and the target image region is the image region in the image to be detected corresponding to the target object's detection region in the image to be detected; The second feature extraction subunit is used to perform a second feature extraction on the first feature map, and to increase the region weight of the target image region during the second feature extraction process to obtain the second feature map of the image to be detected; An image feature determination subunit is used to determine the second feature map as the image feature; A target fault type determination unit is configured to process the image features using a first residual series network to obtain a fourth feature map; input the fourth feature map into a first fault classification model to obtain a first probability of a candidate fault type; process the fourth feature map using a second residual series network to obtain a sixth feature map; input the sixth feature map into a second fault classification model to obtain a second probability of a candidate fault type; determine the target probability of the target object corresponding to the candidate fault type using the first probability and the second probability; determine the target fault type among the candidate fault types based on the target probability; when the target object includes a battery, the target fault type includes a battery fault type; the battery fault type includes at least one of the following: exhaust valve breakage, terminal acid creep, and copper plate corrosion.

9. The apparatus according to claim 8, wherein, The first feature extraction subunit includes: The first convolutional pooling processing subunit is used to perform a first convolutional pooling process on the image to be detected, and to adjust the weight value of the first convolutional kernel in the first convolutional pooling. The first convolutional kernel is a convolutional kernel used to reduce the weight of the region in the other image regions.

10. The apparatus according to claim 8 or 9, wherein, The second feature extraction subunit includes: The second convolutional pooling processing subunit is used to perform a second convolutional pooling process on the first feature map, and adjust the weight value of the second convolutional kernel in the second convolutional pooling. The second convolutional kernel is a convolutional kernel used to increase the weight of the target image region.

11. The apparatus according to claim 8 or 9, wherein, The first fault classification model and the second fault classification model are models pre-trained based on the probability of image feature samples and corresponding annotations.

12. The apparatus according to claim 8 or 9, wherein, The target fault type determination unit includes: The third feature map acquisition subunit is used to perform feature extraction on the second feature map at multiple preset scales to obtain the third feature map corresponding to the image to be detected at multiple scales respectively. The fourth feature map obtaining subunit is used to fuse multiple third feature maps to obtain the fourth feature map of the image to be detected; The first fault classification model subunit is used to input the fourth feature map into the first fault classification model to obtain the first probability.

13. The apparatus according to claim 8 or 9, wherein, The target fault type determination unit includes: The fifth feature map acquisition subunit is used to perform feature extraction on the fourth feature map at multiple preset scales to obtain the fifth feature map of the image to be detected at multiple scales respectively. The sixth feature map obtaining subunit is used to fuse multiple fifth feature maps to obtain the sixth feature map of the image to be detected; The second fault classification model subunit is used to input the sixth feature map into the second fault classification model to obtain the second probability.

14. The apparatus according to claim 8, wherein, The target fault type determination subunit includes: The target probability calculation subunit is used to add the first probability and the second probability to obtain the target probability.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method described in any one of claims 1 to 7.

17. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images