Fault processing method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a fault processing method and device, computer equipment and a computer readable storage medium. The method comprises the steps that fault description information and a fault image of fault equipment are acquired; performing semantic analysis on the fault description information to obtain fault type information; identifying the fault image to obtain an image identification result; and determining fault processing result information based on the fault type information and the image recognition result. By adopting the method provided by the invention, the timeliness of fault processing can be improved, and the labor and maintenance cost of an equipment operator can be reduced.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a fault handling method, apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] Smart home appliances use intelligent technologies such as the Internet of Things and information communication technologies to simplify and optimize user service processes, and can provide users with more convenient user services. Smart home appliances will inevitably malfunction during use. The existing fault handling method for smart home appliances is usually for the user to contact the equipment operator, and then the equipment operator assigns a maintenance specialist to handle the fault. This fault handling method has the problem of untimely fault handling. Summary of the invention

[0003] The embodiments of the present application provide a fault handling method, apparatus, computer equipment, and computer-readable storage medium, which can improve the timeliness of fault handling and reduce the labor and maintenance costs of equipment operators.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] On the one hand, the present application provides a fault handling method, comprising:

[0006] Obtain fault description information and fault image of the faulty device;

[0007] Perform semantic analysis on the fault description information to obtain fault type information;

[0008] Identify the fault image and obtain the image recognition result;

[0009] Based on the fault type information and the image recognition result, the fault handling result information is determined.

[0010] In some implementation schemes of the present application, semantic analysis is performed on the fault description information to obtain fault type information, including:

[0011] Preprocessing the fault description information to obtain coding information of the fault description information;

[0012] Extract features from the coded information to obtain semantic feature information of the fault description information;

[0013] Based on the semantic feature information, the fault type information is determined.

[0014] In some implementation schemes of the present application, feature extraction is performed on the coded information to obtain semantic feature information of the fault description information, including:

[0015] Performing first information extraction on the coded information to obtain first feature information of each word in the fault description information;

[0016] Performing second information extraction on the coded information to obtain second feature information of each word in the fault description information;

[0017] Performing a first self-attention operation on the first feature information to obtain first attention information of each word in the fault description information;

[0018] Perform a second self-attention operation on the second feature information to obtain second attention information of each word in the fault description information;

[0019] splicing the first attention information and the second attention information to obtain spliced ​​information;

[0020] The spliced ​​information is fused to obtain the semantic feature information of the fault description information.

[0021] In some embodiments of the present application, the fault image is identified to obtain an image recognition result, including:

[0022] Perform pixel value conversion on the fault image to obtain pixel value information of the fault image;

[0023] Performing a first convolution operation on the pixel value information to obtain a first convolution feature;

[0024] Perform feature extraction on the first convolution feature to obtain third feature information;

[0025] Performing feature enhancement on the third feature information to obtain fourth feature information;

[0026] Performing dimensionality reduction and pooling processing on the fourth feature information to obtain fifth feature information;

[0027] A second convolution operation is performed on the fifth feature information to obtain an image recognition result.

[0028] In some embodiments of the present application, feature extraction is performed on the first convolution feature to obtain third feature information, including:

[0029] Performing dimension-upgrading processing on the first convolution feature to obtain a first dimension-upgrading feature;

[0030] Performing pooling processing on the first dimension-raising feature to obtain a first pooling feature;

[0031] Performing a third convolution operation on the first dimension-increased feature to obtain a second convolution feature, and performing a fourth convolution operation on the first dimension-increased feature to obtain a third convolution feature;

[0032] The first pooling feature, the second convolution feature and the third convolution feature are fused to obtain a first fused feature;

[0033] Performing dimensionality reduction processing on the first fusion feature to obtain a first dimensionality reduction feature;

[0034] The first dimensionality reduction feature and the first convolution feature are fused to obtain the third feature information.

[0035] In some implementation schemes of the present application, the third feature information is enhanced to obtain the fourth feature information, including:

[0036] Performing dimension-upgrading processing on the third feature information to obtain a second dimension-upgraded feature;

[0037] Performing pooling processing on the second dimension-raising feature to obtain a second pooling feature;

[0038] Performing a fifth convolution operation on the second dimensionally increased feature to obtain a fourth convolution feature, and performing a sixth convolution operation on the second dimensionally increased feature to obtain a fifth convolution feature;

[0039] The second pooling feature, the fourth convolution feature and the fifth convolution feature are fused to obtain a second fused feature;

[0040] Pooling and activation processing are performed on the second fusion feature to obtain weight information;

[0041] Processing the second fused feature based on the weight information to obtain a third fused feature;

[0042] Performing dimensionality reduction processing on the third fusion feature to obtain a second dimensionality reduction feature;

[0043] The second dimension reduction feature and the third feature information are fused to obtain the fourth feature information.

[0044] In some implementation schemes of the present application, based on the fault type information and the image recognition result, the fault handling result information is determined, including:

[0045] Matching the fault type information with the image recognition results;

[0046] When the fault type information matches the image recognition result, determining fault handling information based on the fault type information;

[0047] Generate fault operation instructions based on fault handling information;

[0048] The fault operation instruction is sent to the faulty device so that the faulty device executes the fault operation instruction.

[0049] In a second aspect, an embodiment of the present invention further provides a fault handling device, including:

[0050] An information acquisition unit, used to acquire fault description information and fault image of the faulty device;

[0051] A semantic analysis unit, used to perform semantic analysis on the fault description information to obtain fault type information;

[0052] An image recognition unit, used to recognize the fault image and obtain an image recognition result;

[0053] The fault processing unit is used to determine fault processing result information based on the fault type information and the image recognition result.

[0054] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0055] one or more processors;

[0056] Memory; and

[0057] One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by a processor to implement any fault handling method in the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the fault handling methods in the first aspect.

[0059] The beneficial effects of the present invention are as follows: semantic analysis is performed on fault description information to obtain fault type information, fault images are identified to obtain image recognition results, and fault handling result information is determined based on the fault type information and image recognition results. Users only need to input fault description information and fault images to achieve automatic fault handling, and there is no need for equipment operators to assign maintenance specialists to handle the fault. This can improve the timeliness of fault handling and reduce the labor and maintenance costs of equipment operators. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 is a flowchart of a fault handling method provided by an embodiment of the present invention;

[0062] Figure 2 is a flowchart of a specific embodiment of the fault handling method provided by an embodiment of the present invention;

[0063] Figure 3 is a schematic diagram of the structure of a description and analysis module provided in an embodiment of the present invention;

[0064] Figure 4 is a flowchart of a specific embodiment of feature extraction of coded information provided by an embodiment of the present invention;

[0065] Figure 5 is a structural schematic diagram of an image recognition module provided by an embodiment of the present invention;

[0066] Figure 6 is a flowchart of a specific embodiment of extracting features from the first convolution feature provided by an embodiment of the present invention;

[0067] Figure 7 is a schematic structural diagram of a first feature extraction unit provided in an embodiment of the present invention;

[0068] Figure 8 is a flowchart of a specific embodiment of enhancing the feature of the third feature information provided by an embodiment of the present invention;

[0069] Fig. 9 is a schematic diagram of the structure of a second feature extraction unit provided in an embodiment of the present invention;

[0070] Fig.10 is a principle block diagram of a fault handling device provided by an embodiment of the present invention;

[0071] Fig.11 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0073] In the description of the present application, the terms "first", "second", "third", "fourth", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", "third", "fourth", etc. may explicitly or implicitly include one or more features. In the description of the present application, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0074] The inventor has found through research that it is inevitable that smart home appliances will malfunction during use. Taking the smart campus washing machine as an example, it is inevitable that the smart campus washing machine will have some malfunctions due to excessive use and concentrated use time. Common malfunctions include being in a dehydration state without stopping, foam residue, dehydration function failure, and no water intake after the washing machine is started. The existing fault handling method for smart home appliances is usually that the user contacts the equipment operator, and then the equipment operator assigns a maintenance specialist to handle the fault. The above-mentioned existing fault handling method for smart home appliances has at least the following two problems: ① It takes time to contact the equipment operator and the equipment operator assigns a maintenance specialist to handle the fault, so the fault cannot be solved in time; ② Even very small faults require the equipment operator to assign a maintenance specialist to handle the fault, and the equipment operator has high labor and maintenance costs.

[0075] Based on this, in an embodiment of the present application, fault description information and fault image of the faulty equipment are obtained; semantic analysis is performed on the fault description information to obtain fault type information; the fault image is identified to obtain an image recognition result; based on the fault type information and the image recognition result, the fault handling result information is determined, which can improve the timeliness of fault handling and reduce the labor and maintenance costs of equipment operators.

[0076] The content of this application is further illustrated below through the description of embodiments in conjunction with the accompanying drawings.

[0077] This embodiment provides a fault handling method, such as Figure 1 As shown in , the method includes:

[0078] Step S10: Obtain fault description information and fault image of the faulty device.

[0079] The faulty device is a faulty smart home appliance, that is, a smart home appliance that needs to be troubleshooted. For example, the faulty device may be a faulty smart campus washing machine, television, refrigerator, etc. The fault description information is the description information related to the fault of the faulty device. For example, when the faulty device is a smart campus washing machine, the fault description information may be "always in the dehydration state without stopping", "foam residue", "dehydration function failure", "no water after the washing machine is started", etc.

[0080] The fault image is an image related to the fault of the faulty device. Specifically, when the fault handling method is applied to a cloud server, the fault image can be an image obtained by a user taking a photo of the fault of the faulty device through an imaging module configured by a user terminal (such as a smart phone). The fault description information can be a fault description sentence input by the user through the user terminal. The user can then upload the fault description information and the fault image to the cloud server through the user terminal. The cloud server can then obtain the fault description information and the fault image of the faulty device, and perform fault handling based on the acquired fault description information and the fault image.

[0081] Step S20: perform semantic analysis on the fault description information to obtain fault type information.

[0082] The fault type information is the fault type of the faulty device obtained by performing a semantic analysis on the fault description information. For example, when the fault description information is "always in the dehydration state and does not stop", the fault type information is "01"; when the fault description information is "foam residue", the fault type information is "02"; when the fault description information is "dehydration function failure", the fault type information is "03"; when the fault description information is "no water is added after the washing machine is started", the fault type information is "04".

[0083] In a specific implementation, the fault handling method is applied to a fault handling model, the fault handling model includes a description analysis module, and the step of performing semantic analysis on the fault description information specifically includes: inputting the fault description information into the description analysis module, performing semantic analysis on the fault description information through the description analysis module, and obtaining fault type information.

[0084] In a specific implementation, Figure 2 As shown, step S20 includes:

[0085] Step S21, preprocessing the fault description information to obtain coding information of the fault description information;

[0086] Step S22: extracting features from the coded information to obtain semantic feature information of the fault description information;

[0087] Step S23: Determine the fault type information based on the semantic feature information.

[0088] Preprocessing includes word segmentation and encoding processing of the fault description information. Accordingly, the steps of preprocessing the fault description information specifically include: word segmentation processing of the fault description information to obtain the fault description information after word segmentation processing; encoding processing of the fault description information after word segmentation processing to obtain the encoding information of the fault description information.

[0089] Specifically, the fault description information after word segmentation can be expressed as: X = [CLS] x1x2…x n [SEP], where X represents the fault description information after word segmentation, [CLS] represents the token at the beginning of the text sequence, and [SEP] represents the separator between text sequences. n Indicates the nth word in the fault description information. For example, a word segmenter WordPiece may be used to segment the fault description information to obtain the fault description information after segmentation.

[0090] Furthermore, the encoding information is the encoding information obtained by encoding the fault description information after word segmentation processing, and the steps of encoding the fault description information after word segmentation processing specifically include: mapping the fault description information after word segmentation processing based on a preset word vector matrix to obtain a first feature code; mapping the fault description information after word segmentation processing based on a preset block vector matrix to obtain a second feature code; mapping the fault description information after word segmentation processing based on a preset position vector matrix to obtain a third feature code; combining the first feature code, the second feature code and the third feature code to obtain the encoding information of the fault description information.

[0091] For example, a pre-trained Bidirectional Encoder Representations from Transformer (BERT) model can be used to encode the fault description information after word segmentation to obtain the encoded information of the fault description information. Figure 3 As shown in FIG. 1 , when the BERT model is used to encode the fault description information after word segmentation, the BERT model maps the preprocessed fault description information based on the word vector matrix, the block vector matrix and the position vector matrix to obtain the first feature code p0p1p2…p n p n+1 、The second feature code is e0e1e2…e n e n+1 And the third feature code v [CLS] v1v2…v n v [SEP] .

[0092] After obtaining the coding information of the fault description information, feature extraction can be performed on the coding information to obtain semantic feature information of the fault description information, and then the fault type information is determined based on the semantic feature information. Figure 3As shown, the description analysis module includes a first fully connected layer and a first activation layer, and the step of determining the fault type information based on the semantic feature information specifically includes: inputting the semantic feature information into the first fully connected layer, and outputting the hidden layer features of the fault description information under multiple preset fault categories through the first fully connected layer; inputting the hidden layer features into the first activation layer, and outputting the probability of the fault description information under multiple fault categories through the first activation layer; and determining the fault category corresponding to the maximum probability as the fault category information. For example, the first activation layer outputs the probabilities of fault categories 01, 02, 03, and 04 as 0.8, 0.1, 0.05, and 0.05, respectively, and the fault category information is determined to be 01.

[0093] Specifically, the probability calculation formula is: p i represents the probability of the i-th fault category, m i represents the hidden layer features under the i-th fault category, m j represents the hidden layer features under the jth fault category, and n represents the number of fault categories.

[0094] In a specific implementation, Figure 4 As shown, step S22 includes:

[0095] Step S221: extract first information from the coded information to obtain first feature information of each word in the fault description information;

[0096] Step S222: extract the second information from the coded information to obtain the second feature information of each word in the fault description information;

[0097] Step S223: performing a first self-attention operation on the first feature information to obtain first attention information of each word in the fault description information;

[0098] Step S224: performing a second self-attention operation on the second feature information to obtain second attention information of each word in the fault description information;

[0099] Step S225, splicing the first attention information and the second attention information to obtain spliced ​​information;

[0100] Step S226: Fuse the spliced ​​information to obtain semantic feature information of the fault description information.

[0101] The first feature information is the future information of each word in the fault description information, the second feature information is the historical information of each word in the fault description information, the first attention information is the information obtained by performing a first self-attention operation on the first feature information, and the second attention information is the information obtained by performing a second self-attention operation on the second feature information. By performing the first self-attention operation and the second self-attention operation on the first feature information and the second feature information respectively, attention to important information in the first feature information and the second feature information can be strengthened.

[0102] In a specific implementation, continue to refer to Figure 3 As shown, the description and analysis module also includes a forward long short-term memory network (Forward LSTM), a backward long short-term memory network (Backward LSTM), a first self-attention layer, and a second self-attention layer, wherein the encoded information is the input item of the forward long short-term memory network and the input item of the backward long short-term memory network, the output item of the forward long short-term memory network is the input item of the first self-attention layer, and the output item of the backward long short-term memory network is the input item of the second self-attention layer.

[0103] Among them, the forward long short-term memory network is a network application form in which the long short-term memory network sequentially memorizes in the forward order of the convolution layer, and the backward long short-term memory network has the same structure as the forward long short-term memory network, except that the input of the backward long short-term memory network is the result of the reverse order of the input of the forward long short-term memory network. The number of forward long short-term memory networks and backward long short-term memory networks can be set as needed, for example, the description and analysis module includes three forward long short-term memory networks cascaded in sequence, and the three forward long short-term memory networks contain 128, 64 and 32 neurons respectively, and the description and analysis module also includes three backward long short-term memory networks cascaded in sequence, and the three backward long short-term memory networks contain 128, 64 and 32 neurons respectively.

[0104] Correspondingly, the step of extracting features from the coded information specifically includes: inputting the coded information into a forward long short-term memory network, performing first information extraction on the coded information through the forward long short-term memory network, and obtaining first feature information of each word in the fault description information; inputting the coded information into a backward long short-term memory network, performing second information extraction on the coded information through the backward long short-term memory network, and obtaining second feature information of each word in the fault description information; inputting the first feature information into a first self-attention layer, performing a first self-attention operation on the first feature information through the first self-attention layer, and obtaining first attention information of each word in the fault description information; inputting the second feature information into a second self-attention layer, performing a second self-attention operation on the second feature information through the second self-attention layer, and obtaining second attention information of each word in the fault description information.

[0105] In a specific implementation, the step of performing a first self-attention operation on the first feature information through the first self-attention layer specifically includes: converting the first feature information into first query information, first key information and first value information, determining the first association information of each word in the fault description information based on the first query information and the first key information, the first association information being used to characterize the correlation between the first feature information of each word and the first feature information of other words; determining the first attention information of each word in the fault description information based on the first association information and the first value information. The calculation formula of the first attention information is: y F，i =∑score F，i，j v F，j , q F，i =W F，q x F，i , k F，j =W F，k x F，j , v F，j =W F，v x F，j ,y F，i represents the first attention information of the i-th word, q F，i represents the first query information obtained by converting the first feature information of the i-th word, k F，j represents the first key information obtained by converting the first feature information of the jth word, v F，j represents the first value information obtained by converting the first feature information of the jth word, x F，i represents the first feature information of the i-th word, x F，j represents the first feature information of the jth word, W F，q , W F，k , W F，v is the preset weight matrix.

[0106] In a specific implementation, the step of performing a second self-attention operation on the second feature information through the second self-attention layer specifically includes: converting the second feature information into second query information, second key information and second value information, determining the second association information of each word in the fault description information based on the second query information and the second key information, the second association information is used to characterize the correlation between the second feature information of each word and the second feature information of other words; determining the second attention information of each word in the fault description information based on the second association information and the second value information. The calculation formula of the second attention information is: y B，i =∑score B，i，j v B，j , q B，i =W B，q x B，i , k B，j =W B，k x B，j , v B，j =W B，v x B，j ,y B，i represents the second attention information of the i-th word, q B，i represents the second query information obtained by converting the second feature information of the i-th word, k B，j represents the second key information obtained by converting the second feature information of the jth word, v B，j The second value information obtained by converting the second feature information of the jth word, xB, i represents the second feature information of the i-th word, x B，j represents the second feature information of the jth word, W B，q、 W B，k , W B，v is the preset weight matrix.

[0107] In a specific implementation, continue to refer to Figure 3 As shown, the description and analysis module also includes a fusion unit and a first convolution unit that are cascaded in sequence, wherein the output item of the first self-attention layer and the output item of the second self-attention layer are the input items of the fusion unit, and the output item of the fusion unit is the input item of the first convolution unit. Correspondingly, the step of extracting features from the encoded information specifically also includes: inputting the first attention information and the second attention information into the fusion unit, splicing the first attention information and the second attention information through the fusion unit to obtain spliced ​​information; inputting the spliced ​​information into the first convolution unit, fusing the spliced ​​information through the first convolution unit, and obtaining semantic feature information of the fault description information. The splicing process of the first attention information and the second attention information can be expressed as: h = [y F，1 ,y B，T , …, y F，T， y B，1 ], h represents splicing information, y F，i represents the first attention information of the i-th word, y B，i Represents the second attention information of the i-th word.

[0108] Step S30: Identify the fault image to obtain an image recognition result.

[0109] The image recognition result is the fault type of the faulty device obtained by identifying the faulty image. In order to improve the accuracy of fault handling, this embodiment obtains the fault description information and fault image of the faulty device, and then identifies the fault image to obtain the image recognition result, so that fault handling can be performed based on the fault type information and the image recognition result in subsequent steps.

[0110] In a specific implementation, the fault processing model also includes an image recognition module, and the step of identifying the fault image to obtain an image recognition result specifically includes: inputting the fault image into the image recognition module, identifying the fault image through the image recognition module, and obtaining an image recognition result.

[0111] In a specific implementation, continue to refer to Figure 2 As shown, step S30 includes:

[0112] Step S31, performing pixel value conversion on the fault image to obtain pixel value information of the fault image;

[0113] Step S32: performing a first convolution operation on the pixel value information to obtain a first convolution feature;

[0114] Step S33, extracting the first convolution feature to obtain third feature information;

[0115] Step S34, performing feature enhancement on the third feature information to obtain fourth feature information;

[0116] Step S35, performing dimensionality reduction and pooling processing on the fourth feature information to obtain fifth feature information;

[0117] Step S36: perform a second convolution operation on the fifth feature information to obtain an image recognition result.

[0118] The pixel value information is a three-dimensional pixel value array of the fault image. When performing pixel value conversion on the fault image, the fault image can be converted into a three-dimensional pixel value array according to the resolution and size of the fault image. For example, the fault image can be converted into a 224*224*3 three-dimensional pixel value array according to the resolution and size of the fault image.

[0119] In a specific implementation, Figure 5 As shown, the image recognition module includes a second convolution unit, a first feature extraction unit, a second feature extraction unit, a first point-by-point convolution unit, a first pooling unit, a second point-by-point convolution unit and a third point-by-point convolution unit which are cascaded in sequence, wherein the pixel value information is the input item of the second convolution unit, the output item of the second convolution unit is the input item of the first feature extraction unit, the output item of the first feature extraction unit is the input item of the second feature extraction unit, the output item of the second feature extraction unit is the input item of the first point-by-point convolution unit, the output item of the first point-by-point convolution unit is the input item of the first pooling unit, the output item of the first pooling unit is the input item of the second point-by-point convolution unit, and the output item of the second point-by-point convolution unit is the input item of the third point-by-point convolution unit.

[0120] Correspondingly, the step of identifying the fault image specifically includes: performing pixel value conversion on the fault image to obtain pixel value information of the fault image; inputting the pixel value information into the second convolution unit, performing a first convolution operation on the pixel value information through the second convolution unit, and obtaining a first convolution feature; inputting the first convolution feature into the first feature extraction unit, performing feature extraction on the first convolution feature through the first feature extraction unit, and obtaining third feature information; inputting the third feature information into the second feature extraction unit, performing feature enhancement on the third feature information through the second feature extraction unit, and obtaining fourth feature information; performing dimensionality reduction and pooling processing on the fourth feature information through the first point-by-point convolution unit, the first pooling unit, and the second point-by-point convolution unit in sequence, and obtaining fifth feature information; inputting the fifth feature information into the third point-by-point convolution unit, and performing a second convolution operation on the fifth feature information through the third point-by-point convolution unit, and obtaining an image recognition result.

[0121] In a specific implementation, the second convolution unit can adopt a convolution kernel with an output channel number of 16 and a stride of 2; the first pooling unit can adopt an average pooling layer with a size of 7*7, the first point-by-point convolution unit can adopt a point-by-point convolution layer with an output channel number of 100, a convolution kernel size of 1*1, and a stride of 1; the second point-by-point convolution unit can adopt a point-by-point convolution layer with an output channel number of 50, a convolution kernel size of 1*1, and a stride of 1; the third point-by-point convolution unit can adopt a point-by-point convolution layer with an output channel number of 2, a convolution kernel size of 1*1, and a stride of 1.

[0122] Among them, the first point-by-point convolution unit and the second point-by-point convolution unit do not use batch normalization (BatchNormalization), and the third point-by-point convolution unit does not use an activation function, so that the third point-by-point convolution unit can output the final image recognition result. In addition, the image recognition module can reduce the number of parameters of the model by using the first point-by-point convolution unit, the first pooling unit, the second point-by-point convolution unit and the third point-by-point convolution unit instead of the conventional fully connected layer.

[0123] Considering that the use of Relu6 function and H-swish function as activation functions can overcome the gradient vanishing problem and speed up the training speed to a certain extent, all negative values ​​are set to 0. When the number of training times increases, the weights of some convolution kernels cannot be updated, and a large number of neurons will be inactivated. Therefore, in this embodiment, the second convolution unit, the first feature extraction unit, the second feature extraction unit, the first point-by-point convolution unit and the second point-by-point convolution unit all use the PRelu function (PR) as the activation function, and the PRelu function can be expressed as: It can be seen from the expression of the PRelu function that the PRelu function adds a linear term in the negative value domain, which can reduce neuron inactivation and extract more features. In addition, the a in the PRelu function will be adaptively adjusted according to the data during the training process, so that the trained model is more in line with the fault data.

[0124] In a specific implementation, the number of the first feature extraction unit and the second feature extraction unit can be set as needed. When the first feature extraction unit and the second feature extraction unit are both set to multiple, the multiple first feature extraction units and the multiple second feature extraction units can be alternately set. Figure 5 As shown, the number of the first feature extraction units is set to 7, the number of the second feature extraction units is set to 8, 3 first feature extraction units are cascaded after 3 second feature extraction units, and then 4 first feature extraction units are cascaded after 3 second feature extraction units, and then 5 second feature extraction units are cascaded after 4 first feature extraction units.

[0125] Furthermore, the number of output channels and step sizes of the multiple first feature extraction units may be different, for example, Figure 5 The number of output channels of the seven first feature extraction units shown may be 16, 24, 24, 80, 80, 80, 80, respectively, and the step sizes of the seven first feature extraction units may be 1, 2, 1, 2, 1, 1, 1, respectively. The number of output channels and step sizes of the multiple second feature extraction units may also be different, for example, Figure 5 The number of output channels of the eight second feature extraction units shown may be 40, 40, 40, 112, 112, 160, 160, 160, respectively, and the step sizes of the eight second feature extraction units may be 2, 1, 1, 1, 1, 2, 1, 1, respectively.

[0126] In a specific implementation, Figure 6 As shown, step S33 includes:

[0127] Step S331, performing dimension-upgrading processing on the first convolution feature to obtain a first dimension-upgrading feature;

[0128] Step S332: performing pooling processing on the first dimension-raising feature to obtain a first pooled feature;

[0129] Step S333: performing a third convolution operation on the first dimension-upgraded feature to obtain a second convolution feature, and performing a fourth convolution operation on the first dimension-upgraded feature to obtain a third convolution feature;

[0130] Step S334: Fusing the first pooling feature, the second convolution feature, and the third convolution feature to obtain a first fused feature;

[0131] Step S335: performing dimensionality reduction processing on the first fusion feature to obtain a first dimensionality reduction feature;

[0132] Step S336: fuse the first dimensionality reduction feature and the first convolution feature to obtain third feature information.

[0133] In a specific implementation, Figure 7 As shown, the first feature extraction unit includes a first convolution layer, a first pooling layer, a first depth convolution layer, a second depth convolution layer, a first fusion layer, a second convolution layer and a second fusion layer. The output item of the first convolution layer is the input item of the first pooling layer, the input item of the first depth convolution layer and the input item of the second depth convolution layer, the output item of the first pooling layer, the output item of the first depth convolution layer and the output item of the second depth convolution layer are the input items of the first fusion layer, the output item of the first fusion layer is the input item of the second convolution layer, and the output item of the second convolution layer and the input item of the first convolution layer are the input items of the second fusion layer.

[0134] Correspondingly, the step of extracting the first convolution feature to obtain the third feature information specifically includes: inputting the first convolution feature into the first convolution layer, performing dimensionality upscaling processing on the first convolution feature through the first convolution layer, and obtaining the first upscaling feature; inputting the first upscaling feature into the first pooling layer, and performing pooling processing on the first upscaling feature through the first pooling layer to obtain the first pooling feature; inputting the first upscaling feature into the first deep convolution layer, and performing the third convolution operation on the first upscaling feature through the first deep convolution layer to obtain the second convolution feature; and inputting the first upscaling feature into the second deep convolution layer, and performing the third convolution operation on the first upscaling feature through the second deep convolution layer to obtain the second convolution feature. The convolution layer performs a fourth convolution operation on the first dimensionality-increased feature to obtain a third convolution feature; the first pooling feature, the second convolution feature and the third convolution feature are input into the first fusion layer, and the first pooling feature, the second convolution feature and the third convolution feature are fused through the first fusion layer to obtain a first fusion feature; the first fusion feature is input into the second convolution layer, and the first fusion feature is reduced in dimension through the second convolution layer to obtain a first reduced in dimension feature; the first reduced in dimension feature and the first convolution feature are input into the second fusion layer, and the first reduced in dimension feature and the first convolution feature are fused through the second fusion layer to obtain third feature information.

[0135] In a specific implementation, the first convolution layer may use a convolution kernel of size 1*1, the first pooling layer may use a maximum pooling layer of size 3*3, the first depth convolution layer may use a convolution kernel of size 5*5, the second depth convolution layer may use a convolution kernel of size 3*3, and the second convolution layer may use a linear convolution kernel of size 1*1. In addition, the first convolution layer, the first depth convolution layer, and the second depth convolution layer may use a PR function as an activation function.

[0136] In a specific implementation, Figure 8As shown, step S34 includes:

[0137] Step S341, performing dimension-upgrading processing on the third feature information to obtain a second dimension-upgraded feature;

[0138] Step S342: performing pooling processing on the second dimension-raising feature to obtain a second pooled feature;

[0139] Step S343: performing a fifth convolution operation on the second dimension-upgraded feature to obtain a fourth convolution feature, and performing a sixth convolution operation on the second dimension-upgraded feature to obtain a fifth convolution feature;

[0140] Step S344, fusing the second pooling feature, the fourth convolution feature and the fifth convolution feature to obtain a second fused feature;

[0141] Step S345: pooling and activating the second fusion feature to obtain weight information;

[0142] Step S346: Process the second fused feature based on the weight information to obtain a third fused feature;

[0143] Step S347, performing dimensionality reduction processing on the third fusion feature to obtain a second dimensionality reduction feature;

[0144] Step S348: fuse the second dimensionality reduction feature and the third feature information to obtain fourth feature information.

[0145] In a specific implementation, Fig. 9 As shown, the second feature extraction unit includes a third convolutional layer, a second pooling layer, a third deep convolutional layer, a fourth deep convolutional layer, a third fusion layer, a third pooling layer, a second fully connected layer, a second activation layer, a third fully connected layer, a third activation layer, a fourth fusion layer, a fourth convolutional layer and a fifth fusion layer. Among them, the output items of the third convolutional layer are the input items of the second pooling layer, the input items of the third deep convolutional layer and the input items of the fourth deep convolutional layer, the output items of the second pooling layer, the output items of the third deep convolutional layer and the output items of the fourth deep convolutional layer are the input items of the third fusion layer, the output items of the third fusion layer are the input items of the third pooling layer, the output items of the third pooling layer are the input items of the second fully connected layer, the output items of the second fully connected layer are the input items of the second activation layer, the output items of the second activation layer are the input items of the third fully connected layer, the output items of the third fully connected layer are the input items of the third activation layer, the output items of the third activation layer and the output items of the third fusion layer are the input items of the fourth fusion layer, the output items of the fourth fusion layer are the input items of the fourth convolutional layer, and the output items of the fourth convolutional layer and the input items of the third convolutional layer are the input items of the fifth fusion layer.

[0146] Correspondingly, the step of performing feature enhancement on the third feature information to obtain the fourth feature information specifically includes: inputting the third feature information into the third convolutional layer, performing dimensionality enhancement processing on the third feature information through the third convolutional layer to obtain a second dimensionality enhancement feature; inputting the second dimensionality enhancement feature into the second pooling layer, performing pooling processing on the second dimensionality enhancement feature through the second pooling layer to obtain a second pooling feature; inputting the second dimensionality enhancement feature into the third deep convolutional layer, performing a fifth convolution operation on the second dimensionality enhancement feature through the third deep convolutional layer to obtain a fourth convolutional feature; and inputting the second dimensionality enhancement feature into the fourth deep convolutional layer, performing a sixth convolution operation on the second dimensionality enhancement feature through the fourth deep convolutional layer to obtain a fifth convolutional feature; inputting the second pooling feature, the fourth convolutional feature and the fifth convolutional feature into the third fusion layer, and performing a sixth convolution operation on the second pooling feature and the fourth convolutional feature through the third fusion layer. and the fifth convolutional feature are fused to obtain a second fused feature; the second fused feature is input into the third pooling layer, and the second fused feature is pooled through the third pooling layer to obtain a third pooling feature; the third pooling feature is activated through the second fully connected layer, the second activation layer, the third fully connected layer, and the third activation layer in sequence to obtain weight information; the weight information and the second fused feature are input into the fourth fusion layer, and the weight information and the second fused feature are two-dimensionally multiplied by the fourth fusion layer to obtain the third fused feature; the third fused feature is input into the fourth convolutional layer, and the third fused feature is reduced in dimension through the fourth convolutional layer to obtain a second reduced in dimension feature; the second reduced in dimension feature and the third feature information are input into the fifth fusion layer, and the second reduced in dimension feature and the third feature information are fused through the fifth fusion layer to obtain the fourth feature information.

[0147] In a specific implementation, the third convolution layer may use a convolution kernel of size 1*1, the second pooling layer may use a maximum pooling layer of size 3*3, the third depth convolution layer may use a convolution kernel of size 5*5, and the fourth depth convolution layer may use a convolution kernel of size 3*3. The third convolution layer, the third depth convolution layer, and the fourth depth convolution layer may use a PR function as an activation function.

[0148] Furthermore, the third pooling layer can adopt a global pooling layer, the second activation layer can adopt a Relu function as an activation function, the third activation layer can adopt a PR function as an activation function, and the fourth convolution layer can adopt a linear convolution kernel of size 1*1.

[0149] Step S40: Determine fault handling result information based on the fault type information and the image recognition result.

[0150] The fault handling result information refers to the fault handling result of the faulty device. When the fault handling result information is determined based on the fault type information and the image recognition result, the fault type information and the image recognition result are matched. When the fault type information matches the image recognition result, the faulty device is handled based on the fault type information, and the fault handling can be achieved without contacting the equipment service operator, thereby improving the timeliness of fault handling and reducing the labor and maintenance costs of the equipment service operator.

[0151] On the contrary, when the fault type information and the image recognition result do not match, a message to contact manual customer service is returned to the user terminal. That is, when the fault type information and the image recognition result do not match, the user needs to contact manual customer service to handle the fault.

[0152] In a specific implementation, continue to refer to Figure 2 As shown, step S40 includes:

[0153] Step S41, matching the fault type information with the image recognition result;

[0154] Step S42: when the fault type information matches the image recognition result, determining fault handling information based on the fault type information;

[0155] Step S43: generating a fault operation instruction based on the fault handling information;

[0156] Step S44: Send the fault operation instruction to the faulty device so that the faulty device executes the fault operation instruction.

[0157] The fault type information and the image recognition result match means that the fault type information is consistent with the image recognition result. For example, if the fault type information is "01" and the image recognition result is "always in the dehydration state and does not stop", the fault type information and the image recognition result match; if the fault type information is "01" and the image recognition result is "foam residue", the fault type information and the image recognition result do not match.

[0158] The fault handling information is the fault handling plan corresponding to the fault type information. For example, if the fault type information is "01", the fault handling information is "stop running and open the washing machine door"; if the fault type information is "02", the fault handling information is "re-execute the washing operation"; if the fault type information is "03", the fault handling information is "execute a single dehydration operation"; if the fault type information is "04", the fault handling information is "open the water inlet valve again".

[0159] The fault operation instruction is an operation instruction that can be executed by the faulty device. After determining the fault handling information corresponding to the fault type information, this embodiment converts the fault handling information into a fault operation instruction that can be executed by the faulty device, and sends the fault operation instruction to the faulty device. The faulty device can handle the fault by itself by executing the fault operation, without the need for the equipment service operator to assign a maintenance specialist to perform on-site maintenance, thereby improving the timeliness of fault handling and reducing the labor and maintenance costs of the equipment service operator.

[0160] In order to improve the accuracy of fault handling, before sending the fault operation instruction to the faulty device, this embodiment may also send fault type information and fault handling information to the user terminal, and receive confirmation information returned by the user terminal based on the fault type information and the fault handling information. After receiving the confirmation information returned by the user terminal based on the fault type information and the fault handling information, the fault operation instruction is sent to the faulty device.

[0161] In order to better implement the fault handling method in the embodiment of the present application, based on the fault handling method, a fault handling device is also provided in the embodiment of the present application, such as Fig.10 As shown, the fault handling device includes:

[0162] An information acquisition unit 610 is used to acquire fault description information and fault images of a faulty device;

[0163] A semantic analysis unit 620, configured to perform semantic analysis on the fault description information to obtain fault type information;

[0164] The image recognition unit 630 is used to recognize the fault image and obtain an image recognition result;

[0165] The fault processing unit 640 is used to determine fault processing result information based on the fault type information and the image recognition result.

[0166] In an embodiment of the present application, semantic analysis is performed on the fault description information to obtain fault type information, the fault image is recognized to obtain an image recognition result, and the fault handling result information is determined based on the fault type information and the image recognition result. The user only needs to input the fault description information and the fault image to realize automatic handling of the fault. There is no need to assign a maintenance specialist to handle the fault through the equipment operator, which can improve the timeliness of fault handling and reduce the labor and maintenance costs of the equipment operator.

[0167] In some embodiments of the present application, the semantic analysis unit 620 is specifically used to:

[0168] Preprocessing the fault description information to obtain coding information of the fault description information;

[0169] Extract features from the coded information to obtain semantic feature information of the fault description information;

[0170] Based on the semantic feature information, the fault type information is determined.

[0171] In some embodiments of the present application, the semantic analysis unit 620 is further configured to:

[0172] Performing first information extraction on the coded information to obtain first feature information of each word in the fault description information;

[0173] Performing second information extraction on the coded information to obtain second feature information of each word in the fault description information;

[0174] Performing a first self-attention operation on the first feature information to obtain first attention information of each word in the fault description information;

[0175] Perform a second self-attention operation on the second feature information to obtain second attention information of each word in the fault description information;

[0176] splicing the first attention information and the second attention information to obtain spliced ​​information;

[0177] The spliced ​​information is fused to obtain the semantic feature information of the fault description information.

[0178] In some embodiments of the present application, the image recognition unit 630 is specifically used for:

[0179] Perform pixel value conversion on the fault image to obtain pixel value information of the fault image;

[0180] Performing a first convolution operation on the pixel value information to obtain a first convolution feature;

[0181] Perform feature extraction on the first convolution feature to obtain third feature information;

[0182] Performing feature enhancement on the third feature information to obtain fourth feature information;

[0183] Performing dimensionality reduction and pooling processing on the fourth feature information to obtain fifth feature information;

[0184] A second convolution operation is performed on the fifth feature information to obtain an image recognition result.

[0185] In some embodiments of the present application, the image recognition unit 630 is further used to:

[0186] Performing dimension-upgrading processing on the first convolution feature to obtain a first dimension-upgrading feature;

[0187] Performing pooling processing on the first dimension-raising feature to obtain a first pooling feature;

[0188] Performing a third convolution operation on the first dimension-increased feature to obtain a second convolution feature, and performing a fourth convolution operation on the first dimension-increased feature to obtain a third convolution feature;

[0189] The first pooling feature, the second convolution feature and the third convolution feature are fused to obtain a first fused feature;

[0190] Performing dimensionality reduction processing on the first fusion feature to obtain a first dimensionality reduction feature;

[0191] The first dimensionality reduction feature and the first convolution feature are fused to obtain the third feature information.

[0192] In some embodiments of the present application, the image recognition unit 630 is further used to:

[0193] Performing dimension-upgrading processing on the third feature information to obtain a second dimension-upgraded feature;

[0194] Performing pooling processing on the second dimension-raising feature to obtain a second pooling feature;

[0195] Performing a fifth convolution operation on the second dimensionally increased feature to obtain a fourth convolution feature, and performing a sixth convolution operation on the second dimensionally increased feature to obtain a fifth convolution feature;

[0196] The second pooling feature, the fourth convolution feature and the fifth convolution feature are fused to obtain a second fused feature;

[0197] Pooling and activation processing are performed on the second fusion feature to obtain weight information;

[0198] Processing the second fused feature based on the weight information to obtain a third fused feature;

[0199] Performing dimensionality reduction processing on the third fusion feature to obtain a second dimensionality reduction feature;

[0200] The second dimension reduction feature and the third feature information are fused to obtain the fourth feature information.

[0201] In some embodiments of the present application, the fault processing unit 640 is specifically used to:

[0202] Matching the fault type information with the image recognition results;

[0203] When the fault type information matches the image recognition result, determining fault handling information based on the fault type information;

[0204] Generate fault operation instructions based on fault handling information;

[0205] The fault operation instruction is sent to the faulty device so that the faulty device executes the fault operation instruction.

[0206] The embodiment of the present application further provides a computer device, which integrates any one of the fault handling devices provided in the embodiment of the present application, and the computer device includes:

[0207] one or more processors;

[0208] Memory; and

[0209] One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by a processor to execute the steps of the fault handling method in any of the above-mentioned fault handling method embodiments.

[0210] The present application also provides a computer device that integrates any of the fault handling devices provided in the present application. Fig.11 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0211] The computer device may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Fig.11 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0212] The processor 801 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0213] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0214] The computer device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0215] The computer device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0216] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0217] Obtain fault description information and fault image of the faulty device;

[0218] Perform semantic analysis on the fault description information to obtain fault type information;

[0219] Identify the fault image and obtain the image recognition result;

[0220] Based on the fault type information and the image recognition result, the fault handling result information is determined.

[0221] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0222] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the fault handling methods provided in the embodiments of the present application. For example, the computer program loaded by the processor may execute the following steps:

[0223] Obtain fault description information and fault image of the faulty device;

[0224] Perform semantic analysis on the fault description information to obtain fault type information;

[0225] Identify the fault image and obtain the image recognition result;

[0226] Based on the fault type information and the image recognition result, the fault handling result information is determined.

[0227] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0228] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0229] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0230] The above is a detailed introduction to a fault handling method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A fault handling method, characterized in that: include: Obtain fault description information and fault image of the faulty device; Performing semantic analysis on the fault description information to obtain fault type information; Recognize the fault image to obtain an image recognition result; Fault handling result information is determined based on the fault type information and the image recognition result.

2. The method according to claim 1, characterized in that The performing semantic analysis on the fault description information to obtain fault type information includes: Preprocessing the fault description information to obtain coding information of the fault description information; Extracting features from the coded information to obtain semantic feature information of the fault description information; Based on the semantic feature information, fault type information is determined.

3. The method according to claim 2, characterized in that The extracting features of the coded information to obtain semantic feature information of the fault description information includes: Performing first information extraction on the coded information to obtain first feature information of each word in the fault description information; Performing second information extraction on the coded information to obtain second feature information of each word in the fault description information; Performing a first self-attention operation on the first feature information to obtain first attention information of each word in the fault description information; Performing a second self-attention operation on the second feature information to obtain second attention information of each word in the fault description information; splicing the first attention information and the second attention information to obtain spliced ​​information; The splicing information is fused to obtain semantic feature information of the fault description information.

4. The method according to claim 1, characterized in that The step of identifying the fault image to obtain an image recognition result includes: Performing pixel value conversion on the fault image to obtain pixel value information of the fault image; Performing a first convolution operation on the pixel value information to obtain a first convolution feature; Performing feature extraction on the first convolution feature to obtain third feature information; Performing feature enhancement on the third feature information to obtain fourth feature information; Performing dimensionality reduction and pooling processing on the fourth feature information to obtain fifth feature information; A second convolution operation is performed on the fifth feature information to obtain an image recognition result.

5. The method according to claim 4, characterized in that The step of extracting the first convolution feature to obtain third feature information includes: Performing dimensionality-increasing processing on the first convolution feature to obtain a first dimensionality-increasing feature; Performing pooling processing on the first dimension-raising feature to obtain a first pooling feature; Performing a third convolution operation on the first dimension-upgraded feature to obtain a second convolution feature, and performing a fourth convolution operation on the first dimension-upgraded feature to obtain a third convolution feature; Fusing the first pooling feature, the second convolution feature, and the third convolution feature to obtain a first fused feature; Performing dimensionality reduction processing on the first fusion feature to obtain a first dimensionality reduction feature; The first dimensionality reduction feature and the first convolution feature are fused to obtain third feature information.

6. The method according to claim 4, characterized in that The step of enhancing the third feature information to obtain fourth feature information includes: Performing dimension-upgrading processing on the third feature information to obtain a second dimension-upgraded feature; Performing pooling processing on the second dimension-raising feature to obtain a second pooling feature; Performing a fifth convolution operation on the second dimension-upgraded feature to obtain a fourth convolution feature, and performing a sixth convolution operation on the second dimension-upgraded feature to obtain a fifth convolution feature; Fusing the second pooling feature, the fourth convolution feature, and the fifth convolution feature to obtain a second fused feature; Performing pooling and activation processing on the second fusion feature to obtain weight information; Processing the second fused feature based on the weight information to obtain a third fused feature; Performing dimensionality reduction processing on the third fusion feature to obtain a second dimensionality reduction feature; The second dimensionality reduction feature and the third feature information are fused to obtain fourth feature information.

7. The method according to claim 1, characterized in that The determining of fault processing result information based on the fault type information and the image recognition result includes: Matching the fault type information with the image recognition result; When the fault type information matches the image recognition result, determining fault handling information based on the fault type information; generating a fault operation instruction based on the fault handling information; The fault operation instruction is sent to the faulty device so that the faulty device executes the fault operation instruction.

8. A fault handling device, characterized in that: include: An information acquisition unit, used to acquire fault description information and fault images of a faulty device; A semantic analysis unit, used to perform semantic analysis on the fault description information to obtain fault type information; An image recognition unit, used to recognize the fault image and obtain an image recognition result; A fault processing unit is used to determine fault processing result information based on the fault type information and the image recognition result.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the fault handling method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the fault handling method according to any one of claims 1 to 7.

Images