Model pre-training method and device applied to power equipment inspection report graphics

Abstract

The application discloses a model pre-training method and device applied to power equipment inspection report graphics and texts, and the method comprises the following steps: acquiring images in a power equipment inspection report and text information corresponding to the images; inputting the images into a pre-configured image encoder to obtain image features output by the image encoder; inputting the text information corresponding to the images into a pre-configured text encoder to obtain text features output by the text encoder; performing feature matching training based on the image features and the text features to train a contrast model; and storing model parameters of the trained contrast model. The embodiment of the application can effectively improve the efficiency of training a target model, and in a low-resource computing device, the efficiency loss caused by a model corresponding to a specific scene can be avoided.

Description

Technical Field

[0001] This invention relates to the field of model training, and in particular to a model pre-training method and device for power equipment inspection report graphics and text. Background Technology

[0002] In real-world power grid scenarios, the ways to capture equipment status information are increasing, data transmission efficiency is constantly improving, and the amount of information stored in terminal systems is also increasing. This necessitates expanding the scale of computational parameters for AI models applied in this field, such as image retrieval models and anomaly detection models. However, limited computing resources in terminal devices restrict the upper bound of model size, thus limiting performance improvement. To address this, we propose a pre-trained model deployed on large computing devices. This effectively reduces the model's dependence on the hardware environment when applied to downstream tasks, reduces training time, and improves response speed. This helps relevant personnel improve the efficiency of problem analysis and decision-making.

[0003] In the power industry, previous pre-training methods were usually designed for single modalities such as images or text, which prevented the pre-trained models from learning multimodal interaction information well, thus limiting the applicability of the models. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a model pre-training method and device for power equipment inspection report graphics and text.

[0005] This invention provides a model pre-training method for power equipment inspection report graphics, including:

[0006] Obtain the images and corresponding text information from the power equipment inspection report;

[0007] The image is input to a pre-configured image encoder to obtain the image features output by the image encoder;

[0008] The text information corresponding to the image is input into a pre-configured text encoder to obtain the text features output by the text encoder;

[0009] Based on the image features and the text features, feature matching training is performed to train a comparison model; wherein, the comparison model is used to characterize the correlation between the image features and the text features;

[0010] Store the model parameters of the trained comparison model.

[0011] Furthermore, the model parameters include the output structure parameters of the comparison model; therefore, after storing the model parameters of the comparison model that has been trained, the method further includes: adjusting the output structure parameters according to a number of preset downstream tasks of power equipment, so that the output of the comparison model corresponds to the output results of each of the downstream tasks of power equipment.

[0012] Furthermore, the step of obtaining the images and corresponding text information from the power equipment inspection reports specifically includes: extracting images and text descriptions from several power equipment inspection reports using an OCR model; wherein the text descriptions include line names, equipment names, defect descriptions, and defect levels; constructing an image list composed of the images; iteratively merging the extracted text descriptions using a merging algorithm to obtain a general description list for representing equipment description information; removing duplicate fields from the general description list to obtain a text list; and constructing a mapping relationship between the image list and the text list using a disjoint-set data structure algorithm to ensure that the text information in the text list corresponds one-to-one with the images in the image list.

[0013] Furthermore, the image encoder includes a convolutional neural network and a Transformer encoder; then, the step of inputting the image into a pre-configured image encoder to obtain the image features output by the image encoder specifically includes: cropping the image into a preset format; inputting the cropped image into the convolutional neural network including convolutional kernels and max pooling layers to obtain a local feature map output by the convolutional neural network; wherein, the local feature map includes local features and key features, the convolutional kernel is used to extract the local features, and the max pooling layer is used to extract the key features from the local features; inputting the local feature map into a Transformer encoder including a multi-head self-attention module to obtain the image features output by the Transformer encoder; wherein, the multi-head self-attention module is used to calculate global features from the local feature map.

[0014] Furthermore, the text encoder includes a k-gram text extractor and a text multi-head self-attention module; then, the step of inputting the text information corresponding to the image into the pre-configured text encoder to obtain the text features output by the text encoder specifically includes: inputting the text information corresponding to the image into the k-gram text extractor to perform word segmentation on the text information corresponding to the image, and performing embedding representation processing on the word segmented text information; inputting the text information after embedding representation processing into the text multi-head self-attention module to obtain the text features output by the text multi-head self-attention module.

[0015] Furthermore, the step of training a contrast model by performing feature matching based on the image features and the text features specifically includes: using a contrastive learning method to train feature matching on several image features and several text features obtained from several power equipment inspection reports.

[0016] Furthermore, the step of using a contrastive learning method to perform feature matching training on the image features and the text features specifically includes: performing L2 normalization on each image feature and each text feature; calculating an interaction feature matrix based on the L2 normalized image features and text features; obtaining a contrastive loss function based on the label matrix corresponding to the interaction feature matrix and a preset cross-entropy loss function; and outputting the trained contrastive model when the contrastive loss function converges.

[0017] Furthermore, the calculation of the interaction feature matrix is ​​specifically as follows:

[0018]

[0019] in, The interaction feature matrix, Image features after L2 normalization Here, t represents the text features after L2 normalization, t represents the training parameters, and e represents the natural index.

[0020] The cross-entropy loss function is specifically as follows:

[0021]

[0022] in For true sample values, To predict sample values, N is the number of samples;

[0023] The specific contrastive loss function is as follows:

[0024]

[0025]

[0026]

[0027]

[0028] in, For feature dimension, Indicates the row number of the matrix. Indicates the column index of the matrix. To construct the obtained dimensions and A consistent label matrix.

[0029] Furthermore, adjusting the output structure parameters according to a number of preset downstream tasks of power equipment to ensure that the output of the comparison model corresponds one-to-one with each of the downstream tasks of power equipment specifically includes: if the downstream task of power equipment is a classification task, adding a mapping layer and a first function to the output layer of the comparison model; wherein the first function includes sigmoid and softmax functions; if the downstream task of power equipment is a regression task, using a multilayer perceptron as the output layer of the comparison model; wherein the multilayer perceptron is used to output a correlation coefficient to characterize the degree of matching.

[0030] This invention also provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the model pre-training method for applying the above-described power equipment inspection report graphics and text.

[0031] In summary, the present invention has the following beneficial effects:

[0032] By employing the embodiments of this invention, training data can be automatically acquired and preprocessed. After acquiring the training data, the target model can be trained using the training data and a general model framework. When training the target model, training is performed based on the general model framework. For developers, there is no need to write software code corresponding to the target model; instead, they only need to focus on the training data and prediction task of the target model. Furthermore, this effectively improves the efficiency of training the target model, and on low-resource computing devices, it avoids the efficiency loss caused by models designed for specific scenarios, thus improving training efficiency. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating an embodiment of a model pre-training method for power equipment inspection report graphics provided by the present invention;

[0034] Figure 2 This is a schematic diagram of a specific embodiment of a model pre-training method for power equipment inspection report graphics provided by the present invention;

[0035] Figure 3 This is a schematic diagram of a specific embodiment of the comparative learning method for model pre-training of power equipment inspection report graphics provided by the present invention;

[0036] Figure 4 This is a schematic diagram of a specific embodiment of a text encoder and image encoder for a model pre-training method for power equipment inspection reports provided by the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] See Figure 1 This is a flowchart illustrating an embodiment of the model pre-training method for power equipment inspection report graphics provided by the present invention. The method includes steps S1 to S5, as follows:

[0039] S1, Obtain the images and corresponding text information from the power equipment inspection report;

[0040] Preferably, the step of obtaining the images and corresponding text information from the power equipment inspection reports specifically includes: extracting images and text descriptions from several power equipment inspection reports using an OCR model; wherein the text descriptions include line names, equipment names, defect descriptions, and defect levels; constructing an image list composed of the images; iteratively merging the extracted text descriptions using a merging algorithm to obtain a general description list for representing equipment description information; removing duplicate fields from the general description list to obtain a text list; and constructing a mapping relationship between the image list and the text list using a disjoint-set data structure algorithm, so that the text information in the text list corresponds one-to-one with the images in the image list.

[0041] For example, an OCR model can be used to extract images and text from power equipment inspection reports, and lists of image and text descriptions can be constructed respectively. Items matching the content are in the same position in the list. The text description includes the line name, equipment name, defect description, and defect level. To ensure the generality of the description information, the numbered portion of the fields has been removed.

[0042] See Figure 3 The algorithm iteratively merges common text segments with different descriptions using a merging algorithm. The merged result serves as a representative category of equipment description information (e.g., no defects, rusted parts, foreign objects present). To reduce storage space, a disjoint-set data structure (DFS) algorithm is used to group texts with identical content into one category, sharing storage addresses and maintaining a one-to-one mapping with the content in the image list. The construction process of the DFS and mapping relationship is as follows:

[0043] A. Construct a list of categories of length n. and index list , representing the category corresponding to each text description and the position of the previous description in the same category, respectively. To locate the position of the first description in the same category, . Each cell is assigned its index in the list.

[0044] B. Compile representative descriptive information into a list. The list index serves as the category for different descriptions.

[0045] C. Traversal For each text Iterate through the list ,like If it appears in the middle, then Category , Then recursively traverse backwards from that position. and until At this point, the assignment operation is .

[0046] D. Delete the original text list ,by As a list of categorized text descriptions As an image list and The mapping table.

[0047] The construction work is now complete.

[0048] S2, The image is input to a pre-configured image encoder to obtain the image features output by the image encoder;

[0049] Preferably, the image encoder includes a convolutional neural network and a Transformer encoder; then, the step of inputting the image into a pre-configured image encoder to obtain the image features output by the image encoder specifically includes: cropping the image into a preset format; inputting the cropped image into the convolutional neural network including convolutional kernels and max pooling layers to obtain a local feature map output by the convolutional neural network; wherein, the local feature map includes local features and key features, the convolutional kernels are used to extract the local features, and the max pooling layers are used to extract the key features from the local features; inputting the local feature map into a Transformer encoder including a multi-head self-attention module to obtain the image features output by the Transformer encoder; wherein, the multi-head self-attention module is used to calculate global features from the local feature map.

[0050] For example, an image list is extracted using a convolutional neural network (CNN) and a self-attention module. The features of each sample. The specific implementation steps are as follows:

[0051] Crop the sample images into the same format. , where h represents the size of the image;

[0052] Construct the CNN backbone to obtain local feature maps of the image. The convolutional kernel is used for local feature extraction, and the max pooling layer extracts key features from the output signal of the convolutional kernel. This visual feature map contains local features in the inspection image.

[0053] The feature map obtained from convolution is used as the token input to the Transformer encoder. Global features of this map are then calculated through multi-head self-attention, resulting in the final image features. Where N is the number of samples and M is the dimension of the feature, expressed by the formula: .

[0054] S3, Input the text information corresponding to the image into a pre-configured text encoder to obtain the text features output by the text encoder;

[0055] Preferably, the text encoder includes a k-gram text extractor and a text multi-head self-attention module; then, the step of inputting the text information corresponding to the image into the pre-configured text encoder to obtain the text features output by the text encoder specifically includes: inputting the text information corresponding to the image into the k-gram text extractor to perform word segmentation on the text information corresponding to the image, and performing embedding representation processing on the word segmented text information; inputting the text information after embedding representation processing into the text multi-head self-attention module to obtain the text features output by the text multi-head self-attention module.

[0056] For example, the implementation steps for text feature extraction are as follows:

[0057] The list is segmented using the k-gram method. Feature text in Perform word segmentation and represent it in an embedded form;

[0058] Construct a multi-head self-attention module for text to learn the contextual semantic relationships between words in a sentence, thereby obtaining text features. ,in The number of samples. The dimension of the feature can be represented as: .

[0059] S4, Based on the image features and the text features, perform feature matching training to train a comparison model; wherein, the comparison model is used to characterize the correlation between the image features and the text features;

[0060] Preferably, the step of training a contrast model by performing feature matching based on the image features and the text features specifically includes: using a contrastive learning method to train feature matching on several image features and several text features obtained from several power equipment inspection reports.

[0061] As an improvement to the above scheme, the method of using contrastive learning to perform feature matching training on the image features and the text features specifically includes: performing L2 normalization on each image feature and each text feature; calculating an interaction feature matrix based on the L2 normalized image features and text features; obtaining a contrastive loss function based on the label matrix corresponding to the interaction feature matrix and a preset cross-entropy loss function; and outputting the trained contrastive model when the contrastive loss function converges.

[0062] As a further improvement to the above scheme, the calculation of the interaction feature matrix is ​​specifically as follows:

[0063]

[0064] in, The interaction feature matrix, Image features after L2 normalization Here, t represents the text features after L2 normalization, t represents the training parameters, and e represents the natural index.

[0065] The cross-entropy loss function is specifically as follows:

[0066]

[0067] in For true sample values, To predict sample values, N is the number of samples;

[0068] The specific contrastive loss function is as follows:

[0069]

[0070]

[0071]

[0072]

[0073] in, For feature dimension, Indicates the row number of the matrix. Indicates the column index of the matrix. To construct the obtained dimensions and A consistent label matrix.

[0074] For example, see Figure 3 :

[0075] To maintain consistency in image and text scale, both image and text features are L2 normalized. L2 normalization is as follows: ;

[0076] The image and text pair corresponding to the index in the mapping list is considered a positive sample, and other image and text interaction pairs are considered negative samples. The interaction feature matrix of the two modalities is calculated by the following formula: , where I and T represent the embedding representations of the image and text, respectively, and t is a learnable parameter;

[0077] Construct a contrastive loss function, using the cross-entropy loss function as the loss in a single modal direction, as shown in the following formula: ,in For true sample values, To predict sample values, the loss function for images and text can be calculated based on the true and predicted values ​​using the following two formulas: , Finally, the loss function that combines image and text is: ;

[0078] During pre-training, the contrastive loss function is optimized using the gradient descent algorithm. The encoder parameters are updated using backpropagation. The training process ends when the loss function converges. It is worth noting that the encoder parameters are fixed and do not participate in the training of subsequent downstream models.

[0079] The target task is set as an image multi-class classification task, given a batch of detection image samples and a text set describing several anomalies as input. First, the pre-trained model parameters are input into a convolutional block for learning. The convolutional block includes convolutional layers, batch normalization layers, activation layers, and max-pooling layers. Further, the LeakReLu activation function is used to increase the model's non-linearity, and max-pooling layers are used to compress the data and extract key features. Through the above processing, the inputs from the two modalities are transformed into a hybrid feature map.

[0080] The feature maps, which combine image and text content, are then passed through a CNN convolutional network. Each feature map is then reshaped into an m-dimensional vector, which is combined with the number of channels n to form n*m, which serves as the input to the Transformer encoder.

[0081] Next, global features are learned through the Transformer encoder, specifically including: processing data of different lengths through a normalization layer; improving the performance of the self-attention layer through a multi-head attention mechanism; and converting the input vector into three different vectors: query vector q, key vector k, and value vector v, and compressing these vectors into different matrices Q, K, and V.

[0082] S5 stores the model parameters of the trained comparison model.

[0083] Preferably, the model parameters include the output structure parameters of the comparison model; then, after storing the model parameters of the comparison model that has been trained, the method further includes: adjusting the output structure parameters according to a number of preset downstream tasks of power equipment, so that the output of the comparison model corresponds to the output results of each of the downstream tasks of power equipment.

[0084] As an improvement to the above scheme, the step of adjusting the output structure parameters according to a number of preset downstream tasks of power equipment, so that the output of the comparison model corresponds one-to-one with each of the downstream tasks of power equipment, specifically includes: if the downstream task of power equipment is a classification task, then adding a mapping layer and a first function to the output layer of the comparison model; wherein the first function includes sigmoid and softmax functions; if the downstream task of power equipment is a regression task, then using a multilayer perceptron as the output layer of the comparison model; wherein the multilayer perceptron is used to output a correlation coefficient to characterize the degree of matching.

[0085] It should be noted that, to address the issue of self-attention being insensitive to positional information, positional encodings of the same dimension are added to the original input embedding. After passing through the Drop Path layer, the multi-branch structure is randomly rendered ineffective, and then a multilayer perceptron (MLP) is used to solve the nonlinear problems that a single-layer perceptron cannot handle.

[0086] We use the CB Loss method to solve the data imbalance problem and focal Loss to handle samples that are difficult to distinguish.

[0087] Finally, the feature map is input into the fully connected layer, and then passed through the Softmax function to obtain the final prediction result.

[0088] The batch of image and text data is divided into a training set, a validation set, and a test set. During the training and validation processes, the parameters of the overall model are fine-tuned, and finally, the model's prediction performance is tested on the test set.

[0089] This invention also provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the model pre-training method for applying the above-described power equipment inspection report graphics and text.

[0090] The computer device in this embodiment includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a model pre-training program for power equipment inspection report graphics. When the processor executes the computer program, it implements the steps in the various embodiments of the model pre-training method for power equipment inspection report graphics described above, for example... Figure 1 Steps S1 to S5 are shown.

[0091] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device.

[0092] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device may include, but is not limited to, a processor and memory.

[0093] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting various parts of the computer device via various interfaces and lines.

[0094] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0095] Wherein, if the modules / units integrated into the computer device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0096] In summary, the present invention has the following beneficial effects:

[0097] By employing the embodiments of this invention, training data can be automatically acquired and preprocessed. After acquiring the training data, the target model can be trained using the training data and a general model framework. When training the target model, training is performed based on the general model framework. For developers, there is no need to write software code corresponding to the target model; instead, they only need to focus on the training data and prediction task of the target model. Furthermore, this effectively improves the efficiency of training the target model, and on low-resource computing devices, it avoids the efficiency loss caused by models designed for specific scenarios, thus improving training efficiency.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary hardware platforms, and of course, it can also be implemented entirely by hardware. Based on this understanding, all or part of the technical solution of the present invention that contributes to the background technology can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0099] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A model pre-training method for power equipment inspection report graphics and text, characterized in that, include: Obtain the images and corresponding text information from the power equipment inspection report; The image is input to a pre-configured image encoder to obtain the image features output by the image encoder; The text information corresponding to the image is input into a pre-configured text encoder to obtain the text features output by the text encoder; Based on the image features and the text features, feature matching training is performed to train a comparison model; wherein, the comparison model is used to characterize the correlation between the image features and the text features; Store the model parameters of the trained comparison model; Specifically, acquiring the images and corresponding text information from the power equipment inspection report includes: An OCR model was used to extract images and text descriptions from several power equipment inspection reports; the text descriptions included line names, equipment names, defect descriptions, and defect levels. Construct an image list consisting of the images; The extracted text descriptions are iteratively merged using a merging algorithm to obtain a general description list that represents the device description information. After removing duplicate fields from the general description list, a text list is obtained. A disjoint-set data structure algorithm is used to construct a mapping relationship between the image list and the text list so that the text information in the text list corresponds one-to-one with the images in the image list. The device description information is used to describe any of the following: no defects, rusted parts, or foreign objects. Specifically, the step of training a comparison model based on the image features and the text features through feature matching includes: A contrastive learning method was used to train feature matching on several image features and several text features obtained from several power equipment inspection reports.

2. The model pre-training method for power equipment inspection report graphics as described in claim 1, characterized in that, The model parameters include the output structure parameters of the comparison model; Then, after storing the model parameters of the comparison model that has completed training, the following is also included: The output structure parameters are adjusted according to a number of preset downstream tasks of power equipment, so that the output of the comparison model corresponds to the output results of each of the downstream tasks of power equipment.

3. The model pre-training method for power equipment inspection report graphics as described in claim 1, characterized in that, The image encoder includes a convolutional neural network and a Transformer encoder; Then, the step of inputting the image to a pre-configured image encoder to obtain the image features output by the image encoder specifically includes: The image is cropped to a preset format; The cropped image is input into the convolutional neural network including a convolutional kernel and a max pooling layer to obtain a local feature map output by the convolutional neural network; wherein, the local feature map includes local features and key features, the convolutional kernel is used to extract the local features, and the max pooling layer is used to extract the key features from the local features; The local feature map is input into a Transformer encoder including a multi-head self-attention module to obtain the image features output by the Transformer encoder; wherein, the multi-head self-attention module is used to calculate global features from the local feature map.

4. The model pre-training method for power equipment inspection report graphics as described in claim 1, characterized in that, The text encoder includes a k-gram text extractor and a text multi-head self-attention module; Then, the step of inputting the text information corresponding to the image into a pre-configured text encoder to obtain the text features output by the text encoder specifically includes: The text information corresponding to the image is input into the k-gram text extractor to perform word segmentation on the text information corresponding to the image, and then to perform embedding representation on the segmented text information. The text information after embedding representation processing is input into the text multi-head self-attention module to obtain the text features output by the text multi-head self-attention module.

5. The model pre-training method for power equipment inspection report graphics as described in claim 1, characterized in that, The step of using a contrastive learning method to train feature matching between the image features and the text features specifically includes: L2 normalization is performed on each of the aforementioned image features and each of the aforementioned text features; The interaction feature matrix is ​​calculated based on the L2-normalized image and text features. Based on the label matrix corresponding to the interaction feature matrix and the preset cross-entropy loss function, the contrast loss function is obtained; When the contrastive loss function converges, the trained contrastive model is output. The calculation of the interaction feature matrix is ​​specifically as follows: in, The interaction feature matrix, Image features after L2 normalization Here, t represents the text features after L2 normalization, t represents the training parameters, and e represents the natural index. The cross-entropy loss function is specifically as follows: in For true sample values, To predict sample values, N is the number of samples; The specific contrastive loss function is as follows: in, For feature dimension, Indicates the row number of the matrix. Indicates the column index of the matrix. To construct the obtained dimensions and A consistent label matrix.

6. The model pre-training method for power equipment inspection report graphics as described in claim 2, characterized in that, The step of adjusting the output structure parameters according to a number of preset downstream tasks of power equipment, so that the output of the comparison model corresponds one-to-one with the output results of each of the downstream tasks of power equipment, specifically includes: If the downstream task of the power equipment is a classification task, then a mapping layer and a first function are added to the output layer of the comparison model; wherein, the first function includes sigmoid and softmax functions; If the downstream task of the power equipment is a regression task, then a multilayer perceptron is used as the output layer of the comparison model; wherein, the multilayer perceptron is used to output the correlation coefficient to characterize the degree of matching.

7. A computer device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the model pre-training method for applying to the graphic and textual representation of power equipment inspection reports as described in any one of claims 1 to 6.

Images