Power communication cross-medium data identification method and device and storage medium

Abstract

The application discloses a power communication cross-medium data identification method and device and a storage medium, relates to the technical field of intelligent management of power communication systems, and mainly aims at improving the text recognition accuracy of power communication cross-medium data. The method comprises the following steps: acquiring a non-regular power communication cross-medium data image, and determining initial image features of the non-regular power communication cross-medium data image; inputting the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features; inputting the enhanced image features into a preset image content identification model for image content identification of variable scale features to obtain an initial identification result, and performing sequence alignment on the initial identification result and the content in the non-regular power communication cross-medium data image; performing neighborhood vocabulary correction on the initial identification result after sequence alignment, and determining structured text data corresponding to the non-regular power communication cross-medium data image based on the corrected initial identification result.

Description

Technical Field

[0001] This invention relates to the field of intelligent management technology for power communication systems, and in particular to a method, apparatus and storage medium for cross-media data identification in power communication. Background Technology

[0002] The power communication management system contains a large amount of cross-media data. Among this, electronic documents such as maintenance tickets circulating online are formatted in a standardized manner, with minimal differences in line spacing and a regular layout, primarily containing structured printed information. However, the non-standard data requiring significant processing is concentrated in scanned documents and image files: such as scanned PDF copies of channel verification forms (containing table structures, multi-font printed columns, handwritten signatures, and official seals), project management-related PDF files (containing mixed table text in different fonts within the same document, handwritten annotations, fill-in-the-blank date entries, etc.), and images such as on-site photos, scanned copies of paper construction drawings, and engineering route diagrams (with noise due to shooting angle, scanning quality, etc.). This type of non-standard data is characterized by complex layouts (mixed tables and multi-line text, interspersed handwritten content), diverse fonts (coexistence of various types of printed and handwritten fonts), and prominent interference factors (skewed, blurred, stamped covers). Therefore, to ensure the usability of this non-standard data, it is necessary to identify and process it.

[0003] Currently, fixed-scale features are commonly used to directly extract text content from irregular data. However, due to the varying text sizes and line spacing in irregular image data, fixed-scale features cannot accurately capture text in regular image data. Summary of the Invention

[0004] This invention provides a method, apparatus, and storage medium for cross-media data recognition in power communication, mainly aimed at improving the accuracy of text recognition in cross-media data of power communication.

[0005] According to a first aspect of the present invention, a method for identifying cross-media data in power communications is provided, comprising: Acquire irregular power communication cross-media data images and determine the initial image features of the irregular power communication cross-media data images; The initial image features are input into a preset feature enhancement model for feature enhancement to obtain enhanced image features; The enhanced image features are input into a preset image content recognition model to perform image content recognition with variable scale features, and an initial recognition result is obtained. The initial recognition result is then sequentially aligned with the content in the irregular power communication cross-media data image. The initial recognition results after sequence alignment are corrected for neighboring vocabulary, and the structured text data corresponding to the irregular power communication cross-media data image is determined based on the corrected initial recognition results.

[0006] Optionally, the initial image features are input into a preset feature enhancement model for feature enhancement to obtain enhanced image features, including: Obtain a preset feature enhancement model, wherein the preset feature enhancement model includes a first cascaded convolutional layer, a second cascaded convolutional layer, a third cascaded convolutional layer, and a fourth cascaded convolutional layer for convolutional feature extraction, a first spatial attention layer and a second spatial attention layer for enhancing text region features in an image, and a multi-scale feature fusion layer for fusing image features. The initial image features are input into a preset feature enhancement model. Convolutional features are extracted from the initial image features sequentially through a first cascaded convolutional layer, a second cascaded convolutional layer, a third cascaded convolutional layer, and a fourth cascaded convolutional layer, resulting in the output features of the first, second, third, and fourth cascaded convolutional layers, respectively. A first spatial attention layer and a second spatial attention layer are used to enhance the text region features of the output features of the third and fourth cascaded convolutional layers, respectively, resulting in the text enhancement features output by the first and second spatial attention layers. Finally, a multi-scale feature fusion layer fuses the output features of the first, second, and third cascaded convolutional layers to obtain the enhanced image features.

[0007] Optionally, before inputting the sequence features of the preset dimension into a preset image content recognition model to perform image content recognition and obtain an initial recognition result, the method further includes: Construct a pre-defined initial image content recognition model; Obtain a sample dataset, wherein the sample dataset includes sequence features of sample irregular images with image text content labels of a preset dimension; The sample dataset is divided into a training set and a test set. The training set is used to train the preset initial image content recognition model, and the test set is used to test the trained preset initial image content recognition model. Finally, the trained preset initial image content recognition model that meets the test conditions is taken as the preset image content recognition model.

[0008] Optionally, the enhanced image features are input into a preset image content recognition model to perform variable-scale feature image content recognition, obtaining an initial recognition result, including: Determine the height direction of the enhanced image features, and perform lateral global average pooling on the enhanced image features according to the height direction, and use the pooled enhanced image features as one-dimensional sequence features; The one-dimensional sequence features are mapped to a preset number of character spaces using the fully connected layer in the preset image content recognition model. The initial recognition result is determined based on the mapping result, and the initial recognition result is used as the predicted text sequence in the irregular power communication cross-media data image. The step of aligning the initial identification result with the content in the irregular power communication cross-media data image includes: Determine the tag text sequence in the irregular power communication cross-media data image, and determine the matching path between the predicted text sequence and the tag text sequence; The predicted text sequence is reordered based on each of the matching paths to obtain the reordered predicted text sequence. Determine the confidence level corresponding to each line-order adjusted predicted text sequence, and based on the confidence level, select the target predicted text sequence as the aligned initial recognition result in each line-order adjusted predicted text sequence.

[0009] Optionally, the initial recognition result after sequence alignment is subjected to neighborhood vocabulary correction, including: Determine the data domain to which the irregular power communication cross-media data image belongs; The aligned initial recognition results are segmented to obtain individual words, and based on the data domain, neighboring words to be corrected are determined in each segment. Obtain a preset dictionary corresponding to the data neighborhood, and perform similarity matching between the words in the neighborhood to be corrected and each standard word in the preset dictionary. Based on the similarity matching results, determine the target standard words in the preset dictionary to correct the words in the neighborhood to be corrected. The target standard vocabulary is used to perform vocabulary correction on the neighboring vocabulary to be corrected.

[0010] Optionally, the method further includes: The preset feature enhancement model and the preset image content recognition model are respectively used as target models. The current model parameters of the target models are determined, and a sample model pruning dataset is obtained. The sample model pruning dataset includes sample input data with actual output data labels that are adapted to the prediction scene of the target model. Based on the current model parameters, determine the overfitting constraint terms of the target model; The initial loss function of the target model is determined based on the cross-entropy between the actual output data label corresponding to the sample input data and the predicted output data corresponding to the sample input data predicted by the target model. The overfitting constraint term is added to the initial loss function to obtain the target loss function. The target loss function is then used to iteratively train the channel parameters of each network layer in the target model to obtain the channel parameters of each channel in each network layer after the final iterative training. Each network layer is taken as a target network layer. Based on the absolute value of the channel parameter of each channel in the target network layer, the channel to be pruned is determined in the target network layer, and the channel to be pruned is removed in the target network layer to obtain the pruned target model. The step of inputting the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features includes: The initial image features are input into the pruned preset feature enhancement model for feature enhancement to obtain enhanced image features; The enhanced image features are input into a preset image content recognition model to perform variable-scale feature image content recognition, obtaining initial recognition results, including: The enhanced image features are input into the pruned preset image content recognition model to perform image content recognition and obtain the initial recognition result.

[0011] Optionally, before determining the initial image features of the irregular power communication cross-media data image, the method further includes: The generation feature information of the irregular power communication cross-media data image is determined, wherein the generation feature information includes the underexposure degree, illumination uniformity, and scanning attribute information of the irregular power communication cross-media data image; Based on the generated feature information, the curve constraint conditions of the anti-interference adjustment curve to be generated are determined, wherein the curve constraint conditions include the pixel value range of the image after adjustment based on the anti-interference adjustment curve to be generated, the monotonicity and backpropagation of the anti-interference adjustment curve to be generated. Based on the curve constraints, an anti-interference adjustment curve is constructed for the anti-interference adjustment of the irregular power communication cross-media data image. The interference is adjusted during the generation of the irregular power communication cross-media data image using the anti-interference adjustment curve, resulting in the interference-adjusted irregular power communication cross-media data image. The determination of the initial image features of the irregular power communication cross-media data image includes: Determine the initial image features of the irregular power communication cross-media data image after interference adjustment.

[0012] According to a second aspect of the present invention, a cross-media data identification device for power communication is provided, comprising: An acquisition unit is used to acquire irregular power communication cross-media data images and determine the initial image features of the irregular power communication cross-media data images; The feature enhancement unit is used to input the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features; The sequence alignment unit is used to input the enhanced image features into a preset image content recognition model to perform image content recognition of variable scale features, obtain an initial recognition result, and perform sequence alignment of the initial recognition result with the content in the irregular power communication cross-media data image. The text determination unit is used to perform neighborhood vocabulary correction on the initial recognition result after sequence alignment, and determine the structured text data corresponding to the irregular power communication cross-media data image based on the corrected initial recognition result.

[0013] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described cross-media data identification method for power communication.

[0014] According to a fourth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described cross-media data identification method for power communication.

[0015] According to the present invention, a method, apparatus, and storage medium for identifying cross-media data in power communication are provided. Compared with the current method of directly extracting text content from irregular data using fixed-scale features, the present invention inputs the initial image features of the irregular cross-media data image of power communication into a preset feature enhancement model for feature enhancement to obtain enhanced image features; then inputs the enhanced image features into a preset image content recognition model for image content recognition using variable-scale features to obtain an initial recognition result; and then performs sequence alignment between the initial recognition result and the content in the irregular cross-media data image of power communication; finally, the sequence-aligned initial recognition result is subjected to neighborhood vocabulary correction, and based on the corrected initial recognition result, the structured text data corresponding to the irregular cross-media data image of power communication is determined. Therefore, by pre-enhancing image features of irregular power communication cross-media data images, the influence of blurry images, tilted shooting angles, and scanning blur on text recognition can be eliminated, thereby improving the accuracy of text recognition in irregular power communication cross-media data images. The variable scale of the preset image content recognition model can adapt to arbitrary line spacing and text size in the image, thus improving text recognition accuracy. By aligning the text recognition results of the preset image content recognition model with the text content in the irregular power communication cross-media data image, problems such as line order disorder and character misalignment can be avoided, further improving text recognition accuracy. Neighborhood vocabulary correction of the text recognition results can correct erroneous words in the recognition results, further improving text recognition accuracy. This ensures the communication and application effects of subsequent power data. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart of a cross-media data identification method for power communication provided by an embodiment of the present invention is shown; Figure 2 A flowchart of another method for cross-media data identification in power communication provided by an embodiment of the present invention is shown; Figure 3 This diagram illustrates a text feature enhancement module structure according to an embodiment of the present invention. Figure 4 This diagram illustrates the workflow of the spatial attention layer in a preset feature enhancement model provided by an embodiment of the present invention. Figure 5 This diagram illustrates the workflow of a preset image content recognition model provided in an embodiment of the present invention. Figure 6 This diagram illustrates the structure of a cross-media data identification device for power communication according to an embodiment of the present invention. Figure 7 This invention provides a schematic diagram of the structure of another cross-media data identification device for power communication. Figure 8 A schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0017] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0018] Currently, the method of directly extracting text content from irregular data using fixed-scale features is not suitable for text of varying sizes and line spacing in irregular image data, resulting in the inability of fixed-scale features to accurately capture text in regular image data.

[0019] To address the aforementioned problems, embodiments of the present invention provide a method for cross-media data identification in power communication, such as... Figure 1 As shown, the method includes: 101. Acquire non-uniform power communication cross-media data images and determine the initial image features of non-uniform power communication cross-media data images.

[0020] Among them, non-regular power communication cross-media data images can be images taken from equipment nameplates, on-site construction records, paper construction drawings, etc. at the power equipment site, as well as engineering design routing diagrams, maintenance plan routing diagrams, etc., and can also be scanned images of the above images.

[0021] In this embodiment of the invention, due to issues such as shooting light and angle, images may exhibit irregularities such as blurring and tilting. To improve image quality and facilitate accurate text recognition, anti-interference processing can be applied to the acquired irregular cross-media power communication data image. Based on this, the method includes: determining the generation feature information of the irregular cross-media power communication data image, wherein the generation feature information includes the underexposure degree, illumination uniformity, and scanning attribute information of the irregular cross-media power communication data image; determining the curve constraint conditions for generating an anti-interference adjustment curve based on the generation feature information, wherein the curve constraint conditions include the pixel value range of the image adjusted based on the anti-interference adjustment curve, the monotonicity of the anti-interference adjustment curve, and the retransmissionability of the anti-interference adjustment curve; constructing an anti-interference adjustment curve for anti-interference adjustment of the irregular cross-media power communication data image based on the curve constraint conditions; and using the anti-interference adjustment curve to adjust the interference generated by the irregular cross-media power communication data image to obtain the interference-adjusted irregular cross-media power communication data image.

[0022] Specifically, firstly, grayscale histogram statistics are performed on the irregular power communication cross-media data image. Based on the grayscale histogram, the total number of pixels at each grayscale level in the irregular power communication cross-media data image is counted, and the number of low-resolution pixels with values ​​below a preset threshold (where the preset threshold is set according to actual needs, such as 30) in the histogram is calculated. The ratio of the number of low-resolution pixels to the total number of pixels is taken as the underexposure ratio of the irregular power communication cross-media data image. The degree of underexposure corresponding to this underexposure ratio is determined in a preset configuration table, where the degree of underexposure corresponding to different underexposure ratio ranges is stored. When determining the illumination uniformity of the irregular power communication cross-media data image, the irregular power communication cross-media data image is first divided into multiple non-overlapping image blocks. The grayscale mean is calculated for each image block, and the variance corresponding to each grayscale mean is determined. Finally, the illumination uniformity of the irregular power communication cross-media data image is determined based on the variance. For example, the larger the variance, the more uneven the illumination of the irregular power communication cross-media data image. Scan attribute information refers to information such as scan sharpness and scan tilt. Further, after determining the underexposure degree, illumination uniformity, and scan attribute information of the irregular power communication cross-media data image, the pixel value range, monotonicity, and retransmissionability of the adjusted image are set based on these image generation features. For example, if the irregular power communication cross-media data image is underexposed and has uneven illumination or scanning, the minimum brightness of the output image is constrained to 10, and the maximum brightness to 240 (8-bit image). This ensures that details in the dark areas are visible and highlights are not overexposed. Adaptive equalization is performed on the histogram of the irregular power communication cross-media data image to generate a target histogram distribution, which constrains the output range of the anti-interference adjustment curve. Simultaneously, the anti-interference adjustment curve must satisfy the monotonicity requirement of a strict positive correlation between input and output pixel values ​​to avoid brightness reversal or artifacts. Furthermore, it is necessary to ensure that the slope of each segment of the anti-interference adjustment curve is greater than zero, and overall continuity is ensured through interpolation. Meanwhile, for any output pixel value y, there exists a unique corresponding input pixel value x, ensuring reversible image processing and thus achieving curve backpropagation. Finally, based on the curve constraints, an anti-interference adjustment curve is constructed for dynamic range adjustment of irregular power communication cross-media data images. This ensures the anti-interference adjustment curve is monotonic and gradient backpropagation is possible. A monotonic curve is used to maintain the contrast of the original image, and gradient backpropagation ensures optimization of the curve parameters. Ultimately, each pixel value in the image adjusted by the anti-interference adjustment curve falls within the required range, such as each pixel value in the enhancement result falling within the [0,1] range. The aim is to avoid information loss due to overflow or truncation.

[0023] Furthermore, after generating the anti-interference adjustment curve, the anti-interference adjustment curve can be used to adjust the non-uniform power communication cross-media data image against interference. Based on this, the method includes: determining the value of each pixel in the non-uniform power communication cross-media data image; using the anti-interference adjustment curve to perform a mapping transformation on each pixel value respectively, and the non-uniform power communication cross-media data image after interference adjustment is constituted by each pixel value after mapping transformation.

[0024] Specifically, the anti-interference adjustment curve can take the form of the following:

[0025] in, For pixel values, The curve parameters are for the anti-interference adjustment curve. pixel value The corresponding mapped pixel values.

[0026] Furthermore, after performing anti-interference processing on the irregular cross-media data image of power communication, it is necessary to extract image features from the anti-interference processed irregular cross-media data image of power communication. Specifically, the feature extraction method is as follows: the anti-interference processed irregular cross-media data image of power communication is subjected to denoising and binarization processing to obtain a binarized image. The binarized image is then standardized to initial image features of a preset dimension using preset mean and preset variance. The preset mean and preset variance are set according to actual needs; for example, the preset mean can be μ=[0.485,0.456,0.406], and the preset variance can be σ=[0.229,0.224,0.225]. The preset dimension is also set according to actual needs, such as 512×512×3.

[0027] 102. Input the initial image features into the preset feature enhancement model for feature enhancement to obtain enhanced image features.

[0028] In this embodiment of the invention, in order to improve the feature enhancement effect of the preset feature enhancement model, it is necessary to pre-train and construct the preset feature enhancement model. Based on this, the method includes: constructing a preset initial feature enhancement model; obtaining a sample dataset, wherein the sample dataset includes image features of sample irregular images with enhanced image feature labels; dividing the sample dataset into a training set and a test set; training the preset initial feature enhancement model using the training set; testing the trained preset initial feature enhancement model using the test set; and finally, using the trained preset initial feature enhancement model that meets the test conditions as the preset feature enhancement model.

[0029] Specifically, during model training, a pre-defined initial feature enhancement model is first constructed, followed by the acquisition of a sample dataset. The dataset must contain all necessary files. The data is then converted to a format understandable by the pre-defined initial feature enhancement model. Finally, the model is trained and tested. Specifically, the dataset can be divided first: using randomness or a specific strategy (such as stratified sampling), the sample dataset is divided into training and test sets. The model is then trained using the training set, and tested using the test set to evaluate its performance on unseen data. Metrics such as mCP, precision, and recall on the test set are calculated and recorded. If the model performance does not meet requirements, the training phase can be returned for further iterations or adjustments. This process yields a pre-defined feature enhancement model that meets the requirements. For example, multi-line text images (including PDF scans of documents such as maintenance tickets and channel verification forms, as well as on-site record images) from the power communication field were collected. These images covered layouts such as nested tables and multi-paragraph mixed layouts, and included fonts such as Song, Kai, and cursive handwriting. They also covered interference scenarios such as tilting within 15°, blurred scans, and backlighting. A sample dataset was constructed through full-scene cross-media data collection, refined annotation, and balanced partitioning. The actual image features of the data in this sample dataset were determined and annotated. Then, based on stratified sampling, the sample dataset was divided into training, validation, and test sets according to the proportions determined by actual needs. Stratified sampling ensured that each subset was consistent in terms of text type (printed / handwritten), layout mode (table / paragraph), and interference level. During model training, for example, the model training parameters are configured as follows: optimizer selection, weight decay coefficient 0.01, initial learning rate 1e-4, using cosine annealing learning rate scheduling (T_max=50, η_min=1e-6); batch processing: training set batch size = 32, test set batch size = 16, using 8 threads to load data in parallel to improve input and output efficiency; regularization: adding a regularization term (probability 0.3) after the convolutional layer, data augmentation including random rotation (±15°), Gaussian blur (kernel size 3×3), contrast adjustment (0.8-1.2 times); training epochs: a total of 50 training epochs, using an early stopping strategy (training is terminated if the training set loss does not decrease for 5 consecutive epochs), saving model weights and calculating character accuracy in each epoch; loss function: combined with label smoothing (coefficient 0.1), adding 1.5 times weight to electrical symbols.

[0030] Furthermore, after training and constructing the preset feature enhancement model, it is necessary to use the preset feature enhancement model to enhance the text features of the initial features. Based on this, step 102 specifically includes: obtaining the preset feature enhancement model, wherein the preset feature enhancement model includes a first cascaded convolutional layer, a second cascaded convolutional layer, a third cascaded convolutional layer, and a fourth cascaded convolutional layer for convolutional feature extraction; a first spatial attention layer and a second spatial attention layer for enhancing the text region features in the image; and a multi-scale feature fusion layer for fusing image features; inputting the initial image features into the preset feature enhancement model, and sequentially extracting the convolutional features from the initial image features through the first cascaded convolutional layer, the second cascaded convolutional layer, the third cascaded convolutional layer, and the fourth cascaded convolutional layer, corresponding to... The output features of the first, second, third, and fourth cascaded convolutional layers are obtained. Text region feature enhancement is performed on the output features of the third and fourth cascaded convolutional layers through the first and second spatial attention layers, resulting in text enhancement features output by the first and second spatial attention layers. Finally, the multi-scale feature fusion layer fuses these output features to obtain the enhanced image features.

[0031] Specifically, the preset feature enhancement model includes a hierarchical convolutional feature extraction network, a spatial attention mechanism, and a multi-scale feature fusion layer. The hierarchical convolutional feature extraction network consists of a 4-layer cascaded convolutional-pooling structure (first cascaded convolutional layer, second cascaded convolutional layer, third cascaded convolutional layer, and fourth cascaded convolutional layer). For example, the first to fourth cascaded convolutional layers use 64, 128, 256, and 512 channels respectively. Each layer uses a 3×3 convolution (stride 1, padding 1) + BN (Batch Normalization Layer) + ReLU activation function, combined with 2×2 max pooling (stride 2) to halve the spatial resolution, finally outputting a 32×32×512 feature map. The spatial attention mechanism embeds a spatial attention layer after the third and fourth cascaded convolutional layers, generating a spatial weight mask through global average pooling to enhance the feature weights of text regions and suppress background noise. Multi-scale feature fusion layer: The features of each layer are fused with upsampling through lateral connection to obtain enhanced image features. By enhancing the text features of the image in the above way, layout features such as line spacing and arrangement direction can be preserved, while highlighting the text features in the image, thereby improving the accuracy and efficiency of subsequent text recognition. Meanwhile, the kernel size of convolutional layers 1-4 is the same, such as 3×3, with a stride of 1 and padding of 1, ensuring that the feature map size is continuously halved. The number of channels increases by multiples, such as 64, 128, 256, and 512 respectively. Overfitting is suppressed by BN layers and ReLU activation function, and the final output is a high-dimensional feature map of a preset dimension, such as 32×32×512, which focuses on the semantic features of text regions, thereby improving the effect of text feature enhancement on the image. An attention layer is inserted after the convolutional layers 3 and 4. The feature map is compressed into a single-channel spatial vector by global average pooling. After convolution (output channel 1) and activation function, a weight mask of a preset interval (such as 0-1) is generated. The mask is multiplied point by point with the original feature map, which increases the feature weight of the text region by several times and effectively suppresses the interference of table lines and background noise in the image.

[0032] 103. Input the enhanced image features into the preset image content recognition model to perform image content recognition with variable scale features, obtain the initial recognition result, and align the initial recognition result with the content in the irregular power communication cross-media data image.

[0033] In this embodiment of the invention, a preset image content recognition model is pre-trained and constructed. After constructing the preset feature enhancement model and the preset image content recognition model, in order to reduce computational resources and improve the model prediction speed, the model can be lightweighted. Based on this, the method includes: taking the preset feature enhancement model and the preset image content recognition model as target models respectively; determining the current model parameters of the target models; and obtaining a sample model pruning dataset, wherein the sample model pruning dataset includes sample input data with actual output data labels that are adapted to the prediction scenario of the target models; determining the overfitting constraint term of the target models based on the current model parameters; and determining the overfitting constraint term of the target models according to the actual output data corresponding to the sample input data. The cross-entropy between the data labels and the predicted output data corresponding to the sample input data predicted by the target model is used to determine the initial loss function of the target model. The overfitting constraint term is added to the initial loss function to obtain the target loss function. The target loss function is then used to iteratively train the channel parameters of each network layer in the target model to obtain the channel parameters of each channel in each network layer after the final iterative training. Any network layer in each network layer is taken as a target network layer. Based on the absolute value of the channel parameters of each channel in the target network layer, the channel to be pruned is determined in the target network layer, and the channel to be pruned is removed in the target network layer to obtain the pruned target model.

[0034] Specifically, taking the preset feature enhancement model and the preset image content recognition model as examples of target models, the model parameters can be the weights of each network layer in the target model. Specifically, firstly, the sample input data from the sample dataset (if the target model is the preset feature enhancement model, then the sample data is the initial image features of the irregular power image, and the actual output data is the text-enhanced image features; if the target model is the preset image content recognition model, then the sample input data is the text-enhanced image features corresponding to the irregular power image, and the actual output data is the text recognition result) is input into the target model to predict the output data, obtaining the predicted output data. Based on the difference between the predicted output data and the actual output data corresponding to the same sample input data, the initial loss function L0 is determined, and then the target loss function L is determined according to the following formula:

[0035] in, The strength parameter of the overfitting constraint term, Let be the i-th model parameter. Finally, the target model is trained iteratively multiple times using the target loss function and the sample dataset. Iterative training mainly focuses on the channel parameters of each network layer of the target model, determining the channel parameters of each channel in each network layer after the last iteration. For each network layer, a target channel parameter with an absolute value less than a preset threshold (set according to actual needs) is identified among the channel parameters of its corresponding channels. The channel corresponding to the target channel parameter is then removed, and the model after removing this channel is used as the pruned target model. This embodiment of the invention uses an overfitting constraint term to constrain the loss function, enabling the model to minimize the original loss function during training while maintaining the simplicity of the large model. This helps prevent the large model from overfitting to noise and details in the training data, thereby improving the model's generalization ability on unseen data. In other words, by constraining the complexity of the model parameters, the model becomes more robust and less susceptible to random fluctuations in the training data, thus improving the model's robustness. Simultaneously, by pruning the model, its computational complexity and inference time can be reduced, thereby improving the model's prediction efficiency.

[0036] Furthermore, after pruning the preset feature enhancement model and the preset image content recognition model, the initial image features are input into the pruned preset feature enhancement model for feature enhancement to obtain enhanced image features. Then, the enhanced image features need to be input into the pruned preset image content recognition model for image content recognition and text sequence alignment. Based on this, step 103 specifically includes: determining the height direction of the enhanced image features, and performing horizontal global average pooling on the enhanced image features according to the height direction, and using the pooled enhanced image features as one-dimensional sequence features. The one-dimensional sequence features are mapped to a preset number of character spaces using a fully connected layer in the preset image content recognition model. Based on the mapping result, an initial recognition result is determined, and this initial recognition result is used as the predicted text sequence in the irregular power communication cross-media data image. A label text sequence in the irregular power communication cross-media data image is determined, and a matching path is determined between the predicted text sequence and the label text sequence. The predicted text sequence is reordered based on each matching path to obtain the reordered predicted text sequence. The confidence level corresponding to each reordered predicted text sequence is determined, and based on the confidence level, a target predicted text sequence is selected as the aligned initial recognition result in each reordered predicted text sequence.

[0037] The preset number of categories is set according to actual needs. Specifically, enhanced image features, such as a 32×32×512 two-dimensional feature map, are compressed into a one-dimensional sequence feature of preset length, such as W=32, by horizontal global average pooling along the height direction. The one-dimensional sequence feature is then input into a fully connected layer with a preset number of neurons (such as 128) and mapped to a preset number of categories, such as 62 character spaces (e.g., 62 character spaces contain 10 digits, 26 uppercase and lowercase letters, and 26 power-specific symbols (such as "kV", "Ω", "optical cable")). The fully connected layer outputs a predicted text sequence of preset length, such as M=100 (initial recognition result). Dynamic alignment mechanism: Allows the insertion of whitespace "-" (used to separate multi-line text or for error tolerance) into the predicted text sequence. The Viterbi algorithm (a subset of dynamic programming) is used to solve the matching path between the predicted text sequence and the actual label text sequence of the non-regular power communication cross-media data image, eliminating alignment errors caused by disordered line order in multi-line text (such as reversed rows and columns in a table). Decoding optimization: A dynamic programming algorithm (beam_size=3) is used to generate multiple candidate text sequences (predicted text sequences) based on the matching path. The optimal predicted text sequence (target predicted text sequence) is selected by ranking by confidence. At the same time, a whitespace probability threshold is set, such as whitespace probability threshold ≤0.2, to reduce the interference of redundant characters on the structured text output.

[0038] For example, structured label text sequences can be extracted from irregular power communication images using manual annotation or pre-trained layout analysis models. For instance, for an inspection image containing a three-line structure of "device name-status-time", the label text sequence is defined as a sequential combination of [device name line, status line, time line]. A pre-defined image content recognition model is used for initial image recognition, outputting a predicted text sequence, such as [status line, device name line, time line]. A two-dimensional matrix C[i][j] is constructed, where i is the i-th row of the label text sequence and j is the j-th row of the predicted text sequence. The matrix element C[i][j] represents the matching cost between the i-th label row and the j-th predicted row, calculated as: C[i][j] = 1 - character matching rate (label row i, predicted row j) + row order misalignment penalty. The character matching rate is calculated using edit distance or longest common subsequence, and the row order misalignment penalty increases linearly according to |ij|. A dynamic programming algorithm is used to find the path with the minimum cumulative cost from the top left corner (0,0) to the bottom right corner (N,M) of the matrix. The path must satisfy the monotonicity constraint (movement is only allowed to the right or down) to ensure the rationality of the line order adjustment. For example, for a label sequence length N=3 and a prediction sequence length M=3, the optimal path might be (0,0)→(1,1)→(2,2), indicating that the line order is completely consistent; if there is a misalignment, the path might be (0,1)→(1,2)→(2,2), indicating that the first line of the prediction sequence needs to be adjusted to the second line. Based on the coordinate mapping relationship of the optimal path, the line order of the predicted text sequence is adjusted to be consistent with the label sequence. For example, if the path is (0,1)→(1,2)→(2,2), then the first line of the prediction sequence (original index 0) is moved to the second line (new index 1), and subsequent lines are moved sequentially to obtain the adjusted sequence [predicted line 1, predicted line 0, predicted line 2]. Line padding and cropping: If the predicted sequence length is inconsistent with the label sequence (e.g., label length N=3, predicted length M=2), fill in blank lines or crop redundant lines after adjustment to ensure that the adjusted sequence has the same length as the label sequence. For each adjusted predicted sequence, calculate the comprehensive confidence score Conf, using the formula: Conf = α·character matching rate + β·line order consistency rate + γ·language model score, where α, β, and γ are weighting coefficients (e.g., α=0.5, β=0.3, γ=0.2). The language model score is calculated using a pre-trained power domain language model (e.g., BERT-Power) to assess the sequence's rationality and professionalism. Select the adjusted sequence with the highest confidence score as the final output. If the highest confidence score is lower than a preset threshold (e.g., 0.7), trigger manual review or revert to the original predicted text sequence for re-alignment. This embodiment of the invention, by aligning the text sequence, avoids problems such as disordered text line order, thereby improving the accuracy of text recognition for images.

[0039] 104. Perform neighborhood vocabulary correction on the initial recognition results after sequence alignment, and determine the structured text data corresponding to the irregular power communication cross-media data image based on the corrected initial recognition results.

[0040] Specifically, the initial recognition results after sequence alignment undergo lexical correction within the power industry domain, such as adding a "specialized symbol replacement" operation (e.g., randomly replacing "V" with "kV" and "A" with "mA"). Finally, the text sequence corrected with neighborhood lexical correction is used as structured text data. This embodiment of the invention, by performing domain-specific lexical correction on the text, ensures that the final recognized structured text better meets the needs of the corresponding domain. This embodiment of the invention, through the combination of a preset feature enhancement model and a preset image content recognition model, combined with a power industry-specific adaptation strategy, achieves high-precision and efficient recognition of cross-media data, providing core technical support for the intelligent upgrading of power communication management systems and possessing significant engineering application value.

[0041] According to the present invention, a method for identifying cross-media data in power communication is provided. Compared with the current method of directly extracting text content from irregular data using fixed-scale features, the present invention inputs the initial image features of the irregular cross-media data image of power communication into a preset feature enhancement model for feature enhancement to obtain enhanced image features; then inputs the enhanced image features into a preset image content recognition model for image content recognition using variable-scale features to obtain an initial recognition result; and then performs sequence alignment between the initial recognition result and the content in the irregular cross-media data image of power communication; finally, the sequence-aligned initial recognition result is subjected to neighborhood vocabulary correction, and based on the corrected initial recognition result, the structured text data corresponding to the irregular cross-media data image of power communication is determined. Therefore, by pre-enhancing image features of irregular power communication cross-media data images, the influence of blurry images, tilted shooting angles, and scanning blur on text recognition can be eliminated, thereby improving the accuracy of text recognition in irregular power communication cross-media data images. The variable scale of the preset image content recognition model can adapt to arbitrary line spacing and text size in the image, thus improving text recognition accuracy. By aligning the text recognition results of the preset image content recognition model with the text content in the irregular power communication cross-media data image, problems such as line order disorder and character misalignment can be avoided, further improving text recognition accuracy. Neighborhood vocabulary correction of the text recognition results can correct erroneous words in the recognition results, further improving text recognition accuracy. This ensures the communication and application effects of subsequent power data.

[0042] Furthermore, to better illustrate the above process of identifying cross-media data in power communication, as a refinement and extension of the above embodiments, this invention provides another method for identifying cross-media data in power communication, such as... Figure 2 As shown, the method includes: 201. Acquire non-uniform power communication cross-media data images and determine the initial image features of non-uniform power communication cross-media data images.

[0043] Specifically, irregularity refers to issues such as image tilt, blurriness, inconsistent font sizes, and inconsistent line spacing. Image feature extraction models, such as CNN models, can be used to extract initial image features from irregular cross-media data images in power communication.

[0044] 202. Input the initial image features into the preset feature enhancement model for feature enhancement to obtain enhanced image features.

[0045] 203. Construct a preset initial image content recognition model.

[0046] 204. Obtain the sample dataset, which includes the sequence features of sample irregular images with image text content labels in a preset dimension.

[0047] 205. Divide the sample dataset into a training set and a test set. Use the training set to train the preset initial image content recognition model, and use the test set to test the trained preset initial image content recognition model. Finally, use the trained preset initial image content recognition model that meets the test conditions as the preset image content recognition model.

[0048] 206. Input the enhanced image features into the preset image content recognition model to perform image content recognition with variable scale features, obtain the initial recognition result, and align the initial recognition result with the content in the irregular power communication cross-media data image.

[0049] In this embodiment of the invention, to improve the text content recognition accuracy of the preset image content recognition model, it is necessary to train and construct the preset image content recognition model. During model training, firstly, a preset initial image content recognition model is constructed, and secondly, a sample dataset is obtained. Ensure the dataset contains all necessary files. Convert the data to a format understandable by the preset initial image content recognition model, and finally train and test the model. Specifically, the dataset can be divided first: using random or specific strategies (such as stratified sampling), the sample dataset is divided into a training set and a test set. The model is then trained using the training set, and tested using the test set to evaluate its performance on unseen data. Metrics such as mCP, precision, and recall on the test set are calculated and recorded. If the model performance does not meet the requirements, it can return to the training phase for further iterations or adjustments. This yields a preset image content recognition model with satisfactory accuracy. Afterwards, redundant convolutional kernels are removed through channel pruning (e.g., pruning based on L1 regularized importance scoring), and the model parameters are simplified. For example, the feature extraction layer can use FP16 half-precision computation, and the decoding layer can use INT8 integer computation. Combined with fusion layer technology, the inference speed of a single GPU will be improved, meeting the real-time processing requirements of power systems. It should be noted that the above examples are merely illustrative and do not specifically limit the embodiments of the present invention.

[0050] In this embodiment of the invention, an irregular power image recognition system can be used to convert irregular cross-media power communication data images into structured text data. This system comprises a four-layer structure: a data input layer, a core processing layer, an inference optimization layer, and a result output layer. The data input layer receives the cross-media text image, images it, and outputs a standardized feature tensor; the core processing layer includes three modules: multi-line text feature enhancement, dynamic sequence alignment, and power domain adaptation; the inference optimization layer outputs results through model lightweighting and accuracy optimization; the result output layer generates structured text and labeled images and pushes them to the power communication management system. Figure 3 The diagram shows the structure of the multi-line text image feature enhancement module (i.e., the structural diagram of the preset feature enhancement model). The diagram shows the collaborative mechanism of hierarchical convolution and spatial attention: the four-layer cascaded convolution-pooling structure sequentially realizes feature extraction and resolution compression. The spatial attention layer embedded in the third and fourth layers enhances the text region features by generating weight masks. Finally, the enhanced feature map is output through multi-scale fusion. Figure 4 The workflow of the spatial attention layer is demonstrated. The spatial vector is obtained by global average pooling of the convolution output feature map. The vector is then generated by 1×1 convolution and sigmoid activation. The mask is multiplied point by point with the original feature map to highlight the features of the text region and suppress background noise. Figure 5The process of temporalization and decoding of two-dimensional features (i.e., a schematic diagram of the workflow of a preset image content recognition model) is demonstrated: the two-dimensional feature map is converted into a one-dimensional sequence through lateral global pooling, input to a fully connected layer and mapped to the character space. Dynamic alignment with the label sequence is achieved through the loss function of a preset image content recognition model, such as CTC (Connectionist Temporal Classification). Finally, the output recognition result is optimized by BeamSearch (a search strategy). Loss function optimization: Based on the loss function of the preset image content recognition model, a domain weight factor is introduced. Higher loss weights are assigned to electrical industry symbols (such as "Ω" and "km") and terminology characters, while lower weights are assigned to ordinary characters. Gradient backpropagation strengthens the model's focus on key domain information. The hardware environment of the aforementioned non-uniform power image recognition system can include: training end: an operating system, a deep learning framework, a computing platform, and an A100 GPU (40GB of VRAM) for model training; inference end: a power communication management system server (Intel Xeon Gold 6330 CPU, RTX 3090 GPU), deploying the TensorRT 8.6 inference engine to accelerate real-time inference.

[0051] 207. Perform neighborhood vocabulary correction on the initial recognition results after sequence alignment, and determine the structured text data corresponding to the irregular power communication cross-media data image based on the corrected initial recognition results.

[0052] For embodiments of the present invention, it is also necessary to perform neighborhood vocabulary correction on the text content recognition results after sequence alignment. Based on this, step 2074 specifically includes: determining the data domain to which the irregular power communication cross-media data image belongs; performing word segmentation on the aligned initial recognition results to obtain each word segment, and determining the neighborhood vocabulary to be corrected in each word segment based on the data domain; obtaining a preset dictionary corresponding to the data neighborhood, and performing similarity matching between the neighborhood vocabulary to be corrected and each standard vocabulary in the preset dictionary, and determining the target standard vocabulary for correcting the neighborhood vocabulary to be corrected in the preset dictionary based on the similarity matching results; and performing vocabulary correction on the neighborhood vocabulary to be corrected using the target standard vocabulary.

[0053] Specifically, the domain of an image in an irregular cross-media data image of power communication is determined based on its text content. For example, if the image contains features such as "SCADA (Supervisory Control and Data Acquisition) system" or "RTU (Remote Terminal Unit) device," it is classified as belonging to the "power monitoring and communication domain." If the image contains features such as "optical cable attenuation" or "OTDR (Optical Time-Domain Reflectometer) waveform," it is classified as belonging to the "fiber optic communication domain." The words to be corrected are identified from each segment of the initial recognition result (text content) after alignment. For example, if the co-occurrence frequency of a word in power communication domain documents is higher than a threshold (e.g., 0.7), it is marked as a neighboring word to be corrected. Standard terms in the power communication domain are collected, and a pre-defined dictionary is constructed from these standard terms. For example, this pre-defined dictionary can be a Trie tree structure. Next, the similarity between the neighboring words to be corrected and each standard word in the preset dictionary is calculated. Based on the similarity, the standard words needed to correct the neighboring words are found. For example, if the neighboring word to be corrected is "pitfall," its corresponding standard word is "impedance." Then, the standard words are used to correct the neighboring words, i.e., "pitfall" is corrected to "impedance." Finally, the text with corrected words is used as the structured text corresponding to the irregular cross-media data image of power communication. This invention improves the accuracy of power terminology recognition through domain adaptation optimization, providing an efficient solution for the automated processing of cross-media data such as maintenance tickets and verification forms in power communication management systems, significantly promoting the intelligent transformation of power management.

[0054] According to another method for identifying cross-media data in power communication provided by the present invention, compared with the current method of directly extracting text content from irregular data using fixed-scale features, the present invention inputs the initial image features of the irregular cross-media data image of power communication into a preset feature enhancement model for feature enhancement to obtain enhanced image features; then inputs the enhanced image features into a preset image content recognition model for image content recognition using variable-scale features to obtain an initial recognition result; and then performs sequence alignment between the initial recognition result and the content in the irregular cross-media data image of power communication; finally, performs neighborhood vocabulary correction on the sequence-aligned initial recognition result, and determines the structured text data corresponding to the irregular cross-media data image of power communication based on the corrected initial recognition result. Therefore, by pre-enhancing image features of irregular power communication cross-media data images, the influence of blurry images, tilted shooting angles, and scanning blur on text recognition can be eliminated, thereby improving the accuracy of text recognition in irregular power communication cross-media data images. The variable scale of the preset image content recognition model can adapt to arbitrary line spacing and text size in the image, thus improving text recognition accuracy. By aligning the text recognition results of the preset image content recognition model with the text content in the irregular power communication cross-media data image, problems such as line order disorder and character misalignment can be avoided, further improving text recognition accuracy. Neighborhood vocabulary correction of the text recognition results can correct erroneous words in the recognition results, further improving text recognition accuracy. This ensures the communication and application effects of subsequent power data.

[0055] Furthermore, as Figure 1 In specific implementation, embodiments of the present invention provide a cross-media data identification device for power communication, such as... Figure 6 As shown, the device includes: an acquisition unit 31, a feature enhancement unit 32, a sequence alignment unit 33, and a text determination unit 34.

[0056] The acquisition unit 31 can be used to acquire irregular power communication cross-media data images and determine the initial image features of the irregular power communication cross-media data images.

[0057] The feature enhancement unit 32 can be used to input the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features.

[0058] The sequence alignment unit 33 can be used to input the enhanced image features into a preset image content recognition model to perform image content recognition of variable scale features, obtain an initial recognition result, and perform sequence alignment of the initial recognition result with the content in the irregular power communication cross-media data image.

[0059] The text determination unit 34 can be used to perform neighborhood word correction on the initial recognition result after sequence alignment, and determine the structured text data corresponding to the irregular power communication cross-media data image based on the corrected initial recognition result.

[0060] In specific application scenarios, in order to enhance image features, such as... Figure 7 As shown, the feature enhancement unit 32 includes an acquisition module 321 and a feature enhancement module 322.

[0061] The acquisition module 321 can be used to acquire a preset feature enhancement model, wherein the preset feature enhancement model includes a first cascaded convolutional layer, a second cascaded convolutional layer, a third cascaded convolutional layer, and a fourth cascaded convolutional layer for convolutional feature extraction, a first spatial attention layer and a second spatial attention layer for enhancing the features of text regions in the image, and a multi-scale feature fusion layer for fusing image features.

[0062] The feature enhancement module 32 can be used to input the initial image features into a preset feature enhancement model, and sequentially extract the convolutional features from the initial image features through the first cascaded convolutional layer, the second cascaded convolutional layer, the third cascaded convolutional layer, and the fourth cascaded convolutional layer, respectively, to obtain the output features of the first cascaded convolutional layer, the second cascaded convolutional layer, the third cascaded convolutional layer, and the fourth cascaded convolutional layer. The first spatial attention layer and the second spatial attention layer respectively perform text region feature enhancement on the output features of the third cascaded convolutional layer and the fourth cascaded convolutional layer, respectively, to obtain the text enhancement features output by the first spatial attention layer and the second spatial attention layer. The multi-scale feature fusion layer fuses the output features of the first cascaded convolutional layer, the second cascaded convolutional layer, the text enhancement features output by the first spatial attention layer, and the text enhancement features output by the second spatial attention layer to obtain the enhanced image features.

[0063] In specific application scenarios, in order to construct a preset image content recognition model, the device also includes a construction unit 35.

[0064] The construction unit 35 can be used to construct a preset initial image content recognition model; obtain a sample dataset, wherein the sample dataset includes sequence features of a preset dimension of sample irregular images with image text content labels; divide the sample dataset into a training set and a test set, use the training set to train the preset initial image content recognition model, and use the test set to test the trained preset initial image content recognition model, and finally use the trained preset initial image content recognition model that meets the test conditions as the preset image content recognition model.

[0065] In specific application scenarios, in order to perform image content recognition and sequence alignment using a preset image content recognition model, the sequence alignment unit 33 includes a pooling module 331, a mapping module 332, a first determination module 333, a sequence adjustment module 334, and a selection module 335.

[0066] The pooling module 331 can be used to determine the height direction of the enhanced image features, and perform horizontal global average pooling on the enhanced image features according to the height direction, and use the pooled enhanced image features as one-dimensional sequence features.

[0067] The mapping module 332 can be used to map the one-dimensional sequence features to a preset number of character spaces using the fully connected layer in the preset image content recognition model, determine the initial recognition result based on the mapping result, and use the initial recognition result as the predicted text sequence in the irregular power communication cross-media data image. The first determining module 333 can be used to determine the tag text sequence in the irregular power communication cross-media data image and to determine the matching path between the predicted text sequence and the tag text sequence.

[0068] The sequence adjustment module 334 can be used to adjust the line order of the predicted text sequence based on each of the matching paths, so as to obtain the predicted text sequence after line order adjustment.

[0069] The selection module 335 can be used to determine the confidence level corresponding to each line-order adjusted predicted text sequence, and based on the confidence level, select the target predicted text sequence as the aligned initial recognition result in each line-order adjusted predicted text sequence.

[0070] In specific application scenarios, in order to perform neighborhood word correction on the initial recognition results after sequence alignment, the text determination unit 34 includes a second determination module 341, a word segmentation module 342, a matching module 343, and a correction module 344.

[0071] The second determining module 341 can be used to determine the data domain to which the irregular power communication cross-media data image belongs.

[0072] The word segmentation module 342 can be used to segment the aligned initial recognition result to obtain each word, and based on the data domain, determine the neighboring words to be corrected in each word.

[0073] The matching module 34 can be used to obtain a preset dictionary corresponding to the data neighborhood, and perform similarity matching between the neighborhood words to be corrected and each standard word in the preset dictionary. Based on the similarity matching result, the target standard words for correcting the neighborhood words to be corrected are determined in the preset dictionary.

[0074] The correction module 344 can be used to perform vocabulary correction on the neighboring vocabulary to be corrected using the target standard vocabulary.

[0075] In specific application scenarios, in order to perform lightweight processing on the preset feature enhancement model and the preset image content recognition model respectively, the device further includes: a pruning unit 36.

[0076] The pruning unit 36 ​​can be used to take the preset feature enhancement model and the preset image content recognition model as target models, determine the current model parameters of the target models, and obtain a sample model pruning dataset, wherein the sample model pruning dataset includes sample input data with actual output data labels that are adapted to the prediction scene of the target model; determine the overfitting constraint term of the target model based on the current model parameters; determine the initial loss function of the target model based on the cross-entropy between the actual output data labels corresponding to the sample input data and the predicted output data corresponding to the sample input data predicted by the target model; add the overfitting constraint term to the initial loss function to obtain the target loss function, and use the target loss function to iteratively train the channel parameters of each network layer in the target model to obtain the channel parameters of each channel in each network layer after the final iterative training; take any network layer in each network layer as a target network layer, determine the channel to be pruned in the target network layer based on the absolute value of the channel parameters of each channel in the target network layer, and remove the channel to be pruned in the target network layer to obtain the pruned target model.

[0077] In specific application scenarios, in order to use the pruned model for feature enhancement and content recognition, the feature enhancement unit 32 can also be used to input the initial image features into the pruned preset feature enhancement model for feature enhancement to obtain enhanced image features.

[0078] The sequence alignment unit 33 can also be used to input the enhanced image features into the pruned preset image content recognition model to perform image content recognition and obtain an initial recognition result.

[0079] In specific application scenarios, in order to perform anti-interference processing on irregular power communication cross-media data images, the device also includes an anti-interference unit 37.

[0080] The anti-interference unit 37 can be used to determine the generation feature information of the irregular power communication cross-media data image, wherein the generation feature information includes the underexposure degree, illumination uniformity, and scanning attribute information of the irregular power communication cross-media data image; based on the generation feature information, determine the curve constraint conditions of the anti-interference adjustment curve to be generated, wherein the curve constraint conditions include the pixel value range of the image after adjustment based on the anti-interference adjustment curve to be generated, the monotonicity and retransmissionability of the anti-interference adjustment curve to be generated; based on the curve constraint conditions, construct an anti-interference adjustment curve for anti-interference adjustment of the irregular power communication cross-media data image; use the anti-interference adjustment curve to adjust the interference during the generation of the irregular power communication cross-media data image to obtain the interference-adjusted irregular power communication cross-media data image.

[0081] In specific application scenarios, in order to determine the initial image features, the acquisition unit 31 can also be used to determine the initial image features of the irregular power communication cross-media data image after interference adjustment.

[0082] It should be noted that other corresponding descriptions of the functional modules involved in the cross-media data identification device for power communication provided in this embodiment of the invention can be found in [reference]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0083] Based on the above, Figure 1 Accordingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: acquiring an irregular cross-media data image of power communication and determining initial image features of the irregular cross-media data image; inputting the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features; inputting the enhanced image features into a preset image content recognition model for image content recognition of variable scale features to obtain an initial recognition result, and performing sequence alignment of the initial recognition result with the content in the irregular cross-media data image of power communication; performing neighborhood vocabulary correction on the sequence-aligned initial recognition result, and determining the structured text data corresponding to the irregular cross-media data image of power communication based on the corrected initial recognition result.

[0084] Based on the above, Figure 1 The method shown and as Figure 6 The embodiment of the device shown in the invention also provides a physical structure diagram of a computer device, such as... Figure 8 As shown, the computer device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor. Both the memory 42 and the processor 41 are mounted on a bus 43. When the processor 41 executes the program, it performs the following steps: acquiring an irregular cross-media data image of power communication and determining the initial image features of the irregular cross-media data image; inputting the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features; inputting the enhanced image features into a preset image content recognition model for variable-scale feature image content recognition to obtain an initial recognition result, and performing sequence alignment between the initial recognition result and the content in the irregular cross-media data image; performing neighborhood vocabulary correction on the sequence-aligned initial recognition result, and determining the structured text data corresponding to the irregular cross-media data image based on the corrected initial recognition result.

[0085] Through the technical solution of this invention, the initial image features of an irregular power communication cross-media data image are input into a preset feature enhancement model for feature enhancement to obtain enhanced image features; the enhanced image features are then input into a preset image content recognition model for variable-scale feature image content recognition to obtain an initial recognition result; the initial recognition result is then sequence-aligned with the content in the irregular power communication cross-media data image; finally, the sequence-aligned initial recognition result is subjected to neighborhood vocabulary correction, and based on the corrected initial recognition result, the structured text data corresponding to the irregular power communication cross-media data image is determined. Therefore, by pre-enhancing image features of irregular power communication cross-media data images, the influence of blurry images, tilted shooting angles, and scanning blur on text recognition can be eliminated, thereby improving the accuracy of text recognition in irregular power communication cross-media data images. The variable scale of the preset image content recognition model can adapt to arbitrary line spacing and text size in the image, thus improving text recognition accuracy. By aligning the text recognition results of the preset image content recognition model with the text content in the irregular power communication cross-media data image, problems such as line order disorder and character misalignment can be avoided, further improving text recognition accuracy. Neighborhood vocabulary correction of the text recognition results can correct erroneous words in the recognition results, further improving text recognition accuracy. This ensures the communication and application effects of subsequent power data.

[0086] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for cross-media data identification in power communication, characterized in that, include: Acquire irregular power communication cross-media data images and determine the initial image features of the irregular power communication cross-media data images; The initial image features are input into a preset feature enhancement model for feature enhancement to obtain enhanced image features; The enhanced image features are input into a preset image content recognition model to perform image content recognition with variable scale features, and an initial recognition result is obtained. The initial recognition result is then sequentially aligned with the content in the irregular power communication cross-media data image. The initial recognition results after sequence alignment are corrected for neighboring vocabulary, and the structured text data corresponding to the irregular power communication cross-media data image is determined based on the corrected initial recognition results.

2. The method according to claim 1, characterized in that, The initial image features are input into a preset feature enhancement model for feature enhancement to obtain enhanced image features, including: Obtain a preset feature enhancement model, wherein the preset feature enhancement model includes a first cascaded convolutional layer, a second cascaded convolutional layer, a third cascaded convolutional layer, and a fourth cascaded convolutional layer for convolutional feature extraction, a first spatial attention layer and a second spatial attention layer for enhancing text region features in an image, and a multi-scale feature fusion layer for fusing image features. The initial image features are input into a preset feature enhancement model. Convolutional features are extracted from the initial image features sequentially through a first cascaded convolutional layer, a second cascaded convolutional layer, a third cascaded convolutional layer, and a fourth cascaded convolutional layer, resulting in the output features of the first, second, third, and fourth cascaded convolutional layers, respectively. A first spatial attention layer and a second spatial attention layer are used to enhance the text region features of the output features of the third and fourth cascaded convolutional layers, resulting in the text enhancement features output by the first and second spatial attention layers. Finally, a multi-scale feature fusion layer fuses the output features of the first, second, and third cascaded convolutional layers to obtain the enhanced image features.

3. The method according to claim 1, characterized in that, Before inputting the sequence features of the preset dimension into the preset image content recognition model to perform image content recognition and obtain the initial recognition result, the method further includes: Construct a pre-defined initial image content recognition model; Obtain a sample dataset, wherein the sample dataset includes sequence features of sample irregular images with image text content labels of a preset dimension; The sample dataset is divided into a training set and a test set. The training set is used to train the preset initial image content recognition model, and the test set is used to test the trained preset initial image content recognition model. Finally, the trained preset initial image content recognition model that meets the test conditions is taken as the preset image content recognition model.

4. The method according to claim 1, characterized in that, The enhanced image features are input into a preset image content recognition model to perform variable-scale feature image content recognition, obtaining initial recognition results, including: Determine the height direction of the enhanced image features, and perform lateral global average pooling on the enhanced image features according to the height direction, and use the pooled enhanced image features as one-dimensional sequence features; The one-dimensional sequence features are mapped to a preset number of character spaces using the fully connected layer in the preset image content recognition model. The initial recognition result is determined based on the mapping result, and the initial recognition result is used as the predicted text sequence in the irregular power communication cross-media data image. The step of aligning the initial identification result with the content in the irregular power communication cross-media data image includes: Determine the tag text sequence in the irregular power communication cross-media data image, and determine the matching path between the predicted text sequence and the tag text sequence; The predicted text sequence is reordered based on each of the matching paths to obtain the reordered predicted text sequence. Determine the confidence level corresponding to each line-order adjusted predicted text sequence, and based on the confidence level, select the target predicted text sequence as the aligned initial recognition result in each line-order adjusted predicted text sequence.

5. The method according to claim 1, characterized in that, The initial recognition results after sequence alignment are then subjected to neighborhood vocabulary correction, including: Determine the data domain to which the irregular power communication cross-media data image belongs; The aligned initial recognition results are segmented to obtain individual words, and based on the data domain, neighboring words to be corrected are determined in each segment. Obtain a preset dictionary corresponding to the data neighborhood, and perform similarity matching between the words in the neighborhood to be corrected and each standard word in the preset dictionary. Based on the similarity matching results, determine the target standard words in the preset dictionary to correct the words in the neighborhood to be corrected. The target standard vocabulary is used to perform vocabulary correction on the neighboring vocabulary to be corrected.

6. The method according to claim 1, characterized in that, The method further includes: The preset feature enhancement model and the preset image content recognition model are respectively used as target models. The current model parameters of the target models are determined, and a sample model pruning dataset is obtained. The sample model pruning dataset includes sample input data with actual output data labels that are adapted to the prediction scene of the target model. Based on the current model parameters, determine the overfitting constraint terms of the target model; The initial loss function of the target model is determined based on the cross-entropy between the actual output data label corresponding to the sample input data and the predicted output data corresponding to the sample input data predicted by the target model. The overfitting constraint term is added to the initial loss function to obtain the target loss function. The target loss function is then used to iteratively train the channel parameters of each network layer in the target model to obtain the channel parameters of each channel in each network layer after the final iterative training. Each network layer is taken as a target network layer. Based on the absolute value of the channel parameter of each channel in the target network layer, the channel to be pruned is determined in the target network layer, and the channel to be pruned is removed in the target network layer to obtain the pruned target model. The step of inputting the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features includes: The initial image features are input into the pruned preset feature enhancement model for feature enhancement to obtain enhanced image features; The enhanced image features are input into a preset image content recognition model to perform variable-scale feature image content recognition, obtaining initial recognition results, including: The enhanced image features are input into the pruned preset image content recognition model to perform image content recognition and obtain the initial recognition result.

7. The method according to claim 1, characterized in that, Before determining the initial image features of the irregular power communication cross-media data image, the method further includes: The generation feature information of the irregular power communication cross-media data image is determined, wherein the generation feature information includes the underexposure degree, illumination uniformity, and scanning attribute information of the irregular power communication cross-media data image; Based on the generated feature information, the curve constraint conditions of the anti-interference adjustment curve to be generated are determined, wherein the curve constraint conditions include the pixel value range of the image after adjustment based on the anti-interference adjustment curve to be generated, the monotonicity and backpropagation of the anti-interference adjustment curve to be generated. Based on the curve constraints, an anti-interference adjustment curve is constructed for the anti-interference adjustment of the irregular power communication cross-media data image. The interference is adjusted during the generation of the irregular power communication cross-media data image using the anti-interference adjustment curve, resulting in the interference-adjusted irregular power communication cross-media data image. The determination of the initial image features of the irregular power communication cross-media data image includes: Determine the initial image features of the irregular power communication cross-media data image after interference adjustment.

8. A cross-media data identification device for power communication, characterized in that, include: An acquisition unit is used to acquire irregular power communication cross-media data images and determine the initial image features of the irregular power communication cross-media data images; The feature enhancement unit is used to input the initial image features into a preset feature enhancement model for feature enhancement to obtain enhanced image features; The sequence alignment unit is used to input the enhanced image features into a preset image content recognition model to perform image content recognition of variable scale features, obtain an initial recognition result, and perform sequence alignment of the initial recognition result with the content in the irregular power communication cross-media data image. The text determination unit is used to perform neighborhood vocabulary correction on the initial recognition result after sequence alignment, and determine the structured text data corresponding to the irregular power communication cross-media data image based on the corrected initial recognition result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images