Text controllable semantic perception image enhancement method for power equipment inspection

Abstract

The invention relates to a text controllable semantic perception image enhancement method and system for power equipment inspection, and the method comprises the steps: obtaining a to-be-enhanced inspection image collected by a power equipment inspection terminal, carrying out the feature extraction of the to-be-enhanced inspection image, and obtaining the image state information; and obtaining text control information corresponding to the inspection task, fusing the text control information and the image state information to generate a joint state representation, inputting the joint state representation into the trained parameter generation model, and outputting a low-dimensional image enhancement parameter vector for controlling the inspection image enhancement process by the parameter generation model. And constructing a parameterized mapping function according to the low-dimensional image enhancement parameter vector, performing pixel-level mapping processing on the to-be-enhanced inspection image by using the parameterized mapping function, and outputting an enhanced inspection image for defect detection or state evaluation of the power equipment as a final enhancement result. The method is suitable for efficient enhancement processing of the power equipment inspection image in a complex environment.

Description

Technical Field

[0001] This application relates to the fields of intelligent inspection of power systems and computer vision technology, and in particular to a text-controlled semantic perception image enhancement method for power equipment inspection. Background Technology

[0002] In power system operation and maintenance, vision-based inspection of power equipment is a crucial method for identifying potential equipment hazards. However, the quality of inspection images is often affected by complex lighting conditions, weather conditions, and dirt on the equipment itself, resulting in blurred defect features, insufficient contrast, heavy reliance on manual experience for judgment, and low efficiency, making it unsuitable for power equipment inspection in complex environments. Therefore, how to achieve efficient image enhancement processing for power equipment inspection in complex environments is an urgent problem to be solved. Summary of the Invention

[0003] Therefore, it is necessary to address the above problems by providing a text-controlled semantic perception image enhancement method, system, device, and storage medium for power equipment inspection that can be applied to power equipment inspection in complex environments.

[0004] The first aspect of this application provides a text-controlled semantic-aware image enhancement method for power equipment inspection, including: The system acquires the inspection image to be enhanced collected by the power equipment inspection terminal, extracts features from the inspection image to obtain image state information that characterizes brightness distribution, contrast statistics and color distribution; Obtain the text control information corresponding to the inspection task, and fuse the text control information with the image state information to generate a joint state representation; The joint state representation is input into the trained parameter generation model, which outputs a low-dimensional image enhancement parameter vector to control the inspection image enhancement process; the low-dimensional image enhancement parameter vector is used to characterize the pixel mapping relationship. A parameterized mapping function is constructed based on the low-dimensional image enhancement parameter vector; the parameterized mapping function includes a lookup table mapping function and / or a curve-based mapping function, and the parameterized mapping function satisfies the brightness mapping monotonicity constraint and the edge gradient preservation constraint; The parameterized mapping function is used to perform pixel-level mapping processing on the inspection image to be enhanced, and the enhanced inspection image is output as the final enhancement result; the enhanced inspection image is used for defect detection or condition assessment of power equipment.

[0005] In one embodiment, feature extraction is performed on the image to be enhanced to obtain image state information characterizing brightness distribution, contrast statistics, and color distribution, including: The image to be enhanced is preprocessed by normalizing its size and pixel values ​​to the [0,1] range; The preprocessed image is converted from the RGB color space to a luminance-chrominance separated color space. Image state features are extracted from the converted image, including at least one of the following: histogram distribution of the luminance channel, statistical moment features of the chrominance channel, and global contrast measure.

[0006] In one embodiment, the parameter generation model is constructed based on a maximum entropy reinforcement learning framework, including a policy network and a value function network; the policy network receives the joint state representation and outputs the conditional probability distribution of the low-dimensional image enhancement parameter vector; the optimization objective function of the parameter generation model is to maximize the weighted sum of the expected cumulative reward and the policy entropy.

[0007] In one embodiment, the low-dimensional image enhancement parameter vector includes: a sequence of control point coordinates for a Bézier curve used to define the brightness mapping; a vector of node values ​​used to fill a one-dimensional brightness lookup table or a three-dimensional color lookup table; and at least one of a contrast scaling factor and a saturation bias used to adjust global attributes.

[0008] In one embodiment, when constructing the nth-order Bézier curve mapping function, the low-dimensional image enhancement parameter vector contains n+1 control point coordinates. ,in , For the normalized input intensity, The corresponding output intensity; the normalized intensity value for any input pixel. x The output intensity value after mapping y Calculated using the following formula: ; ; in, It is a Bernstein basis polynomial. The coefficients are binomial coefficients.

[0009] In one embodiment, when constructing the three-dimensional lookup table mapping function, the low-dimensional image enhancement parameter vector defines the output color value of each grid vertex on a regular cubic grid in the RGB or YCbCr color space; the regular cubic grid is uniformly divided into L nodes in each color dimension, forming... Unit cube elements; for the input color vector The mapping process includes: Determine the unit cube cell containing the input vector, and read the predefined output color values ​​at the 8 vertices of that unit cube cell. ,in ; Calculate the input vector relative to the bottom left vertex of the unit cube cell. normalized offset The final output color value is calculated using trilinear interpolation. : ; ; in, These are the interpolation weights.

[0010] In one embodiment, the parameterized mapping function is used to perform pixel-level mapping processing on the inspection image to be enhanced, which is implemented by a dedicated hardware acceleration unit; the hardware acceleration unit includes: A configuration interface is provided to receive lookup table data from the software layer. High-speed on-chip memory is used to store the lookup table data; A parallel multi-path pixel processing engine is used to perform synchronous table lookup and interpolation operations on the input pixel stream; The output interface is used to output the enhanced pixel stream in video timing.

[0011] In one embodiment, the method further includes: During the deployment phase, the enhanced inspection images are evaluated, and the image enhancement parameters are adaptively adjusted online based on the evaluation results without updating the parameters of the parameter generation model, so that the enhancement results meet the preset multi-objective constraints.

[0012] In one embodiment, during the training phase of the parameter generation model, optimization is performed based on the reward signal calculated by the cross-modal semantic awareness model, including: The image encoder of the cross-modal semantic perception model extracts the visual feature vector of the enhanced output image. Text feature vectors of semantic instructions of the target text are extracted through a text encoder. ; Calculate visual feature vectors With text feature vectors The cosine similarity between them serves as the base value for the reward signal. : ; For base value Standardization is performed to obtain the reward signal used for training. : ,in and The moving average and standard deviation are calculated based on historical rewards. Using the reward signal r The parameters of the policy network are updated using a soft policy iterative algorithm. φ With the parameters of the value function network .

[0013] A second aspect of this application provides a text-controlled semantic-aware image enhancement system for power equipment inspection, comprising: The input module is used to acquire the inspection image to be enhanced collected by the power equipment inspection terminal, and to extract features from the inspection image to obtain image status information that characterizes the brightness distribution, contrast statistics and color distribution. The state representation and fusion module is used to acquire text control information corresponding to the inspection task, fuse the text control information with the image state information, and generate a joint state representation. A semantically guided reinforcement learning decision module is used to input the joint state representation into a trained parameter generation model, and the parameter generation model outputs a low-dimensional image enhancement parameter vector for controlling the inspection image enhancement process; the low-dimensional image enhancement parameter vector is used to represent pixel mapping relationships. A parameterized mapping function construction module is used to construct a parameterized mapping function based on the low-dimensional image enhancement parameter vector; the parameterized mapping function includes a lookup table mapping function and / or a curve-based mapping function, and the parameterized mapping function satisfies the brightness mapping monotonicity constraint and the edge gradient preservation constraint; The image enhancement execution module is used to perform pixel-level mapping processing on the inspection image to be enhanced using the parameterized mapping function, and output the enhanced inspection image as the final enhancement result; the enhanced inspection image is used for power equipment defect detection or condition assessment.

[0014] A third aspect of this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method.

[0015] A fourth aspect of this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0016] A fifth aspect of this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0017] The aforementioned text-controlled semantic-aware image enhancement method, system, device, and storage medium for power equipment inspection involves acquiring an inspection image to be enhanced, collected by a power equipment inspection terminal, extracting features from the image to obtain image state information representing brightness distribution, contrast statistics, and color distribution. Text control information corresponding to the inspection task is acquired and fused with the image state information to generate a joint state representation. This joint state representation is input into a trained parameter generation model, which outputs a low-dimensional image enhancement parameter vector to control the image enhancement process. This low-dimensional parameter vector represents pixel mapping relationships. A parameterized mapping function is constructed based on the low-dimensional parameter vector. This parameterized mapping function includes a lookup table mapping function and a curve-based mapping function, satisfying brightness mapping monotonicity constraints and edge gradient preservation constraints. The parameterized mapping function is then used to perform pixel-level mapping processing on the inspection image to be enhanced, outputting an enhanced inspection image for power equipment defect detection or condition assessment as the final enhancement result. By guiding the generation of image enhancement parameters through text semantics, the enhancement results are kept consistent with the semantic goals of the inspection task. This approach is highly real-time and suitable for efficient enhancement processing of power equipment inspection images in complex environments. Attached Figure Description

[0018] Figure 1 This is a flowchart of a text-controlled semantic-aware image enhancement method for power equipment inspection in one embodiment. Figure 2 This is a flowchart illustrating a text-controlled semantic-aware image enhancement method for power equipment inspection in one embodiment. Figure 3 This is a block diagram of a text-controlled semantic-aware image enhancement system for power equipment inspection in one embodiment. Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0021] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0022] Traditional image enhancement methods, such as histogram equalization, homomorphic filtering, or deep learning enhancement models designed for natural scenes, often lack prior knowledge of the special structure and defect features of power equipment. The enhancement effect is not targeted and may over-amplify noise or destroy key structural edge information, which is not conducive to subsequent automatic defect identification.

[0023] In recent years, intelligent image processing technology combined with natural language interaction has provided new ideas for inspection. However, general text-image editing methods, which aim to change image content or style, are not suitable for inspection enhancement tasks that require strict maintenance of equipment structural integrity and measurement consistency. In addition, such models are complex and computationally expensive, making them difficult to deploy on mobile inspection terminals with limited computing resources.

[0024] On the other hand, lookup tables (LUTs) and parameterized curves are widely used in industrial image processing pipelines due to their computational efficiency, determinism, and ease of implementation in embedded hardware or FPGAs. However, the parameters of traditional methods rely on manual tuning or simple global optimization, and they cannot intelligently understand semantic instructions from inspectors, such as "enhance the contrast of the heated area" or "highlight dirt on the insulator panel," and adaptively generate the optimal enhancement parameters.

[0025] Therefore, there is currently a lack of a dedicated image enhancement solution that can understand inspection semantic commands, adaptively enhance images based on the characteristics of power equipment images, and simultaneously meet the requirements of industrial-grade real-time performance and interpretability of processing results.

[0026] Based on this, this application provides a text-controlled semantic-aware image enhancement method for power equipment inspection, addressing the technical challenges of existing general enhancement technologies in simultaneously achieving semantic relevance to the inspection scenario, controllability and interpretability of the enhancement process, and real-time deployment at the edge. This method aims to construct a dedicated framework capable of understanding the inspector's natural language descriptions, automatically generating interpretable enhancement parameters, and efficiently executing on inspection terminals or edge servers. It combines cross-modal semantic understanding, maximum entropy reinforcement learning, and interpretable image processing operators to form a complete closed loop from inspection instructions to image enhancement.

[0027] In one embodiment, such as Figure 1 As shown, a text-controlled semantic-aware image enhancement method for power equipment inspection is provided, including: Step S110: Obtain the image to be enhanced from the power equipment inspection terminal, extract features from the image to be enhanced, and obtain image status information to characterize brightness distribution, contrast statistics, and color distribution.

[0028] The power equipment inspection terminal may include drones, robots, handheld terminals, etc., and the image to be enhanced contains information on the surface structure and defect features of the power equipment. After acquiring the image to be enhanced, feature extraction is performed on the image to obtain image state information used to characterize brightness distribution, contrast statistics, and color distribution.

[0029] Step S120: Obtain the text control information corresponding to the inspection task, and fuse the text control information with the image state information to generate a joint state representation.

[0030] The text control information is used to indicate the enhancement targets or display style of the inspection image, such as text descriptions like "enhance the oil level gauge reflection," "improve the contrast at the wear area of ​​the wire," and "suppress overexposure in the highlight area of ​​the porcelain insulator." The text control information is fused with the image state information to generate a joint state representation.

[0031] Step S130: Input the joint state representation into the trained parameter generation model, and output a low-dimensional image enhancement parameter vector to control the inspection image enhancement process.

[0032] The low-dimensional image enhancement parameter vector is used to represent pixel mapping relationships. The parameter generation model is built upon the SAC framework and employs a maximum entropy reinforcement learning framework, including a policy network. Sum-value function network. Policy network. Receive the joint state representation obtained in the previous step As input, the output is a low-dimensional image enhancement parameter vector. conditional probability distribution , These are the policy network parameters. The core idea of ​​the SAC algorithm is to maximize the expected cumulative reward while maximizing the policy entropy. Entropy represents the randomness of the policy; higher entropy means more thorough exploration, which helps discover more diverse yet equally rewarding enhancement schemes and avoids premature convergence to a suboptimal solution. In this embodiment, the optimization objective function of the parameter generation model is... To maximize the weighted sum of expected cumulative reward and policy entropy: ; in, As a reward signal, at time step t, the state is... And perform the action It was obtained later. As a discount factor, This is used to weigh the importance of current rewards against future rewards. The entropy temperature coefficient is an adjustable hyperparameter used to balance the weight between reward maximization and policy randomness. It is a strategy In a given state Entropy under a given state is used to measure the uncertainty of action selection in that state, and is defined as follows: . This indicates that the strategy is... Steady-state distribution of state-action pairs resulting from interaction with the environment.

[0033] Step S140: Construct a parameterized mapping function based on the low-dimensional image enhancement parameter vector. The parameterized mapping function includes a lookup table mapping function and / or a curve-based mapping function, and satisfies the brightness mapping monotonicity constraint and the edge gradient preservation constraint to ensure that the enhancement does not distort the original structure of the device.

[0034] Specifically, a specific enhancement mapping function is instantiated based on the low-dimensional image enhancement parameter vector output by the policy network. When constructing an n-order Bézier curve mapping function, the low-dimensional image enhancement parameter vector contains the coordinates of n+1 control points. ,in , For the normalized input intensity, This corresponds to the output intensity. For simplicity, input coordinates... Equal interval values ​​are typically used to output coordinates. This is then output by the policy network. Bézier curves provide a smooth, continuous, and easily controllable shape mapping method. For any pixel in the image, the normalized input intensity value... The output intensity value y, mapped through this curve, is obtained by weighted summation using Bernstein basis polynomials: ; Among them, parameters t Usually equal to the input value x , This refers to the i-th nth Bernstein basis polynomial, which determines the contribution weight of the i-th control point to the point on the curve corresponding to parameter t: ; These are the binomial coefficients, and their calculation formula is: ; When choosing to construct a 3D LUT mapping function, the low-dimensional image enhancement parameter vector defines the target color value that each vertex should output in a regularly discretized RGB or YCbCr color cube mesh. Assuming the lookup table uniformly divides each color dimension into L nodes, the entire cube is then divided into... There are a total of [number] small cube units. There are vertices. For any input color vector... The enhancement process follows these steps: First, locate the unit cube containing the input color; then, read the predefined output color values ​​at the eight vertices of the cube. Then, calculate the normalized 3D offset of the input color point within this unit cube. Finally, a trilinear interpolation algorithm is used to smoothly calculate the final output color value based on the output values ​​and offsets of the eight vertices. Trilinear interpolation is a natural extension of linear and bilinear interpolation in three-dimensional space, and its calculation formula is: ; Among them, interpolation weights Normalized offset of the input point within the cell The calculation formula is as follows: ; This method ensures the continuity and smoothness of color mapping across the entire spectrum.

[0035] Step S150: Use the parameterized mapping function to perform pixel-level mapping processing on the image to be enhanced, and output the enhanced image as the final enhancement result.

[0036] Among them, the enhanced inspection images are used for defect detection or condition assessment of power equipment. The parameter generation model is optimized based on the perceptual evaluation results of the inspection images during the training phase, and the model parameters are fixed during the inspection deployment phase, and only the parameterized mapping function is executed to output the enhanced inspection images.

[0037] The aforementioned text-controlled semantic-aware image enhancement method for power equipment inspection guides the generation of image enhancement parameters through text semantics, ensuring that the enhancement results are consistent with the semantic goals of the inspection task. During the deployment phase, it only relies on parameterized mapping functions to perform pixel-level operations, without requiring deep neural network inference. It has low computational complexity, strong real-time performance, and is suitable for efficient enhancement processing of power equipment inspection images in complex environments.

[0038] In one embodiment, step S110 involves extracting features from the image to be enhanced to obtain image state information that characterizes brightness distribution, contrast statistics, and color distribution, including steps 112 to 116.

[0039] Step 112: Perform preprocessing on the image to be enhanced by normalizing its size and pixel values ​​to the [0,1] range. Size normalization ensures a consistent processing flow, while pixel value normalization improves numerical stability.

[0040] Step 114: Convert the preprocessed image from the RGB color space to a luminance-chrominance separated color space. The color space can be HSV, YCbCr, or Lab color space. This conversion conforms to human visual characteristics and facilitates independent analysis and enhancement of luminance and color information.

[0041] Step 116: Extract image state features from the transformed image that can summarize its global visual attributes. Image state features (i.e., visual feature vectors) The goal is to characterize the statistical properties of an image in a compact vector form. Image state features include, but are not limited to: histogram distribution of the luminance channel, reflecting the distribution of light and dark areas in the image; statistical moment features of the chrominance channel (statistical mean, variance, etc.), characterizing the overall color tendency and fluctuation of the image; and global contrast measurement, which assesses the degree of difference between the light and dark parts of the image.

[0042] Specifically, for the input image I to be enhanced, the histogram distribution H(L) of its brightness channel L can be calculated using the following formula: ; in, W and H For the width and height of the image, B The number of histogram intervals is the preset value. This is the Dirac function; the function value is 1 when the expression inside the parentheses is 0, and 0 otherwise. k For range index .

[0043] The statistical moment characteristics of the chroma channel include the mean. and standard deviation These describe the average level and dispersion of the color, respectively: ; ; in, c Represents either the a or b chroma channel. Global contrast measure. C The root mean square contrast ratio (RMSContrast) of the luminance channel can be used; this is a widely used no-reference quality metric. ; Furthermore, the text control information is the target text semantic instruction. This target text semantic instruction is converted into a high-dimensional text feature vector by a pre-trained text encoder. This text encoder, trained on large-scale text-image pairs, is able to map abstract semantic descriptions to a dense vector space.

[0044] Finally, the extracted visual feature vectors With text feature vectors The fusion process is typically achieved through a learnable fully connected neural network layer, mapping features from two different modalities to a common subspace to form a final joint state representation s. This joint state representation s serves as the input to the parameter generation model. ; in, and The learnable weight matrix and bias vector of the fusion layer. This represents the operation of concatenating two vectors along their feature dimensions. This is a linear rectification activation function used to introduce nonlinearity.

[0045] In one embodiment, during the training phase of the parameter generation model, optimization is performed based on the reward signal calculated by the cross-modal semantic awareness model, including steps 1 to 4: Step 1, Feature Extraction: Extract the visual feature vector of the enhanced output image using the image encoder of the cross-modal semantic perception model. Text feature vectors of semantic instructions of the target text are extracted through a text encoder. .

[0046] Specifically, the image encoder of the cross-modal semantic perception model with frozen parameters extracts the enhanced output image. Deep visual feature vectors Simultaneously, the target text semantic instructions are extracted through a text encoder using a cross-modal semantic perception model. semantic feature vector These two encoders have been pre-trained on massive amounts of internet image and text data, enabling them to map images and text to a shared semantic feature space.

[0047] Step 2, Similarity Calculation and Reward Generation: Calculate the visual feature vector. With text feature vectors The cosine similarity between them serves as the base value for the reward signal. .

[0048] Calculate visual feature vectors in a shared feature space. With text feature vectors The cosine similarity between the vectors. Cosine similarity measures the degree of proximity of two vectors in a direction, with a value range of [-1, 1]. A larger value indicates a stronger semantic correlation. This cosine similarity is used as the base value for the reward signal. It quantitatively reflects the extent to which the enhanced inspection image conforms to the semantic requirements of the text instruction: ; in, D The dimension of the feature vector. This represents the vector dot product operation. This represents the L2 norm of a vector.

[0049] Step 3, Reward Standardization: Standardize the base value Standardization is performed to obtain the reward signal used for training. .

[0050] To stabilize the reinforcement learning training process and prevent drastic changes in the reward scale, the original reward needs to be standardized. In this embodiment, runtime motion statistics are used to calculate the reward signal for the current training. : ; in, and Based on historical rewards within a recent window The calculated moving average and moving standard deviation ensure that the reward signal roughly follows a standard normal distribution with zero mean and unit variance, which is beneficial for gradient optimization.

[0051] Step 4, Strategy Update: Use Reward Signals r The parameters of the policy network are updated using a soft policy iterative algorithm. Parameters of the AND-valued function network .

[0052] Specifically, the standardized reward signal r Current status The action performed (low-dimensional image augmentation parameter vector) And the next state As a transfer sample, it is stored in the experience replay buffer. In the middle. Then, from the experience replay buffer. A small batch of samples is randomly sampled, and the policy gradient update formula of the SAC algorithm is used to update the parameters of the policy network simultaneously. and two soft Q Parameters of the value network and the automatically adjusted entropy temperature coefficient Through numerous such iterations, the parameter generation model gradually learns to output image enhancement parameters that yield higher semantic rewards, i.e., parameters that better match the target text description.

[0053] Furthermore, during training, the parameters of the policy network... Minimize the specific policy loss function using gradient descent. To update: ; in, D It is an experience replay buffer that stores historical interaction data. and These are two independently trained, identically structured soft state-action value function networks used to estimate the state... Next action The expected cumulative return that can be obtained after using two Q Finding the minimum value in a network is a common technique to reduce overestimation of value.

[0054] The low-dimensional image augmentation parameter vector is the final output of the policy network. These are specific, interpretable numerical values, not the weights of the neural network. Depending on the type of boosting operator chosen, these parameters can include one or more combinations of the following: 1. Bézier curve control point parameters: For example, a curve containing n+1 ordinate values. The vector is used to define an nth-order Bézier curve that maps input brightness / color values ​​to output values, and the x-coordinates of the control points are usually evenly distributed.

[0055] 2. Lookup table node values: For a one-dimensional LUT, the parameter is a list of output values ​​of length L; for a three-dimensional LUT, the parameter is a list of output values ​​of length L. The vectors sequentially define the output color of each grid node in the RGB color cube.

[0056] 3. Global adjustment parameters: For example, contrast scaling factor Saturation bias These are used for global linear or nonlinear adjustments.

[0057] In one embodiment, a parameterized mapping function is used to perform pixel-level mapping processing on the image to be enhanced, implemented through a dedicated hardware acceleration unit. The hardware acceleration unit includes: a configuration interface for receiving lookup table data from the software layer; a high-speed on-chip memory for storing the lookup table data; a parallel multi-channel pixel processing engine for performing synchronous lookup and interpolation operations on the input pixel stream; and an output interface for outputting the enhanced pixel stream in video timing sequence.

[0058] To meet the stringent requirements of online enhancement of high-definition images or real-time video streams during power equipment inspection, the processing using parameterized mapping functions is designed to be efficiently implemented through a dedicated hardware acceleration unit. This hardware acceleration unit acts as a coprocessor, specifically responsible for performing time-consuming pixel-level mapping calculations. Its workflow and advantages are as follows: After initialization or parameter updates, the main control software pre-calculates the constructed enhancement mapping function into a fixed-precision 3D lookup table (3D LUT) and loads this lookup table data into the high-speed on-chip memory of the hardware acceleration unit via a high-speed configuration interface. When processing video streams, the input pixel stream is fed into this hardware acceleration unit. The hardware acceleration unit employs a highly parallel and pipelined design: the parallel multi-channel pixel processing engine contains multiple parallel pixel decoding and address generation channels, capable of processing multiple pixels simultaneously. Specifically, the address generation module quickly calculates the corresponding cube cell and eight vertex addresses in the 3D LUT based on the pixel's RGB values, and calculates the lookup table address for each pixel in parallel. Then, the storage controller reads the output values ​​of these eight vertices in parallel, completing the parallel calculation of the lookup table address for each pixel. Finally, a dedicated interpolation logic unit reads adjacent node values ​​from high-speed on-chip memory based on the lookup table address, performs linear or trilinear interpolation calculations, and outputs the enhanced pixel stream via the output interface, supporting real-time processing throughput for high-definition or ultra-high-definition video frames. Let the input pixel stream sequence be... The output pixel stream sequence is The entire hardware processing can be formally described as follows: ; in, The function calculates the base address and offset of a pixel in the LUT based on the pixel value. The function performs interpolation calculations. Assume the hardware acceleration unit has... A parallel pixel processing pipeline with a working clock frequency of [frequency missing]. Then the peak pixel processing throughput of this unit for: ; In one embodiment, the method further includes: during the deployment phase, providing feedback evaluation on the enhanced inspection image, and, without updating the parameter generation model parameters, adaptively adjusting the image enhancement parameters online based on the feedback evaluation results so that the enhancement results meet preset multi-objective constraints.

[0059] In the training phase, image enhancement parameters are learned based on the perception evaluation results. In the deployment phase, without relying on deep neural network inference, parameterized pixel mapping processing is performed only based on the image enhancement parameters and their online adjustment results to achieve real-time image enhancement suitable for power equipment inspection.

[0060] like Figure 2 As shown, the text-controlled semantic-aware image enhancement method for power equipment inspection provided in this application generates low-dimensional, interpretable image enhancement parameters to control the image enhancement process by jointly modeling the image state information of the power equipment inspection image with text control information, rather than directly generating the enhanced image. During the training phase, an optimization mechanism based on perception evaluation results is introduced to train the parameter generation model. After the model training is completed, the deployment phase begins. In the deployment phase, the parameter generation model parameters are fixed, and the enhancement process no longer calls the perception evaluation model or deep neural network. Instead, pixel-level enhancement processing is performed on the inspection image only through a parameterized mapping function to reduce computational complexity and improve real-time performance. Considering the image characteristics of power equipment inspection scenarios, brightness mapping monotonicity constraints and edge gradient preservation constraints are applied to the parameterized mapping function to avoid damage to the equipment surface structure and defect boundary features during the image enhancement process.

[0061] This application creatively combines cross-modal semantic understanding, maximum entropy reinforcement learning, and interpretable parametric image enhancement techniques, and specifically applies them to power line inspection scenarios. Compared with existing technologies, this application has the following advantages: First, high-order semantic controllable enhancement was achieved in power grid inspection scenarios. By using models such as CLIP (Contrastive Language-Image Pre-Training, a multimodal pre-trained neural network), the high-level operational intentions described by inspectors in natural language, such as "enhancing heated areas" and "highlighting insulator contamination," were quantified into reward signals. These signals guided the reinforcement learning agent to explore and make decisions, achieving semantic-driven enhancement for specific defect features under conditions without reference images. This allows inspectors to directly control the enhancement direction through intuitive text commands, greatly improving the targeting and efficiency of inspection work.

[0062] Secondly, the transparency, interpretability, and reliability of the enhancement process are ensured. The agent learns and outputs interpretable LUT node parameters or Bézier curve control points used in power industry image processing, rather than the black-box weights of deep neural networks. This not only makes the enhancement effect completely transparent to professionals, facilitating manual verification and fine-tuning of automated results, but also, by applying brightness mapping monotonicity constraints and edge gradient preservation constraints, strictly guarantees the spatial smoothness of the output image and the consistency with the equipment structure. This completely avoids artifacts or structural distortions that may be caused by pixel-based generation methods, ensuring that the enhanced image can be used for accurate defect analysis and condition assessment.

[0063] Then, the barriers between advanced algorithms and industrial deployment were broken down to meet the real-time requirements of power line inspection. This application adopts an excellent paradigm of "complex training, simple inference." The training phase utilizes complex deep reinforcement learning for semantic decision-making; while in the inference and deployment phase, the complex decision results are simplified into highly efficient table lookup or curve calculation operations. In particular, through co-design with a dedicated hardware lookup table acceleration unit, the algorithm can be efficiently deployed on resource-constrained platforms such as drones, inspection robots, or edge servers, achieving real-time enhancement processing of high-definition or even 4K inspection video streams, perfectly balancing the algorithm's advanced nature with engineering implementation needs.

[0064] Finally, this solution addresses specific technical challenges in power line inspection. Compared to general image enhancement methods, this approach can adaptively handle common problems in inspection images, such as low-contrast defects, strong reflections, and shadow occlusion. Guided by semantic rewards, the model can learn to generate optimal parameters that specifically enhance the target region while suppressing interference, thereby significantly improving the accuracy and robustness of the subsequent Automatic Defect Recognition (ADI) algorithm.

[0065] In summary, this application combines the advantages of cross-modal semantic understanding, deep reinforcement learning, and traditional interpretable image processing tools, achieving significant improvements in multiple dimensions such as inspection semantic targeting, process controllability, result reliability, and real-time deployment. It provides an innovative and practical dedicated image enhancement solution for intelligent inspection of power equipment.

[0066] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0067] Based on the same inventive concept, this application also provides a text-controlled semantic-aware image enhancement system for power equipment inspection, which implements the aforementioned text-controlled semantic-aware image enhancement method for power equipment inspection. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the text-controlled semantic-aware image enhancement system for power equipment inspection provided below can be found in the limitations of the text-controlled semantic-aware image enhancement method for power equipment inspection described above, and will not be repeated here.

[0068] In one embodiment, such as Figure 3 As shown, a text-controlled semantic-aware image enhancement system for power equipment inspection is provided, comprising: The input module 110 is used to acquire the image to be enhanced from the power equipment inspection terminal, extract features from the image to be enhanced, and obtain image status information to characterize the brightness distribution, contrast statistics and color distribution.

[0069] The state representation and fusion module 120 is used to acquire text control information corresponding to the inspection task, fuse the text control information with the image state information, and generate a joint state representation. The semantically guided reinforcement learning decision module 130 is used to input the joint state representation into the trained parameter generation model, and the parameter generation model outputs a low-dimensional image enhancement parameter vector to control the inspection image enhancement process; the low-dimensional image enhancement parameter vector is used to represent the pixel mapping relationship.

[0070] The parameterized mapping function construction module 140 is used to construct parameterized mapping functions based on low-dimensional image enhancement parameter vectors. The parameterized mapping functions include lookup table mapping functions and / or curve-based mapping functions, and the parameterized mapping functions satisfy brightness mapping monotonicity constraints and edge gradient preservation constraints.

[0071] The image enhancement execution module 150 is used to perform pixel-level mapping processing on the inspection image to be enhanced using a parameterized mapping function, and output the enhanced inspection image as the final enhancement result; the enhanced inspection image is used for defect detection or condition assessment of power equipment.

[0072] In one embodiment, the system further includes a parameter adjustment module, which is used to provide feedback evaluation on the enhanced inspection image during the deployment phase, and to adaptively adjust the image enhancement parameters online based on the feedback evaluation results without updating the parameters of the parameter generation model, so that the enhancement results meet the preset multi-objective constraints.

[0073] In one embodiment, the system further includes a training feedback module for optimizing the reward signal calculated by the cross-modal semantic perception model during the training phase of the parameter generation model.

[0074] The modules in the aforementioned text-controlled semantic-aware image enhancement system for power equipment inspection can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0075] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a text-controlled semantic-aware image enhancement method for power equipment inspection. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0076] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] In one embodiment, a computer device is also provided. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0078] In one embodiment, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0079] In one embodiment, a computer program product is also provided. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0080] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0081] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A text-controlled semantic-aware image enhancement method for power equipment inspection, characterized in that, include: The system acquires the inspection image to be enhanced collected by the power equipment inspection terminal, extracts features from the inspection image to obtain image state information that characterizes brightness distribution, contrast statistics and color distribution; Obtain the text control information corresponding to the inspection task, and fuse the text control information with the image state information to generate a joint state representation; The joint state representation is input into the trained parameter generation model, which outputs a low-dimensional image enhancement parameter vector to control the inspection image enhancement process; the low-dimensional image enhancement parameter vector is used to characterize the pixel mapping relationship. A parameterized mapping function is constructed based on the low-dimensional image enhancement parameter vector; the parameterized mapping function includes a lookup table mapping function and / or a curve-based mapping function, and the parameterized mapping function satisfies the brightness mapping monotonicity constraint and the edge gradient preservation constraint; The parameterized mapping function is used to perform pixel-level mapping processing on the inspection image to be enhanced, and the enhanced inspection image is output as the final enhancement result. The enhanced inspection images are used for defect detection or condition assessment of power equipment.

2. The method according to claim 1, characterized in that, Feature extraction is performed on the image to be enhanced to obtain image state information characterizing brightness distribution, contrast statistics, and color distribution, including: The image to be enhanced is preprocessed by normalizing its size and pixel values ​​to the [0,1] range; The preprocessed image is converted from the RGB color space to a luminance-chrominance separated color space. Image state features are extracted from the converted image, including at least one of the following: histogram distribution of the luminance channel, statistical moment features of the chrominance channel, and global contrast measure.

3. The method according to claim 2, characterized in that, The parameter generation model is constructed based on the maximum entropy reinforcement learning framework, including a policy network and a value function network; the policy network receives the joint state representation and outputs the conditional probability distribution of the low-dimensional image enhancement parameter vector; the optimization objective function of the parameter generation model is to maximize the weighted sum of the expected cumulative reward and the policy entropy.

4. The method according to claim 3, characterized in that, The low-dimensional image enhancement parameter vector includes: a sequence of control point coordinates for a Bézier curve used to define the brightness mapping; a vector of node values ​​used to fill a one-dimensional brightness lookup table or a three-dimensional color lookup table; and at least one of a contrast scaling factor and a saturation bias used to adjust global attributes.

5. The method according to claim 4, characterized in that, When constructing the nth-order Bézier curve mapping function, the low-dimensional image enhancement parameter vector contains the coordinates of n+1 control points. ,in , For the normalized input intensity, The corresponding output intensity; the normalized intensity value for any input pixel. x The output intensity value after mapping y Calculated using the following formula: ； ； in, It is a Bernstein basis polynomial. The coefficients are binomial coefficients.

6. The method according to claim 4, characterized in that, When constructing the 3D lookup table mapping function, the low-dimensional image enhancement parameter vector defines the output color value of each vertex of a regular cubic grid in the RGB or YCbCr color space; the regular cubic grid is uniformly divided into L nodes in each color dimension, forming... Unit cube cells; for the input color vector The mapping process includes: Determine the unit cube cell containing the input vector, and read the predefined output color values ​​at the 8 vertices of that unit cube cell. ,in ; Calculate the input vector relative to the bottom left vertex of the unit cube cell. normalized offset The final output color value is calculated using trilinear interpolation. : ； ； in, These are the interpolation weights.

7. The method according to claim 1, characterized in that, The parameterized mapping function is used to perform pixel-level mapping processing on the inspection image to be enhanced, which is implemented by a dedicated hardware acceleration unit; the hardware acceleration unit includes: A configuration interface is provided to receive lookup table data from the software layer. High-speed on-chip memory is used to store the lookup table data; A parallel multi-path pixel processing engine is used to perform synchronous table lookup and interpolation operations on the input pixel stream; The output interface is used to output the enhanced pixel stream in video timing.

8. The method according to any one of claims 1 to 7, characterized in that, Also includes: During the deployment phase, the enhanced inspection images are evaluated, and the image enhancement parameters are adaptively adjusted online based on the evaluation results without updating the parameters of the parameter generation model, so that the enhancement results meet the preset multi-objective constraints.

9. The method according to claim 8, characterized in that, During the training phase of the parameter generation model, optimization is performed based on the reward signal calculated by the cross-modal semantic awareness model, including: The image encoder of the cross-modal semantic perception model extracts the visual feature vector of the enhanced output image. Text feature vectors of semantic instructions of the target text are extracted through a text encoder. ; Calculate visual feature vectors With text feature vectors The cosine similarity between them serves as the base value for the reward signal. : ； For base value Standardization is performed to obtain the reward signal used for training. : ,in and The moving average and standard deviation are calculated based on historical rewards. Using the reward signal r The parameters of the policy network are updated using a soft policy iterative algorithm. φ With the parameters of the value function network .

10. A text-controlled semantic-aware image enhancement system for power equipment inspection, characterized in that, include: The input module is used to acquire the inspection image to be enhanced collected by the power equipment inspection terminal, and to extract features from the inspection image to obtain image status information that characterizes the brightness distribution, contrast statistics and color distribution. The state representation and fusion module is used to acquire text control information corresponding to the inspection task, fuse the text control information with the image state information, and generate a joint state representation. A semantically guided reinforcement learning decision module is used to input the joint state representation into a trained parameter generation model, and the parameter generation model outputs a low-dimensional image enhancement parameter vector for controlling the inspection image enhancement process; the low-dimensional image enhancement parameter vector is used to represent pixel mapping relationships. A parameterized mapping function construction module is used to construct a parameterized mapping function based on the low-dimensional image enhancement parameter vector; the parameterized mapping function includes a lookup table mapping function and / or a curve-based mapping function, and the parameterized mapping function satisfies the brightness mapping monotonicity constraint and the edge gradient preservation constraint; The image enhancement execution module is used to perform pixel-level mapping processing on the inspection image to be enhanced using the parameterized mapping function, and output the enhanced inspection image as the final enhancement result. The enhanced inspection images are used for defect detection or condition assessment of power equipment.

Images