Construction site image enhancement method based on hierarchical semantic gating and region degradation evaluation

Abstract

The application discloses a construction site image enhancement method based on hierarchical semantic gating and region degradation evaluation, comprising the following steps: according to the space action range involved in the information in a preset illegal behavior description text, dividing the semantic units in the text into overall semantic description and local semantic description; acquiring a construction site image, extracting global image features and image region features, respectively combining the overall semantic representation and the local semantic representation, generating a semantic space gating matrix, and obtaining a region degradation representation matrix; based on the region degradation representation matrix, calculating the strategy score of each image region for three image enhancement processing strategies, and generating a region enhancement weight matrix according to the semantic space gating matrix; mapping the region enhancement weight matrix to a pixel space consistent with the construction site image and performing edge preserving smoothing, then combining the restricted residual of the three enhancement branches and the corresponding spatial enhancement weight to complete pixel-by-pixel weighted fusion, and obtaining an enhanced construction site image.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to a method for enhancing construction site images based on hierarchical semantic gating and regional degradation assessment. Background Technology

[0002] Construction site images often contain information about a large work area and overall behavioral relationships, as well as key local visual targets such as safety helmets, safety belts, and reflective clothing. Due to differences in spatial scale, texture density, brightness / color distribution, and imaging visibility among different semantic contents, construction site images often exhibit significant spatial non-uniformity and multiple types of degradation superimposed.

[0003] Currently, most enhancement or improvement methods related to construction site images adopt a uniform enhancement strategy based on global image statistics (such as global brightness histogram, global contrast, and global sharpening parameters), such as uniform gamma correction, uniform contrast enhancement, or uniform sharpening. Although these methods are simple to implement and computationally stable, the enhancement parameters remain consistent across the entire image, making it difficult to address the differential degradation in different semantic regions.

[0004] Some methods first perform semantic segmentation or semantic classification, and then assign different enhancement intensities or types to different regions based on their semantics. For example, different contrast or brightness strategies are used for "personnel regions" and "background regions," or the enhancement intensity is increased for key target regions. Such methods can "know where to enhance" to some extent, but they are usually still relatively coarse-grained region-level or category-level scheduling.

[0005] Another approach involves generating multiple enhanced versions (e.g., contrast-enhanced, sharpened, and brightness-corrected versions) and then weighting the enhancement results using fusion weights (which can come from attention mechanisms or confidence estimation). Fusion can improve adaptability, but in construction site images, there may be abrupt texture changes at regional boundaries. If the spatial distribution of fusion weights is unconstrained, it can easily lead to jittering at the boundaries, cross-regional leakage, or the generation of artifacts.

[0006] Because construction site images simultaneously contain semantic regions of different scales, the aforementioned existing routes are prone to revealing the following problems in practical use: Uniform enhancement of the entire image can affect the overall structure of a large working area, while failing to provide sufficiently accurate compensation for the degradation of details in key local targets. The result may be insufficient enhancement of semantically relevant regions, excessive enhancement of non-critical regions, or unclear local details.

[0007] Construction site images may simultaneously exhibit uneven lighting (localized underexposure / overexposure), blurring due to camera shake, lens defocus, and loss of detail caused by image compression. Different regions correspond to different degradation types, therefore the same enhancement strategy cannot be applied to all regions simultaneously; furthermore, directly assigning enhancements based solely on semantic associations is insufficient to reflect the true degradation type and intensity differences in each region.

[0008] If semantic allocation is performed without constraints on how the enhancement should function stably in pixel space, the fusion weights may become discontinuous at texture edges, leading to abrupt color / brightness changes, amplified noise, or damage to local structures at the boundaries. In particular, when fusion weights propagate across boundaries, they weaken the accuracy and naturalness of the enhancement.

[0009] To address the problems of the existing technologies mentioned above, a construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment is designed. Summary of the Invention

[0010] This invention provides a construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment, comprising: S1 divides the semantic units in the text into overall semantic description and local semantic description based on the spatial scope of the information in the pre-defined text describing the violation, and encodes them into overall semantic representation and local semantic representation respectively. S2 acquires images of the construction site, extracts global image features and image region features, combines the overall semantic representation and local semantic representation to generate a semantic space gating matrix, and evaluates the degradation degree of each image region to obtain a region degradation representation matrix. S3 calculates the strategy scores for each image region for brightness correction enhancement, local contrast enhancement, and edge texture detail enhancement based on the region degradation representation matrix, and generates a region enhancement weight matrix based on the semantic space gating matrix. S4 maps the region enhancement weight matrix to the pixel space consistent with the construction site image and performs edge smoothing. Then, it combines the pixel residuals of the brightness correction enhancement branch, local contrast enhancement branch and edge texture detail enhancement branch with the corresponding spatial enhancement weights to complete pixel-by-pixel weighted fusion. After fusion, the pixel values ​​are limited to a preset effective range to obtain the enhanced construction site image.

[0011] The beneficial effects of this invention are: First, by acquiring the region enhancement weight matrix and the pixel-wise semantic space gating matrix, and combining them with the pixel-wise brightness correction direction information, a spatial mapping relationship is established between the enhancement control signal and the construction site image, thereby achieving "region-adaptive enhancement selection based on semantic association". This design ensures that subsequent enhancement is not indiscriminately processed across the entire image, but rather that more precise enhancement control is performed in the corresponding regions for different degradation types, improving the relevance and stability of the enhancement.

[0012] Secondly, guided filtering of the initial spatial enhancement weights and brightness correction direction is performed using the brightness map of the construction site image as a guide, ensuring smooth edges and avoiding checkerboard artifacts or cross-boundary leakage caused by weight jitter at texture / object boundaries. At the same time, the enhancement weights are constrained and normalized based on pixel-by-pixel semantic gating to suppress over-enhancement in semantically irrelevant areas, ensuring that the enhancement effect is mainly concentrated in positions highly correlated with the degradation type, improving visual naturalness and reducing the risk of artifacts.

[0013] Third, three types of candidate enhancement results are generated in parallel for the same construction site image: brightness correction enhancement result, local contrast enhancement result, and edge and texture detail enhancement result, thereby covering different degradation dimensions commonly found in construction sites. This parallel candidate generation method enables the system to have "multi-strategy repair capability," allowing it to adaptively select a more suitable enhancement type based on spatial weights in the subsequent fusion stage, improving robustness and adaptability to scenes with complex lighting and multiple degradations coexisting.

[0014] Fourth, the three enhancement results are first compared with the original image to obtain corresponding residual maps. Then, the three types of residuals are weighted and fused pixel by pixel based on the normalized spatial enhancement weights. At the same time, the output is limited to the effective range by Clip to avoid visual artifacts caused by overflow and extreme values. Since the enhancement weights are smoothed at the edges after being mapped to the pixel space and are subject to semantic gating constraints, this fusion can balance local enhancement and overall naturalness: the enhancement is more sufficient in areas with high semantic relevance and obvious degradation, while the enhancement weights are reduced accordingly in areas with low degradation or weak semantic relevance, thereby preserving the original image content as much as possible and finally obtaining an enhancement result that is more in line with the real appearance of the construction site. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is the overall model framework diagram of this embodiment.

[0017] Figure 2 This is a framework diagram of the dual-branch gating semantic calibration module in this embodiment. Detailed Implementation

[0018] To facilitate understanding by those skilled in the art, the structure of the present invention will now be described in further detail with reference to the accompanying drawings. It should be understood that, unless otherwise specified, the order of the steps mentioned in this embodiment can be adjusted according to actual needs, and they can even be executed simultaneously or partially simultaneously.

[0019] like Figure 1 As shown, this embodiment of the invention provides a construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment, including: S1 divides the semantic units in the text into overall semantic description and local semantic description based on the spatial scope of the information in the pre-defined text describing the violation, and encodes them into overall semantic representation and local semantic representation respectively. S101 performs cleaning, trimming, word segmentation, semantic parsing, expansion, and standardization on the preset description text of violations to obtain a set of semantic units; In this step, the on-site monitoring system pre-collects relevant violation regulations documents and expert-annotated data from the construction site. For the original description text of the violation (e.g., "Workers were not wearing safety helmets correctly in the construction area, and their reflective vests were not worn correctly"), the processing flow in this embodiment is as follows: Cleaning and trimming: Remove special symbols, meaningless particles, and redundant stop words from the text, and trim and refine it into the core semantic expression; Word segmentation and semantic analysis: Using Chinese word segmentation tools and syntactic analyzers to parse the text structure and extract the subject of the violation (e.g., "workers"), the violation behavior / state (e.g., "not wearing", "not dressed"), and the object of the violation (e.g., "safety helmet", "reflective clothing"). Text expansion and standardization: To address common synonym substitutions and habitual expressions in the field, a thesaurus and synonymous context expansion are introduced. The parsed key semantic expansions are summarized into several standardized, independent and consistent semantic units, forming a semantic unit set.

[0020] S102 For each semantic unit, the semantic unit containing semantic elements of the construction site scene, work area, or overall behavioral relationship is determined as the overall semantic description; In this step, the preset judgment conditions for the overall semantics include at least one of the following: Semantic units contain semantic elements of the construction site, work conditions, and overall work framework; A semantic unit contains spatial elements that can characterize the scope of a space, and the spatial elements form a semantic relationship with the conditions for the occurrence of the violation or the behavioral elements. The semantic unit provides an overall description of the relationship between "personnel - behavior - occurrence in scope / work area" for the occurrence of violations, and does not focus on the fine-grained parts of protective equipment.

[0021] S103 defines each semantic unit as a local semantic description by defining the semantic unit containing semantic elements of personal protective equipment and its safety status or local behavioral status. In this step, the preset judgment conditions for local semantics include at least one of the following: Semantic units contain semantic elements of personal protective equipment; The semantic unit contains a safety status element corresponding to the protective equipment, and the safety status element includes at least one of missing, not worn, not used correctly, or worn incorrectly; Semantic units contain semantic elements of local parts / local locations that can be correlated with fine-grained visual inspections and can reflect changes in local protection status.

[0022] For example, "working at heights" and "working near edges" can be used as overall semantic descriptions; "helmet wearing status," "safety belt fastening status," and "reflective clothing wearing status" can be used as local semantic descriptions. The semantic hierarchy is used to determine the spatial scope of action, but does not directly specify the specific image enhancement processing type.

[0023] S104 inputs the overall semantic description and the local semantic description into the text encoding network for semantic encoding, respectively, to obtain the overall semantic representation matrix and the local semantic representation matrix: , in, The matrix represents the overall semantic representation. Indicates the first The vector corresponding to the overall semantic description of the item. , Indicates the total number of semantic descriptions. The dimension of the semantic description vector; , in, The local semantic representation matrix is ​​represented. Indicates the first The vector corresponding to each local semantic description. Indicates the number of local semantic descriptions.

[0024] In this step, the text encoding network and the image encoding network adopt the corresponding encoding network in the visual-language encoding model that has completed the semantic alignment of text and image in advance, so that the semantic representation of text and image features have a pre-established cross-modal semantic correspondence. The linear mapping matrix used to unify the semantic association space is pre-determined, and the parameters of the encoding network and the linear mapping matrix are fixed when performing construction site image enhancement.

[0025] S2 acquires images of the construction site, extracts global image features and image region features, combines the overall semantic representation and local semantic representation to generate a semantic space gating matrix, and evaluates the degradation degree of each image region to obtain a region degradation representation matrix. S201 acquires construction site images captured by on-site cameras, performs size adjustment and pixel value normalization processing on them, resulting in the normalized image: , in, This represents a normalized image of the construction site. and These represent the image height and image width, respectively. The number 3 indicates the number of color channels, and the pixel values ​​are normalized to a range of zero to one. S202 inputs construction site images into an image coding network with multiple coding stages for feature extraction, obtaining global image features that retain spatial correspondences: , in, Represents global image features. Indicates the number of global image feature units. Indicates the feature dimension; In this step, each feature unit corresponds to a spatial location in the construction site image. The image encoding network is the image encoder corresponding to the text encoding network in the aforementioned vision-language encoding model, and can be a convolutional encoder or a hierarchical visual Transformer encoder. This obtains a global representation with spatial correspondence, enabling subsequent cross-modal attention to locate the image position, rather than just performing global vector matching.

[0026] S203 extracts shallow image features from the three shallow coding stages of the image coding network, and completes spatial size and feature dimension alignment through bilinear interpolation and linear mapping. The aligned shallow image features are then aggregated and encoded to obtain image region features. , , in, Indicates the first Shallow image features output from each shallow coding stage This indicates the corresponding shallow coding stage. This represents shallow image features aligned by spatial dimensions and feature dimensions. This indicates the aggregation of shallow image features. Indicates the subsequent encoding stage. Represents image region features, This indicates the number of image regions. Each row in the image region feature corresponds to one image region in the construction site image.

[0027] In this step, multi-scale shallow information is fused into a local representation that is more suitable for region enhancement / region quality assessment, and a correspondence between region indexes and original image pixels is established to ensure that subsequent degradation indices can be calculated and semantic gating has a clear area of ​​application.

[0028] S204 maps global image features and overall semantic representation to a unified semantic association space and calculates the overall semantic space response; S2041 linearly maps the global image features and the overall semantic representation matrix to a unified semantic association space of the same dimension, respectively, to obtain the global query representation and the overall semantic key representation: , , , in, This indicates a global query. This represents the overall semantic key representation. and Let represent the linear mapping matrices between global image features and overall semantic representation, respectively. The dimension of the unified semantic association space is represented by a linear mapping matrix that maintains the pre-established semantic correspondence between the image and text and is fixed when performing image enhancement.

[0029] S2042 calculates the overall semantic association matrix based on the global query representation and the overall semantic key representation: , in, Represents the overall semantic association matrix, matrix elements Indicates the first The global image feature unit and the first The degree of correlation between the overall semantic descriptions is determined by the transpose operation, which performs a term-by-term dot product between the corresponding image representation and the text representation. The square root is used to scale the dot product result. Since the two representations reside in a unified semantic association space, the dot product result is used to characterize the degree of cross-modal semantic association.

[0030] S2043 performs Softmax normalization on the overall semantic association matrix along the image spatial location dimension to obtain the overall semantic weight matrix: , S2044 is obtained by taking the maximum value of the overall semantic weight matrix in the overall semantic description dimension to obtain the overall semantic response: , S2045 normalizes the overall semantic response matrix row by row to obtain normalized responses, and suppresses normalized responses that do not meet the preset overall semantic association threshold to obtain the filtered overall semantic space response: , , in, This represents the overall semantic weight matrix after normalization along the spatial location dimension of the image. Indicates the first The maximum overall semantic response at each position This represents the normalized response. Indicates the position index involved in the extreme value calculation. This prevents positive numbers with a denominator of zero. This indicates the overall semantic threshold. Indicates an indicator function, This indicates the preset overall semantic association threshold. This represents the overall semantic space response after filtering.

[0031] In this step, spatial locations in the image that are more likely to involve violations / require processing are located from the overall semantics (e.g., macroscopic descriptions related to spatial scope) and a response that can be used for regional-scale fusion is output.

[0032] S205 maps image region features and local semantic representations to a unified semantic association space and calculates the local semantic space response; S2051 linearly maps the image region features and the local semantic representation matrix to a unified semantic association space of the same dimension, respectively, to obtain the region query representation and the local semantic key representation: , , in, This indicates a region query. This represents a local semantic key representation. and Let represent the linear mapping matrices between image region features and local semantic representations, respectively. The dimension representing the unified semantic association space; S2052 calculates the local semantic association matrix using the region query representation and the local semantic key representation: , S2053 performs maximum activation on the local semantic association matrix along the local semantic description dimension to obtain the local semantic response of each image region: , S2054 When the maximum activation response in the local semantic association matrix is ​​less than a preset local semantic association threshold, suppression is applied to the local semantic response to obtain the filtered local semantic space response: , in, Represents the local semantic association matrix. This represents a local semantic threshold indicator. This indicates a preset local semantic association threshold. This represents the filtered local semantic response. Since the region query representation and the local semantic key representation reside in the same unified semantic association space, their dot product is used to characterize the degree of association between the corresponding image region and the local semantic description.

[0033] In this step, semantic matching strengths related to local security status / detail features are extracted at the region granularity to improve the accuracy of region selection.

[0034] S206 adjusts the overall semantic space response to the same spatial scale as the local semantic space response, and then combines it with the local semantic space response according to a preset ratio and restricts it within a preset range to obtain the semantic space gating matrix: , , , in, Represents the overall semantic space response aligned with the scale of the image region. This indicates a spatial scale alignment operation. Indicates the first The semantic gating value of an image region Represents the semantic space gating matrix. and These represent the preset ratio coefficients of the overall semantic space response and the local semantic space response, respectively. The semantic space gating matrix only determines the spatial scope of image enhancement. By fusing the overall and local semantics, the confidence / intensity of the enhancement should be prioritized for each image region, thus grounding semantic constraints in a clear spatial selection logic.

[0035] S207 evaluates the brightness deviation, contrast deficiency, and blurriness of each image region in the construction site image to obtain the region degradation characterization matrix.

[0036] S2071 converts the construction site image into a brightness image, resulting in a brightness image set. Based on the spatial correspondence of image region features, the brightness image is divided into multiple image regions. S2072 calculates the average brightness of each image region and determines the brightness deviation based on the difference between the average brightness and the preset target brightness. , , , in, Indicates the first Brightness image of each image region This indicates the average brightness of the area. Indicates the preset target brightness. Indicates the direction of brightness correction. Indicates brightness deviation. This represents a limiting function that restricts values ​​to a preset range. This indicates a positive number to prevent the denominator from being zero. A positive brightness correction direction increases the brightness of the area, while a negative direction decreases the brightness of the area.

[0037] S2073 calculates the standard deviation of brightness for each image region and determines the degree of contrast deficiency based on the difference between the standard deviation of brightness and the preset target contrast. , , in, Indicates the first The standard deviation of brightness in each image region Indicates the preset target contrast. This indicates the degree of contrast deficiency. When the standard deviation of the area brightness reaches or exceeds the preset target contrast, the degree of contrast deficiency is set to zero.

[0038] S2074 calculates the Laplacian response variance for each image region and determines the degree of blur based on the difference between the Laplacian response variance and a preset sharpness threshold. , , in, Indicates the first The Laplacian response variance of each image region This indicates the preset resolution threshold. Indicates the degree of blur. The lower the Laplacian response variance, the more blurred the image region.

[0039] S2075 concatenates the brightness deviation, contrast deficiency, and blurriness of each image region in order of region to form a region degradation characterization matrix: , in, Represents the region degradation characterization matrix. It indicates the number of image regions; the three columns of the matrix correspond to the brightness deviation, the degree of insufficient contrast, and the degree of blurring, respectively.

[0040] In this step, semantic gating restricts the processing area during the weight enhancement stage, thereby achieving quality assessment guided by semantics.

[0041] S3 calculates the strategy scores for each image region for brightness correction enhancement, local contrast enhancement, and edge texture detail enhancement based on the region degradation representation matrix, and generates a region enhancement weight matrix based on the semantic space gating matrix. The procedure for this step is as follows: Figure 2 As shown, the specific steps are as follows: S301 calculates the policy score for each image region against each enhancement type, based on the region degradation representation matrix: , in, Indicates the first The image region is for the first The strategy score for enhancing processing This indicates the corresponding degradation index. This indicates the corresponding preset startup threshold. This indicates the corresponding preset response slope; Select 1, 2, and 3, which correspond to brightness correction, local contrast enhancement, and detail enhancement, respectively. In this step, the policy score is used in subsequent steps to generate the relative weighting of different enhancements in the region and the independent execution strength of the enhancements.

[0042] S302 arranges the policy scores by region index and enhancement processing type index to obtain the enhancement policy matrix: , in, This represents the enhancement strategy matrix, with its three columns corresponding to brightness correction enhancement, local contrast enhancement, and edge texture detail enhancement, respectively. In this step, the enhancement policy matrix serves as the input for subsequent calculations that compare and normalize different enhancement processes.

[0043] S303 performs Softmax normalization on the enhancement policy matrix along the enhancement processing type dimension to obtain the relative weight allocation matrix: , in, Indicates the first The first image region The relative allocation weights of the enhancement processes This represents the relative weight distribution matrix. The index indicating the enhancement processing type used for summation is one for the same image region, where the sum of the three relative weight assignments is one. In this step, the relative weights are subsequently combined element-wise with the semantic space gating matrix and the independent execution strength to obtain the region enhancement weights.

[0044] S304 applies Sigmoid activation to each element in the enhancement policy matrix and performs threshold truncation amplification to obtain an independent execution strength matrix: , in, Indicates the first The first image region This enhances the independent execution strength of the processing. This represents the independent execution strength matrix, and Sigmoid represents the activation function. When the degradation index is not higher than the corresponding activation threshold, the independent execution strength is zero. In this step, the independent execution strength is used to characterize the execution / non-execution and the degree of execution of the r-th enhancement process in the i-th image region. The independent execution strength is used as the input for subsequent element-wise multiplication to suppress enhancement processes that have not reached the activation threshold.

[0045] S305 extends the semantic space gating matrix along the augmentation processing type dimension and multiplies it element-wise with the relative allocation weight matrix and the independent execution strength matrix to obtain the region augmentation weight matrix: , , in, Represents the region enhancement weight matrix. Indicates the first The first image region Region enhancement weights for enhancement processing, Represents a three-dimensional vector of uniformity. This indicates element-wise multiplication.

[0046] In this step, the semantic space gating matrix is ​​used to constrain the enhancement to only apply to image regions that meet the semantic conditions; the relative allocation weight matrix is ​​used to determine the relative allocation among the three enhancement processes within the same region; and the independent execution intensity matrix is ​​used to determine the degree of independent execution of each enhancement process, thus obtaining the region enhancement weight matrix together.

[0047] S4 maps the region enhancement weight matrix to the pixel space consistent with the construction site image and performs edge smoothing. Then, it combines the pixel residuals of the brightness correction enhancement branch, local contrast enhancement branch and edge texture detail enhancement branch with the corresponding spatial enhancement weights to complete pixel-by-pixel weighted fusion. After fusion, the pixel values ​​are limited to a preset effective range to obtain the enhanced construction site image.

[0048] In this step, to address common degradation issues in construction sites (such as localized overexposure / underexposure, low contrast, unclear edges / textures, and loss of detail due to coatings or dust), we first determine which areas should be prioritized for enhancement based on semantic gating. Then, we generate three candidate enhancement paths based on brightness correction direction information, local contrast enhancement, and unsharpened mask detail enhancement. Subsequently, we calculate the residual of each path relative to the original image, and perform constrained residual fusion under the spatial weights of smoothing and normalization constraints to finally obtain the enhanced construction site image. The specific steps are as follows: S401 performs pixel-space mapping on the region enhancement weight matrix, semantic space gating matrix, and brightness deviation correction direction to obtain an initial spatial enhancement weight map, a pixel-by-pixel semantic gating map, and an initial brightness correction direction map consistent with the pixel grid of the construction site image.

[0049]

[0050]

[0051] in, Indicates the first The initial spatial augmentation weight map corresponding to each augmentation process. Represents a pixel-by-pixel semantic gating graph. This represents the initial brightness correction pattern. This indicates a rearrangement based on the spatial arrangement of image regions. Indicates bilinear interpolation; In this step, a pixel-wise semantic gating graph is introduced to constrain the weights in subsequent steps; a pixel-wise luminance correction direction graph is introduced to guide the direction and intensity of luminance correction enhancement.

[0052] S402 uses the brightness image of the construction site as the guiding image, performs guided filtering on the initial spatial enhancement weight map and the initial brightness correction direction map, and obtains the enhancement weight map and pixel-wise brightness correction direction map after edge smoothing. Then, it performs constraint normalization on the enhancement weight after edge smoothing according to the pixel-wise semantic gating map. , , , in, A brightness image representing a construction site. This indicates guided filtering. This represents the spatial augmentation weight map after ensuring the edges remain smooth. This represents the spatial augmentation weight map after normalization constraints. This represents the pixel-by-pixel brightness correction pattern; In this step, directly using the region output for the enhancement weight map can easily result in checkerboard patterns or noise jitter at texture boundaries, leading to local over-enhancement or tone shift. Guided filtering uses a brightness map of the construction site as the guide image to ensure smoother weight changes at object edges while avoiding cross-boundary leakage. Pixel-wise semantic gating constrains and normalizes the enhancement weights, giving greater weight to regions highly correlated with semantic degradation types, while suppressing weights in irrelevant regions.

[0053] For example, in the same construction drawing, there is a sudden change in contrast at the boundary between the cement floor and the steel structure. If edge smoothing is not performed, the weight map will jump at the boundary, resulting in over-sharpening of the steel structure edge. This step uses brightness image-guided filtering from the construction site image to prevent the spatial enhancement weight map from spreading across regions near the ground / steel structure boundary. At the same time, a pixel-by-pixel semantic gating map is used to suppress weights in regions with low semantic gating, thereby achieving "enhancement only in relevant degraded areas".

[0054] S403 performs multiple types of enhancement processing on construction site images, generating brightness correction enhancement results, local contrast enhancement results, and edge texture detail enhancement results, respectively. S4031 performs adaptive brightness correction on the construction site image based on the pixel-by-pixel brightness correction pattern, resulting in a brightness correction enhancement effect: , in, This indicates the brightness correction result. This indicates brightness correction processing; In this step, luminance correction can be performed using adaptive gamma correction controlled by a pixel-by-pixel luminance correction pattern: Images of the construction site and its brightness image Pixel values ​​normalized to For pixel position Correcting the direction value based on pixel-by-pixel brightness Determine the gamma coefficient:

[0055] This coefficient is used to correct the brightness of the corresponding pixel:

[0056] Then adjust each color channel according to the brightness ratio before and after correction to obtain the brightness correction and enhancement result:

[0057] in, Indicates color channels, To prevent positive numbers with a denominator of zero, a direction value greater than zero makes unsaturated pixels tend to brighten, a direction value less than zero makes them tend to darken, and a direction value of zero keeps the original pixel value.

[0058] S4032 performs local contrast enhancement on the construction site image, and the contrast enhancement result is as follows: , in, This indicates the result of contrast enhancement. This indicates local contrast enhancement processing; In this step, an adaptive histogram equalization with contrast limitation (such as the CLAHE approach) can be used. By limiting the shear threshold, local oversaturation and noise amplification can be avoided.

[0059] S4033 performs edge and texture detail enhancement on the construction site image, resulting in the following detail enhancement:

[0060] in, This indicates the results of enhanced detail. This indicates enhanced detail processing; In this step, edge and texture detail enhancement can be performed using an unsharpened mask. By selecting an appropriate blur kernel size, the scale of sharpening can be controlled to avoid treating noise as texture sharpening.

[0061] S404 calculates the pixel residuals of the three enhancement results relative to the construction site image, and obtains the residual map:

[0062] in, , and These represent the pixel changes related to brightness correction, local contrast enhancement, and edge texture detail enhancement relative to the construction site image; the three pixel residual maps are all correlated with the image. They have the same spatial dimensions and number of color channels.

[0063] In this step, the residual represents an incremental repair enhancement relative to the original image, making it easier to control the fusion amplitude and reduce color cast.

[0064] S405 performs pixel-by-pixel weighted fusion of pixel residuals based on the normalized pixel spatial enhancement weight map, and limits the pixel values ​​of the weighted fusion result to a preset effective range to obtain the enhanced construction site image:

[0065] in, The term represents the enhanced construction site image, the summation term represents the three spatially weighted pixel residuals, and Clip represents the pixel values ​​of the fusion result being limited to a preset effective range of zero to one.

[0066] In this step, when an image region has a high degree of association with the preset semantics and exhibits significant degradation, the image enhancement processing corresponding to this degradation type receives a larger spatial enhancement weight; when the image region has a low degree of degradation or a low degree of association with the preset semantics, the corresponding spatial enhancement weight is reduced, so that the region basically retains its original image content.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0072] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

Claims

1. A construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment, characterized in that, include: S1 divides the semantic units in the text into overall semantic description and local semantic description based on the spatial scope of the information in the pre-defined text describing the violation, and encodes them into overall semantic representation and local semantic representation respectively. S2 acquires images of the construction site, extracts global image features and image region features, combines the overall semantic representation and local semantic representation to generate a semantic space gating matrix, and evaluates the degradation degree of each image region to obtain a region degradation representation matrix. S3 calculates the strategy scores for each image region for brightness correction enhancement, local contrast enhancement, and edge texture detail enhancement based on the region degradation representation matrix, and generates a region enhancement weight matrix based on the semantic space gating matrix. S4 maps the region enhancement weight matrix to the pixel space consistent with the construction site image and performs edge smoothing. Then, it combines the pixel residuals of the brightness correction enhancement branch, local contrast enhancement branch and edge texture detail enhancement branch with the corresponding spatial enhancement weights to complete pixel-by-pixel weighted fusion. After fusion, the pixel values ​​are limited to a preset effective range to obtain the enhanced construction site image.

2. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 1, characterized in that, S1 includes: S101 performs cleaning, trimming, word segmentation, semantic parsing, expansion, and standardization on the preset description text of violations to obtain a set of semantic units; S102 For each semantic unit, the semantic unit containing semantic elements of the construction site scene, work area, or overall behavioral relationship is determined as the overall semantic description; S103 defines each semantic unit as a local semantic description by defining the semantic unit containing semantic elements of personal protective equipment and its safety status or local behavioral status. S104 inputs the overall semantic description and the local semantic description into the text encoding network for semantic encoding, respectively, to obtain the overall semantic representation matrix and the local semantic representation matrix.

3. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 1, characterized in that, S2 includes: S201 acquires construction site images captured by on-site cameras, performs size adjustment and pixel value normalization processing on them, and the normalized image is obtained. S202 inputs the construction site image into an image coding network with multiple coding stages for feature extraction, and obtains global image features that retain spatial correspondence; S203 extracts shallow image features from the three shallow coding stages of the image coding network, and completes the alignment of spatial size and feature dimension through bilinear interpolation and linear mapping. The aligned shallow image features are then aggregated and encoded to obtain image region features. S204 maps global image features and overall semantic representation to a unified semantic association space and calculates the overall semantic space response; S205 maps image region features and local semantic representations to a unified semantic association space and calculates the local semantic space response; S206 adjusts the overall semantic space response to the same spatial scale as the local semantic space response, and then combines it with the local semantic space response according to a preset ratio and limits it to a preset range to obtain the semantic space gating matrix; S207 evaluates the brightness deviation, contrast deficiency, and blurriness of each image region in the construction site image to obtain the region degradation characterization matrix.

4. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 3, characterized in that, S204 includes: S2041 linearly maps the global image features and the overall semantic representation matrix to a unified semantic association space of the same dimension, respectively, to obtain the global query representation and the overall semantic key representation; S2042 calculates the overall semantic association matrix based on the global query representation and the overall semantic key representation; S2043 performs Softmax normalization on the overall semantic association matrix along the image spatial location dimension to obtain the overall semantic weight matrix; S2044 is obtained by taking the maximum value of the overall semantic weight matrix in the overall semantic description dimension to obtain the overall semantic response; S2045 normalizes the overall semantic response to obtain a normalized response, and suppresses normalized responses that do not meet the threshold conditions according to the preset overall semantic association threshold, so as to obtain the filtered overall semantic space response.

5. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 3, characterized in that, S205 includes: S2051 linearly maps the image region features and the local semantic representation matrix to a unified semantic association space of the same dimension, respectively, to obtain the region query representation and the local semantic key representation; S2052 calculates the local semantic association matrix using the region query representation and the local semantic key representation; S2053 performs maximum activation operation on the local semantic association matrix in the local semantic description dimension to obtain the local semantic response of each image region; When the maximum activation response in the local semantic association matrix is ​​less than the preset local semantic association threshold, S2054 suppresses the local semantic response to obtain the filtered local semantic space response.

6. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 3, characterized in that, S207 includes: S2071 converts the construction site image into a brightness image, resulting in a brightness image set. Based on the spatial correspondence of image region features, the brightness image is divided into multiple image regions. S2072 calculates the average brightness of each image region and determines the brightness deviation based on the difference between the average brightness and the preset target brightness. S2073 calculates the standard deviation of brightness for each image region and determines the degree of contrast deficiency based on the difference between the standard deviation of brightness and the preset target contrast. S2074 calculates the Laplacian response variance for each image region and determines the degree of blur based on the difference between the Laplacian response variance and the preset sharpness threshold. S2075 stitches together the brightness deviation, contrast deficiency, and blurring degree of each image region in the order of the regions to form a region degradation characterization matrix.

7. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 1, characterized in that, S3 includes: S301 calculates the strategy score of each image region for each enhancement processing type based on the region degradation representation matrix; S302 arranges the strategy scores by region index and enhancement processing type index to obtain the enhancement strategy matrix; S303 performs Softmax normalization on the enhancement strategy matrix along the enhancement processing type dimension to obtain the relative weight allocation matrix; S304 performs Sigmoid activation on each element in the enhancement policy matrix and then performs threshold truncation amplification to obtain an independent execution strength matrix. S305 extends the semantic space gating matrix along the augmentation processing type dimension and multiplies it element-wise with the relative allocation weight matrix and the independent execution strength matrix to obtain the region augmentation weight matrix.

8. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 1, characterized in that, S4 includes: S401 performs pixel space mapping on the region enhancement weight matrix, semantic space gating matrix and brightness deviation correction direction to obtain an initial spatial enhancement weight map, a pixel-by-pixel semantic gating map and an initial brightness correction direction map that are consistent with the pixel grid of the construction site image. S402 uses the brightness image of the construction site image as the guide image, performs guided filtering on the initial spatial enhancement weight map and the initial brightness correction direction map to obtain the enhancement weight map and the pixel-wise brightness correction direction map after edge smoothing, and performs constraint normalization on the enhancement weight after edge smoothing according to the pixel-wise semantic gating map. S403 performs multiple types of enhancement processing on construction site images, generating brightness correction enhancement results, local contrast enhancement results, and edge texture detail enhancement results, respectively. S404 calculates the pixel residuals of the three enhancement results relative to the construction site image, and obtains the residual map; The S405 performs pixel-by-pixel weighted fusion of pixel residuals based on the normalized pixel space enhancement weight map, and limits the pixel values ​​of the weighted fusion result to a preset effective range to obtain the enhanced construction site image.

9. The construction site image enhancement method based on hierarchical semantic gating and regional degradation assessment according to claim 8, characterized in that, S403 includes: S4031 performs adaptive brightness correction on the construction site image based on the pixel-by-pixel brightness correction pattern to obtain the brightness correction enhancement result; S4032 performs local contrast enhancement on the construction site image, and obtains the contrast enhancement result; S4033 performs edge and texture detail enhancement on the construction site image, resulting in enhanced detail.

Images