Filling industry low light image restoration method and device based on text semantic guidance

Abstract

The application relates to the technical field of visual detection in the filling industry, in particular to a filling industry low-light image restoration method and device based on text semantic guidance, and a brand-new image restoration model is proposed based on an SFHformer model, a text guidance semantic modulation module, a double-branch multi-scale strip modeling module and a self-adaptive feature enhancement module are added, semantic guidance is carried out in the image restoration process, and opposite direction structures are modeled, edge and texture information are strengthened, statistical information of channels is extracted, effective features are enhanced, and the quality of image detail restoration is improved. The technical problems that the existing low-light image restoration algorithm has the problems of insufficient brightness recovery, incomplete detail restoration and insufficient noise suppression capacity in the filling industry scene, and further leads to the problems of insufficient image brightness, low contrast, blurred edges and lost details in the image collected in the industrial filling production process, which affect the accuracy of subsequent visual tasks such as barrel cap positioning, defect detection and instance segmentation.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology in the filling industry, and in particular to a method and apparatus for low-light image restoration in the filling industry based on text semantic guidance. Background Technology

[0002] In industrial bottling production, factors such as changes in lighting, insufficient camera exposure, and reflections from metal barrels can easily lead to problems in acquired images, including insufficient brightness, low contrast, blurred edges, and loss of detail. This negatively impacts the accuracy of subsequent visual tasks such as lid positioning, defect detection, and instance segmentation. Therefore, low-light image restoration is a crucial preprocessing step in industrial machine vision workflows. Existing low-light image restoration methods primarily employ traditional image enhancement and deep learning-based approaches.

[0003] Traditional image enhancement methods can improve image brightness to some extent, but they are limited by the following three aspects: uneven lighting distribution in industrial scenes can easily lead to local over-enhancement or under-enhancement; the surface of metal barrels is highly reflective, and color distortion and noise amplification are likely to occur after enhancement; the texture details of the barrel lid and the edge of the barrel body are complex, and traditional methods are difficult to adaptively recover effective structural information.

[0004] Deep learning-based methods can achieve better brightness enhancement and detail restoration than traditional methods, but they have three shortcomings: most models rely solely on the image's own features for restoration, lacking prior guidance on degenerate semantics; they lack the ability to model the structure of edge contours, strip textures, and local details in industrial scenes; and they tend to retain redundant background responses and noise interference during feature reconstruction, resulting in unclear details and insufficient color consistency in the restored image.

[0005] Therefore, a fast industrial low-light image restoration model is needed to restore images of filling barrels. This model can incorporate low-light degradation semantic priors, enhance the directional structure representation of industrial images, suppress redundant noise responses, and possess detail recovery capabilities. This will provide a clear, stable, and reliable image foundation for subsequent visual tasks such as target detection, instance segmentation, defect identification, and localization analysis. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a method and apparatus for low-light image restoration in the filling industry based on text semantic guidance, which solves the technical problems of insufficient brightness restoration, incomplete detail restoration and insufficient noise suppression ability of existing low-light image restoration algorithms in filling industry scenarios. As a result, the images acquired during the industrial filling production process have insufficient brightness, low contrast, blurred edges and loss of details, which affect the accuracy of subsequent visual tasks such as lid positioning, defect detection and instance segmentation.

[0007] In a first aspect, the present invention provides a method for low-light image restoration in the filling industry based on text semantic guidance, the method comprising:

[0008] An image restoration model is constructed, which includes a shallow feature extraction layer (SFEL), an encoder, a deep feature modeling stage (DFMS), a decoder, and an image reconstruction layer (IRL). The encoder, the DFMS, and the decoder all include an SFHformer Block module.

[0009] To add explicit textual semantic guidance to the image restoration model, firstly, textual prompts required for the restoration of low-light degraded images are constructed. The textual prompts are input into the Degraded Text Prior Encoder (DTPE) to extract text embedding vectors, and semantic prior vectors are generated through mean fusion. Subsequently, the semantic prior vectors are injected into the SFHformer Block module using the Feature Text Modulation (FTM) module, providing explicit textual semantic guidance for the image restoration process.

[0010] Acquire low-light degraded images.

[0011] The acquired low-light degraded image is input into the image restoration model. Combined with the explicit textual semantic guidance provided in the above steps, the low-light degraded image is restored, and the restored image is finally output.

[0012] Furthermore, the Degenerate Text Prior Encoder (DTPE) employs a Da-clip-based pre-trained text encoder to encode the input text prompts into fixed-dimensional text embedding vectors. When more than one text prompt is required, each text prompt is first encoded separately using the Degenerate Text Prior Encoder (DTPE) to obtain a 512-dimensional text embedding vector corresponding to the number of prompts. After aggregation using a mean fusion strategy, a semantic prior vector is obtained.

[0013] Furthermore, the Feature Text Modulation (FTM) module maps the semantic prior vector to a space consistent with the number of visual channels through two independent linear mappings, generating channel-level scaling parameters and offset parameters. The channel-level scaling parameters are used to adaptively scale the channel features, enhancing responses related to degenerate semantics. The offset parameters are used to supplement offset information, enhancing the flexibility of feature expression.

[0014] Furthermore, the constructed image restoration model also includes a dual-branch multi-scale strip modeling module (DMSM); the dual-branch multi-scale strip modeling module (DMSM) performs structural modeling along the height and width directions respectively, and combines multi-scale strip convolution with an adaptive direction fusion unit to enhance the image restoration model's ability to express edge contours and texture details.

[0015] Furthermore, the dual-branch multi-scale strip modeling module DMSM first processes the input features along the height and width directions. Compression is performed to model structural information in both directions; in the height branch, the input features are first processed... Aggregate each column and extract statistical information along the height direction to obtain... The overall response information of directional features is then extracted through average pooling, and max pooling is used to capture features of edge regions. Subsequently, the two pooling results are concatenated along the channel dimension and fused through 1×1 convolution, enabling the image restoration model to comprehensively utilize the overall response and local salient response in the directional features. The width branch uses the same operation to extract statistical information along the width direction. Then, the height and width direction feature responses are respectively input into multi-scale strip convolution, and strip convolution with dilation rates of 1, 2, and 3 is used to construct a multi-scale receptive field in a single direction, capturing local detail information and long-distance dependencies. The two branches are then normalized by the Sigmoid function to obtain the height and width direction attention weights. Finally, the height and width direction attention weights are input into the attention optimization fusion module AOF to complete the adaptive weighted fusion of attention features. The height and width attention weights are first input into the shared gated mapping branch SGMB for feature extraction, and then compressed into height and width response intensities through global average pooling. These are then stacked and input into a linear mapping layer, where they are normalized using the Softmax activation function to obtain the directional fusion coefficients. and This is used to measure the contribution ratio of two feature fusions, and adaptive feature fusion is achieved by explicitly learning the importance between directions.

[0016] Furthermore, the constructed image restoration model also includes an adaptive feature enhancement module (AFE); the AFE enhances effective structural features, suppresses background noise and redundant responses, and improves the quality of image detail restoration.

[0017] Furthermore, the adaptive feature enhancement module (AFE) first processes the input features... Global average pooling and global max pooling operations are performed separately to obtain two channel description vectors of size C×1×1, which are used to represent the statistical information and significant response of the feature in the global scope, respectively. These vectors are then fed into a shared multilayer perceptron (Shared MLP) to model the dependencies between different channels and obtain the corresponding channel responses. and Then, the two branches are added element by element, and the weights representing the importance of each channel are obtained by passing the Sigmoid activation function. Next, the input features are analyzed. Element-wise modulation is performed to obtain enhanced features. Finally, feature updates are performed using residuals, and the output of the adaptive feature enhancement module (AFE) is... .

[0018] Furthermore, after the dual-branch multi-scale strip modeling module DMSM and the adaptive feature enhancement module AFE complete independent modeling, their outputs are fused to obtain the fused features. Features after fusion It fully integrates the complementary advantages of directional structure modeling and channel response optimization.

[0019] Secondly, the present invention also provides a low-light image restoration device for the filling industry based on text semantic guidance, the device comprising:

[0020] An image restoration model construction module is used to construct an image restoration model. The image restoration model includes a shallow feature extraction layer (SFEL), an encoder (Encoder), a deep feature modeling stage (DFMS), a decoder (Decoder), and an image reconstruction layer (IRL). The encoder, the DFMS, and the decoder all include an SFHformer Block module.

[0021] The Text-Guided Semantic Modulation Module (TGSM) is used to construct text prompts required for low-light degraded image restoration. The text prompts are input into the Degraded Text Prior Encoder (DTPE) to extract text embedding vectors, and semantic prior vectors are generated through mean fusion. Subsequently, the Feature Text Modulation Module (FTM) is used to inject the semantic prior into the SFHformer Block module, providing explicit text semantic guidance for the image restoration process.

[0022] The image acquisition module is used to acquire low-light degraded images;

[0023] The image restoration module is used to input the acquired low-light degraded image into the image restoration model, and combine it with the explicit text semantic guidance provided by the text-guided semantic modulation module TGSM to restore the low-light degraded image, and finally output the restored image.

[0024] Furthermore, the aforementioned text semantic-guided low-light image restoration device for the filling industry also includes a dual-branch multi-scale strip modeling module (DMSM) and an adaptive feature enhancement module (AFE). The DMSM is used to perform structural modeling of the encoded deep features along the height and width directions respectively, and combines multi-scale strip convolution with an adaptive direction fusion unit to enhance the model's ability to express edge contours and texture details. The AFE is used to enhance the effective structural features of the encoded deep features, suppress background noise and redundant responses, and improve the quality of image detail restoration.

[0025] Compared with the prior art, this application has the following beneficial effects:

[0026] This invention addresses the problem in existing image restoration technologies that rely solely on visual features to restore low-light images, lacking explicit semantic constraints, resulting in insufficient structural feature response and unclear restoration direction. It utilizes a text-guided semantic modulation module (TGSM) to first extract text embedding vectors from the text using a degenerate text prior encoder (DTPE), then fuses these vectors to generate a semantic prior vector. Finally, a feature text modulation module (FTM) injects this semantic prior vector into the SFHformer Block of the image restoration model, providing explicit semantic guidance for image restoration and accurately capturing the image recovery direction.

[0027] This invention addresses the problems of blurred contours, weakened texture details, and degraded structural information in low-light images from the filling industry. It utilizes a dual-branch multi-scale strip modeling module (DMSM) to perform structural modeling of edge contours, barrel surface textures, and directional structural information in filling industry images along both the height and width directions. Furthermore, it combines multi-scale strip convolution with an adaptive directional fusion unit to enhance the model's ability to express edge contours and texture details.

[0028] This invention addresses the problem of redundant responses and noise in the features extracted by the encoder, which affects the quality of image restoration. It utilizes an adaptive feature enhancement module (AFE) to extract statistical information of channels through global pooling operations, thereby enhancing effective features and improving the quality of image detail restoration. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1The flowchart illustrates a text-based semantic-guided low-light image restoration method for the filling industry, according to an exemplary embodiment.

[0031] Figure 2 This is a structural diagram of the image restoration model TGFE-SFHformer according to an exemplary embodiment.

[0032] Figure 3 This is a structural diagram of a text-guided semantic modulation module (TGSM) according to an exemplary embodiment.

[0033] Figure 4 This is a structural diagram of a dual-branch multi-scale strip modeling module (DMSM) according to an exemplary embodiment.

[0034] Figure 5 This is a structural diagram of an adaptive feature enhancement module (AFE) according to an exemplary embodiment.

[0035] Figure 6 The results of an ablation experiment on the number of prompt words in the Text-Guided Semantic Modulation Module (TGSM) according to an exemplary embodiment are shown.

[0036] Figure 7 The results of the text guidance semantic modulation module TGSM for the experimental ablation of prompt words are shown according to an exemplary embodiment.

[0037] Figure 8 This is a flowchart illustrating the TGFE-SFHformer image restoration model for restoring an input low-light degraded image according to an exemplary embodiment.

[0038] Figure 9 This is a business flow diagram illustrating the use of a text-based semantic-guided low-light image restoration method in the filling industry, according to an exemplary embodiment.

[0039] Figure 10 This is a flowchart illustrating the workflow of a filling machine using a text semantic-guided low-light image restoration method for the filling industry, according to an exemplary embodiment.

[0040] Figure 11 This is a framework diagram of a text-based semantic guidance-guided low-light image restoration device for the filling industry, according to an exemplary embodiment.

[0041] In the above figure:

[0042] 100. Image Restoration Model Construction Module, 110. Shallow Feature Extraction Layer (SFEL), 120. Encoder, 130. Deep Feature Modeling Stage (DFMS), 140. Two-Branch Multi-Scale Strip Modeling Module (DMSM), 150. Adaptive Feature Enhancement Module (AFE), 160. Decoder, 170. Image Reconstruction Layer (IRL), 200. Text-Guided Semantic Modulation Module (TGSM), 300. Image Acquisition Module, 400. Image Restoration Module. Detailed Implementation

[0043] To better explain the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of this application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0044] Example 1

[0045] See Figure 1 This embodiment provides a method for low-light image restoration in the filling industry based on text semantic guidance. The method includes:

[0046] S10, construct the image restoration model TGFE-SFHformer.

[0047] See Figure 2 In step S10, the constructed image restoration model TGFE-SFHformer includes a shallow feature extraction layer SFEL, an encoder, a deep feature modeling stage DFMS, a dual-branch multi-scale strip modeling module DMSM, an adaptive feature enhancement module AFE, a decoder, and an image reconstruction layer IRL. The encoder, the deep feature modeling stage DFMS, and the decoder all include stacked SFHformer Block modules, and the SFHformerBlock module includes a text-guided semantic modulation module TGSM.

[0048] In this embodiment, the image restoration model TGFE-SFHformer is based on the existing SFHformer model, and incorporates the text-guided semantic modulation module TGSM, the dual-branch multi-scale strip modeling module DMSM, and the adaptive feature enhancement module AFE. Therefore, the specific structures of the shallow feature extraction layer SFEL, the encoder, the deep feature modeling stage DFMS, the decoder, and the image reconstruction layer IRL in the image restoration model TGFE-SFHformer can refer to the existing SFHformer model, and will not be elaborated here.

[0049] See Figure 3 The text-guided semantic modulation module (TGSM) includes a degenerate text prior encoder (DTPE) and a feature text modulation module (FTM). The TGSM first introduces manually constructed degenerate descriptive text as a degenerate semantic prior, which is then fed into the DTPE for encoding. Finally, the text is aggregated using a mean fusion strategy to obtain a semantic prior vector. Subsequently, it is fed into the Feature Text Modulation (FTM) module, where a channel-level scaling parameter γ and an offset parameter β are generated through linear mapping to adjust the visual features. Channel-level scaling and bias modulation are performed to obtain the visual features after text semantic modulation. .

[0050] The text-guided semantic modulation module TGSM transforms low-light enhancement-related text prompts into semantic prior information through degenerate text prior encoding and feature text modulation, and injects it into the backbone network features. This can effectively enhance the image restoration model TGFE-SFHformer's perception of the direction of brightness restoration, contrast enhancement and detail repair, and make up for the lack of explicit semantic guidance in traditional low-light restoration models.

[0051] See Figure 4 The dual-branch multi-scale strip modeling module (DMSM) decomposes two-dimensional features from both height and width directions. It models the edge contours, barrel textures, and directional structural information in industrial images along both height and width directions, extracting edge contours, barrel textures, and directional structural information from the image. It also dynamically allocates the response ratios of different directional features using adaptive directional fusion, effectively enhancing the ability of the image restoration model TGFE-SFHformer to express structural details in industrial low-light images, compensating for edge blurring and texture weakening. Furthermore, by combining multi-scale strip convolution with the adaptive directional fusion unit, it further enhances the ability of the image restoration model TGFE-SFHformer to express structural information and texture details.

[0052] See Figure 5 The adaptive feature enhancement module (AFE) is used to dynamically optimize channel features. It extracts channel statistical information through global pooling operations, enhances effective feature responses, and suppresses redundant responses related to background or noise. This effectively improves the detail expression capability of the restored image, compensates for the weakening of key structural information in the encoded features, and improves the expression quality of features related to structural information such as target edge contours, region boundaries, and local textures. As a result, it improves the restoration quality of industrial low-light images and provides clear and reliable image input for subsequent visual tasks such as bucket lid positioning, bucket surface defect detection, and instance segmentation.

[0053] S20, add explicit textual semantic guidance to the image restoration model TGFE-SFHformer. First, construct the text prompts required for the restoration of low-light degraded images. Input the text prompts into the Degraded Text Prior Encoder (DTPE) to extract text embedding vectors, and generate semantic prior vectors through mean fusion. Then, use the Feature Text Modulation (FTM) module to inject the semantic prior vectors into the SFHformer Block module, providing explicit textual semantic guidance for the image restoration process.

[0054] Specifically, the Degenerate Text Prior Encoder (DTPE) employs a Da-clip-based pre-trained text encoder to encode the input text prompts into fixed-dimensional text embedding vectors. To ensure the stability of the text embedding vectors, the parameters of the DTPE are frozen during model training, serving only as a text embedding extraction module. If the task requires multiple text representations, the DTPE can be used to encode each text prompt separately, resulting in multiple 512-dimensional text embedding vectors. Let the text embedding vector corresponding to the i-th text prompt be... Multiple text embedding vectors are aggregated using a mean fusion strategy to obtain a semantic prior vector. The calculation formula is as follows.

[0055]

[0056] Where N represents the number of input text prompts, which can be set according to the specific task.

[0057] Semantic prior vectors during model training and inference As a shared semantic condition input into the SFHformer Block module, it retains the complementary semantics of multiple text prompts while reducing the impact of deviations caused by single text expressions.

[0058] Specifically, the Feature Text Modulation (FTM) module modulates the semantic prior vectors through two independent linear mappings. Mapping to a space consistent with the number of visual channels generates channel-level scaling parameters. and offset parameters Channel-level scaling parameters Used for adaptive scaling of channel features to enhance responses related to degenerate semantics; offset parameter Used to supplement offset information and enhance the flexibility of feature representation, the calculation formulas for the two are as follows: , ,in, and Represents the learnable parameter matrix, and This is a learnable bias term.

[0059] Before the Feature Text Modulation (FTM) module is embedded into the LGPM module in the SFHformer Block module, the normalized features are modulated to achieve multi-stage consistent semantic guidance.

[0060] In this embodiment, to understand the impact of the number of text prompt words in the Text-Guided Semantic Modulation (TGSM) module on the performance of the image restoration model TGFE-SFHformer, appropriate semantic guidance can effectively improve the model's restoration capability. Experimental results for image restoration using different numbers of text prompt words are shown below. Figure 6 As shown.

[0061] from Figure 6 It can be seen that for the evaluation metrics PSNR, SSIM, and LPIPS, the optimal result is achieved when the number of text prompt words is 5; for the NIQE metric, the optimal result is achieved when the number of prompt words is 2. Therefore, in this embodiment, the optimal configuration of the image restoration model TGFE-SFHformer is selected as 5 text prompt words.

[0062] In this embodiment, to further analyze the impact of the semantic content of text prompts on the performance of the image restoration model TGFE-SFHformer, ablation experiments were designed with different semantic types of prompts, assuming a fixed number of 5 text prompts. The prompts were categorized into three types: normal semantic prompts consistent with the task objective, negative semantic prompts contrary to the task objective, and irrelevant semantic prompts unrelated to the task objective. The experimental results are as follows: Figure 7 As shown.

[0063] Experimental results show that normal semantic cue words achieve the best results for all three evaluation metrics: PSNR, SSIM, and LPIPS. Since semantic information consistent with the task objective can provide more effective degradation priors for the image restoration model TGFE-SFHformer, improving image brightness restoration, structure preservation, and perceptual quality, the TGFE-SFHformer selects cue words highly relevant to the task to fully leverage the guiding role of text priors.

[0064] S30, acquire low-light degradation image.

[0065] S40, the acquired low-light degraded image is input into the image restoration model TGFE-SFHformer, and combined with the explicit text semantic guidance provided in step S20, the low-light degraded image is restored, and the restored image is finally output.

[0066] See Figure 8 Step S40 specifically includes the following steps:

[0067] S41, the low-light degraded image first passes through the shallow feature extraction layer SFEL to obtain initial features;

[0068] S42, then the encoder encodes and downsamples the data to obtain deep semantic features at different scales;

[0069] S43, then deep feature extraction is performed through the deep feature modeling stage DFMS;

[0070] S44, the encoded deep features are input into the dual-branch multi-scale strip modeling module DMSM, which performs structural modeling along the height and width directions respectively, and combines multi-scale strip convolution with adaptive direction fusion unit to enhance the model's ability to express edge contours and texture details.

[0071] Specifically, the dual-branch multi-scale strip modeling module DMSM first processes the input features along the height and width directions. Compression is performed to model structural information in both directions; in the height branch (HeightBranch), the input features are first processed... Aggregate each column and extract statistical information along the height direction to obtain... The overall response information of directional features is then extracted using average pooling, and max pooling is used to capture features of edge regions. Subsequently, the two pooling results are concatenated along the channel dimension and fused using a 1×1 convolution, enabling the image restoration model TGFE-SFHformer to comprehensively utilize both the overall and local salient responses in the directional features. The width branch uses the same operation to extract statistical information along the width direction. Then, the height and width direction feature responses are input into multi-scale strip convolutions, using strip convolutions with dilation rates of 1, 2, and 3 to construct multi-scale receptive fields in a single direction, simultaneously capturing local details and long-range dependencies. Both branches are then normalized using the Sigmoid function to obtain the height direction attention weights. Attention weights in the width direction Next, the height-oriented attention weights are adjusted. Attention weights in width direction Input attention optimization and fusion module AOF to complete adaptive weighted fusion of attention features; height-oriented attention weights Attention weights in width direction First, the shared gated mapping branch SGMB is input for feature extraction. Then, global average pooling is used to compress the features into response intensities in the height and width directions. These two responses are then stacked and input into a linear mapping layer. After normalization using the Softmax activation function, the directional fusion coefficients are obtained. and This is used to measure the contribution ratio of two feature fusions. Adaptive feature fusion is achieved by explicitly learning the importance between directions. The output is the directional fusion feature. The calculation formula is: ,in, The learnable residual scaling factor is used to control the injection intensity of the enhanced features of the dual-branch multi-scale strip modeling module (DMSM).

[0072] S45, while the encoded deep features are input into the dual-branch multi-scale strip modeling module DMSM, the encoded deep features are also input into the adaptive feature enhancement module AFE to enhance effective structural features, suppress background noise and redundant responses, and improve the quality of image detail restoration.

[0073] Specifically, the Adaptive Feature Enhancement (AFE) module first processes the input features... Global average pooling and global max pooling operations are performed separately to obtain two channel description vectors of size C×1×1, which are used to represent the statistical information and significant response of the feature in the global scope, respectively. These vectors are then fed into a Shared MLP to model the dependencies between different channels and obtain the corresponding channel responses. and Then, the two branches are added element by element, and the weights representing the importance of each channel are obtained by passing the Sigmoid activation function. Next, the input features are analyzed. Element-wise modulation is performed to obtain enhanced features. Enhanced features The calculation formula is: Finally, to ensure the stability of the model during the enhancement process, residuals are used for feature updating to obtain the output of the adaptive feature enhancement module (AFE). The calculation formula is: ,in, These are learnable parameters used to adaptively control the injection strength of enhanced information.

[0074] Using the above method, the adaptive feature enhancement module (AFE) can highlight structural information such as target edge contours, region boundaries, and local textures without destroying the original feature distribution, and suppress redundant responses related to background or noise, thus effectively improving the ability to express detailed features of the restored image.

[0075] S46, after the dual-branch multi-scale strip modeling module DMSM and the adaptive feature enhancement module AFE complete independent modeling, their outputs are fused to obtain the fused features. Features after fusion It fully integrates the complementary advantages of directional structure modeling and channel response optimization. The calculation formula is as follows: ,in, and The learnable residual scaling factor is used to control the injection intensity of the enhanced information of the dual-branch multi-scale strip modeling module (DMSM) and the adaptive feature enhancement module (AFE), respectively. Both are optimized together with the network parameters during training.

[0076] S47, then the decoder gradually upsamples and decodes, and the features of the encoding stage are fused through skip connections.

[0077] S48, and finally the restored image is output through the image reconstruction layer IRL.

[0078] The following is a detailed description of the application of this method in visual recognition and inspection of filling barrels using a specific embodiment. It is applicable to low-light image restoration in filling industrial production scenarios such as hazardous chemical filling, container lid inspection, and barrel surface appearance inspection. It can provide high-quality input images for subsequent visual tasks such as target detection, instance segmentation, defect identification, and localization analysis.

[0079] Taking a fully automatic filling machine as an example, the fully automatic filling machine has a host computer, a slave computer, an industrial camera for acquiring images of packaging containers, a filling business execution module, image restoration software, and machine vision recognition and detection software. The image restoration software embeds the aforementioned image restoration model TGFE-SFHformer, and the machine vision recognition and detection software embeds a machine vision recognition and detection model. The base image restoration model TGFE-SFHformer includes a shallow feature extraction layer SFEL, an encoder, a deep feature modeling stage DFMS, a dual-branch multi-scale strip modeling module DMSM, an adaptive feature enhancement module AFE, a decoder, and an image reconstruction layer IRL. The encoder, the deep feature modeling stage DFMS, and the decoder all include stacked SFHformer Block modules, and the SFHformer Block module includes a text-guided semantic modulation module TGSM.

[0080] See Figure 9 , Figure 9 This demonstrates the overall process of filling operations using a text-based semantic-guided low-light image restoration method.

[0081] First, the host computer, slave computer, image restoration software, and machine vision recognition and detection software communicate in real time. After the host computer issues a detection task, the industrial camera captures real-time images, which are then preprocessed and input into the base image restoration model TGFE-SFHformer for low-light degradation image restoration. The output restored image is then input into the machine vision recognition and detection model for machine vision tasks such as lid recognition or barrel defect detection. The processing result is then returned to the slave computer, which controls the filling business execution module to complete the subsequent filling task.

[0082] See Figure 10 , Figure 10 The flowchart illustrates the workflow of a filling machine using a text-based semantic-guided low-light image restoration method for the filling industry. The specific process includes:

[0083] S1, Image Acquisition Step: Acquire low-light degradation images of packaging containers on the filling production line.

[0084] S2, Image preprocessing step, performs preprocessing operations such as size adjustment and normalization on the acquired low-light degradation image of the packaging container to make it meet the input requirements of the image restoration model TGFE-SFHformer.

[0085] S3, the text semantic guidance step, constructs text prompts related to the restoration of low-light degraded images of packaging containers, such as "enhance brightness in low-light images", "suppress noise in dark regions", "reduce brightness degradation and detail loss", etc. The text prompts are input into the degraded text prior encoder DTPE to extract text embedding vectors, and semantic prior vectors are generated by mean fusion. Subsequently, the semantic prior vectors are injected into the SFHformer Block module using the Feature Text Modulation (FTM) module to provide explicit semantic guidance for the image restoration process.

[0086] S4, Feature extraction step: Input the preprocessed low-light degradation image of the packaging container into the backbone network of the base image restoration model TGFE-SFHformer, extract multi-scale visual features, and combine them with textual semantic guidance information to enhance the perception ability of the base image restoration model TGFE-SFHformer to low-light degradation areas.

[0087] S5, the structural modeling and feature enhancement step, inputs the encoded deep features into the dual-branch multi-scale strip modeling module DMSM to model the packaging container edge, surface texture and directional structural information along the height and width directions respectively; at the same time, it is input into the adaptive feature enhancement module AFE to enhance the effective structural features and suppress background noise and redundant responses, thereby improving the quality of image detail restoration.

[0088] S6, the image reconstruction step, inputs the enhanced features into the decoder, restores the image spatial resolution through stepwise upsampling and skip connections, and finally generates a restored image with enhanced brightness and clear details.

[0089] S7, the identification and detection step, then inputs the restored image into the machine vision identification and detection model for lid identification and positioning, barrel surface defect detection or status identification.

[0090] S8 controls the execution steps, and the processing results are returned to the lower-level machine. The lower-level machine then controls the filling business execution module to complete the subsequent filling task.

[0091] Example 2

[0092] See Figure 11This embodiment provides a text-based semantic guidance-guided low-light image restoration device for the filling industry, the device comprising:

[0093] The image restoration model construction module is used to construct the image restoration model TGFE-SFHformer. The image restoration model TGFE-SFHformer includes a shallow feature extraction layer (SFEL), an encoder, a deep feature modeling stage (DFMS), a dual-branch multi-scale strip modeling module (DMSM), an adaptive feature enhancement module (AFE), a decoder, and an image reconstruction layer (IRL). The encoder, the deep feature modeling stage (DFMS), and the decoder all include the SFHformer Block module.

[0094] The Text-Guided Semantic Modulation Module (TGSM) is used to construct text prompts required for the restoration of low-light degraded images. The text prompts are input into the Degraded Text Prior Encoder (DTPE) to extract text embedding vectors, and semantic prior vectors are generated through mean fusion. Subsequently, the semantic prior is injected into the SFHformer Block module using the Feature Text Modulation Module (FTM) to provide explicit text semantic guidance for the image restoration process.

[0095] The image acquisition module is used to acquire low-light degraded images.

[0096] The image restoration module performs preprocessing operations such as resizing and normalization on the acquired low-light degraded image to meet the input requirements of the base image restoration model TGFE-SFHformer. The preprocessed low-light degraded image is then input into the backbone network of the TGFE-SFHformer to extract multi-scale visual features. Combined with textual semantic guidance information, this enhances the TGFE-SFHformer's ability to perceive low-light degraded regions. The encoded deep features are then input into the dual-branch multi-scale strip modeling module DMSM to model the packaging container edges, surface textures, and directional structural information along the height and width directions, respectively. Simultaneously, these features are input into the adaptive feature enhancement module AFE to enhance effective structural features and suppress background noise and redundant responses, thereby improving the quality of image detail restoration. Finally, the enhanced features are input into the decoder, which restores the image spatial resolution through stepwise upsampling and skip connections, ultimately generating a restored image with enhanced brightness and clear details.

[0097] The principle of the low-light image restoration device for the filling industry based on text semantic guidance is similar to that of the low-light image restoration method for the filling industry based on text semantic guidance. Therefore, the implementation of the low-light image restoration device for the filling industry based on text semantic guidance can refer to the implementation of the method, and the repeated parts will not be described again.

[0098] Example 3

[0099] Based on the same technical concept, this embodiment also provides an electronic device, including at least one processor and a memory connected to at least one processor. In this embodiment, the processor and the memory are connected via a bus, but the specific connection medium between the processor and the memory is not limited.

[0100] The processor is the control center of the electronic device. It can connect to various parts of the electronic device through various interfaces and lines, and performs data processing by running or executing instructions stored in memory and retrieving data stored in memory. Optionally, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user page, and application programs, while the modem processor mainly handles issuing instructions.

[0101] The processor may be a general-purpose processor that can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application.

[0102] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc.

[0103] In this embodiment, the memory stores a computer program that, when executed by the processor, causes the processor to perform the text semantic-guided low-light image restoration method for the filling industry described above.

[0104] Since the electronic device is the same as the electronic device in the method of this embodiment, and the principle of the electronic device in solving the problem is similar to that of the method, the implementation of the electronic device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0105] The foregoing has provided a detailed description of a text-based semantic guidance method and apparatus for low-light image restoration in the filling industry. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0106] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented using software plus necessary general-purpose hardware platforms, or of course, using hardware. Furthermore, when the computer program in the memory is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium.

Claims

1. A method for low-light image restoration in the filling industry based on text semantic guidance, characterized in that, The method includes: An image restoration model is constructed, which includes a shallow feature extraction layer (SFEL), an encoder, a deep feature modeling stage (DFMS), a decoder, and an image reconstruction layer (IRL). The encoder, the deep feature modeling stage (DFMS), and the decoder all include an SFHformer Block module. To add explicit textual semantic guidance to the image restoration model, firstly, text prompts required for the restoration of low-light degraded images are constructed. The text prompts are input into the Degraded Text Prior Encoder (DTPE) to extract text embedding vectors, and semantic prior vectors are generated through mean fusion. Subsequently, the semantic prior vectors are injected into the SFHformerBlock module using the Feature Text Modulation (FTM) module, providing explicit textual semantic guidance for the image restoration process. Acquire low-light degraded images; The acquired low-light degraded image is input into the image restoration model. Combined with the explicit textual semantic guidance provided in the above steps, the low-light degraded image is restored, and the restored image is finally output.

2. The method according to claim 1, characterized in that, The Degenerate Text Prior Encoder (DTPE) employs a pre-trained text encoder based on Da-clip to encode the input text prompts into fixed-dimensional text embedding vectors. When more than one text prompt is required, the DTPE is used to encode each text prompt separately, resulting in 512-dimensional text embedding vectors for the corresponding number of prompts. These vectors are then aggregated using a mean fusion strategy to obtain semantic prior vectors.

3. The method according to claim 2, characterized in that, The Feature Text Modulation (FTM) module maps semantic prior vectors to a space consistent with the number of visual channels through two independent linear mappings, generating channel-level scaling parameters and offset parameters. The channel-level scaling parameters are used to adaptively scale the channel features and enhance responses related to degraded semantics. Offset parameters are used to supplement offset information and enhance the flexibility of feature representation.

4. The method according to claim 1, characterized in that, The constructed image restoration model also includes a dual-branch multi-scale strip modeling module (DMSM); the dual-branch multi-scale strip modeling module (DMSM) performs structural modeling along the height and width directions respectively, and combines multi-scale strip convolution with an adaptive direction fusion unit to enhance the image restoration model's ability to express edge contours and texture details.

5. The method according to claim 4, characterized in that, The dual-branch multi-scale strip modeling module DMSM first processes the input features along the height and width directions. Compression is performed to model the structural information in both directions; In the height branch, first analyze the input features. Aggregate each column and extract statistical information along the height direction to obtain... Then, average pooling is used to extract the overall response information of directional features, and max pooling is used to capture the features of edge regions. Subsequently, the two pooling results are concatenated along the channel dimension and fused using a 1×1 convolution, so that the image restoration model can comprehensively utilize the overall response and local salient response in the directional features. The width branch uses the same operation to extract statistical information along the width direction. Then, the height direction feature response and the width direction feature response are respectively input into multi-scale strip convolution. Strip convolution with dilation rates of 1, 2, and 3 is used to construct a multi-scale receptive field in a single direction, while capturing local detail information and long-distance dependencies. The two branches... After normalization using the Sigmoid function, the height and width attention weights are obtained. These weights are then input into the Attention Optimization and Fusion (AOF) module to achieve adaptive weighted fusion of attention features. The height and width attention weights are first input into the shared gated mapping branch SGMB for feature extraction, then compressed into height and width response intensities using global average pooling. These intensities are then stacked and input into a linear mapping layer, where they are normalized using the Softmax activation function to obtain the directional fusion coefficients. and This is used to measure the contribution ratio of two feature fusions, and adaptive feature fusion is achieved by explicitly learning the importance between directions.

6. The method according to claim 5, characterized in that, The constructed image restoration model also includes an adaptive feature enhancement module (AFE); the AFE enhances effective structural features, suppresses background noise and redundant responses, and improves the quality of image detail restoration.

7. The method according to claim 6, characterized in that, The adaptive feature enhancement module (AFE) first processes the input features. Global average pooling and global max pooling operations are performed separately to obtain two channel description vectors of size C×1×1, which are used to represent the statistical information and significant response of the feature in the global scope, respectively. These vectors are then fed into a Shared MLP to model the dependencies between different channels and obtain the corresponding channel responses. and Then, the two branches are added element by element, and the weights representing the importance of each channel are obtained by passing the Sigmoid activation function. Next, the input features are analyzed. Element-wise modulation is performed to obtain enhanced features. Finally, feature updates are performed using residuals to obtain the output of the adaptive feature enhancement module (AFE). .

8. The method according to claim 7, characterized in that, After the dual-branch multi-scale strip modeling module DMSM and the adaptive feature enhancement module AFE complete their independent modeling, their outputs are fused to obtain the fused features. Features after fusion It fully integrates the complementary advantages of directional structure modeling and channel response optimization.

9. A low-light image restoration device for the filling industry based on text semantic guidance, characterized in that, The device includes: An image restoration model construction module is used to construct an image restoration model. The image restoration model includes a shallow feature extraction layer (SFEL), an encoder (Encoder), a deep feature modeling stage (DFMS), a decoder (Decoder), and an image reconstruction layer (IRL). The encoder, the deep feature modeling stage (DFMS), and the decoder all include the SFHformerBlock module. The Text-Guided Semantic Modulation Module (TGSM) is used to construct text prompts required for low-light degraded image restoration. The text prompts are input into the Degraded Text Prior Encoder (DTPE) to extract text embedding vectors, and semantic prior vectors are generated through mean fusion. Subsequently, the Feature Text Modulation Module (FTM) is used to inject the semantic prior into the SFHformer Block module, providing explicit text semantic guidance for the image restoration process. The image acquisition module is used to acquire low-light degraded images; The image restoration module is used to input the acquired low-light degraded image into the image restoration model, and combine it with the explicit text semantic guidance provided by the text-guided semantic modulation module TGSM to restore the low-light degraded image, and finally output the restored image.

10. The apparatus according to claim 9, characterized in that, It also includes a dual-branch multi-scale strip modeling module (DMSM) and an adaptive feature enhancement module (AFE). The DMSM is used to perform structural modeling of the encoded deep features along the height and width directions, and combines multi-scale strip convolution with an adaptive direction fusion unit to enhance the model's ability to express edge contours and texture details. The AFE is used to enhance the effective structural features of the encoded deep features, suppress background noise and redundant responses, and improve the quality of image detail restoration.

Images