Text-driven power worker re-identification self-correction method and system

Abstract

The application discloses a kind of based on text drive's electric power operator re-identification self-correction method and system, belong to computer vision and figure re-identification technology.It includes: for the image of electric power operator to be searched, first, obtain image dataset and carry out multi-attribute dimension labeling;Second, train visual language model using dataset, obtain special model;Subsequently, the image features of the image to be searched are extracted, text query is obtained by special model, the visual features of dataset image are combined with text semantic features, and the mixed similarity score of the image to be searched and dataset image is calculated to obtain candidate image set, and further obtain feedback instruction;Text query, feedback instruction and the image to be searched are used to update text query and iteratively, and finally output candidate image set as similar search result.The method realizes the iterative self-correction of figure re-identification in closed loop architecture, and can effectively improve the figure re-identification precision in complex scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and person re-recognition technology, specifically relating to a text-driven self-correction method and system for re-recognition of power workers. Background Technology

[0002] Re-identification of power workers is a core technology in the field of power safety, aiming to match and retrieve the identity of the same worker across non-overlapping camera perspectives. In existing technologies, the mainstream approach to person re-identification is to learn discriminative visual embedding features of pedestrians based on convolutional neural networks or visual transformers, and then complete the retrieval task through nearest neighbor search. However, this purely visual approach is susceptible to severe occlusion, cluttered backgrounds, changes in shooting viewpoint, and interference from negative samples with highly similar appearances in complex real-world environments, leading to a significant decrease in identity matching accuracy.

[0003] In recent years, visual language models, with their powerful multimodal semantic understanding and instruction-following capabilities, have provided a new direction for the development of person re-identification technology. These technologies convert query images into text descriptions and perform text-guided pedestrian retrieval, thus mitigating the semantic understanding limitations of purely visual methods to some extent. However, existing person re-identification methods based on visual language models all adopt a static, "one-time pass" open-loop paradigm. They generate a text description or feature representation of the query image only once before performing the retrieval, without any correction mechanism for initial interpretation errors. This results in two major drawbacks: First, the image-to-text conversion process suffers from an information bottleneck, easily omitting fine-grained visual details crucial for pedestrian identification, such as logos, textures, and shoe styles, thus failing to support accurate identity retrieval. Second, visual language models are prone to generating hallucinatory attributes under challenging conditions such as poor lighting and pedestrian occlusion, and these errors continue to propagate during the retrieval process, ultimately leading to distorted retrieval results.

[0004] Meanwhile, existing visual language models are general-purpose models, lacking the fine-grained attribute sensitivity and identity verification logic required for person re-identification tasks, making it difficult to accurately capture pedestrian identity clues. Furthermore, existing technologies have not established an effective iterative optimization mechanism, and cannot improve retrieval accuracy by verifying candidate results, discovering contradictions, and refining search criteria in a way that resembles human retrieval behavior. In different person re-identification tasks such as standard scenarios, occluded scenarios, and fabric-changing scenarios, it is difficult to achieve universal and efficient identity matching. Therefore, a new person re-identification method is urgently needed to improve the accuracy of re-identification of power workers in complex scenarios. Summary of the Invention

[0005] The technical problem this invention aims to solve is that existing methods for re-identifying power workers suffer from significant discrepancies between visual features and semantic concepts of identity, information bottlenecks in image-to-text conversion, and a lack of fine-grained attribute recognition and verification capabilities required for person re-identification in general visual language models. These issues lead to low accuracy and poor robustness in power worker re-identification under complex scenarios. Therefore, this invention improves the model's identity matching accuracy and cross-scenario adaptability by customizing and optimizing a general visual language model into a dedicated power worker re-identification model.

[0006] In a first aspect, this invention proposes a text-driven self-correction method for re-identifying power workers, comprising: S1, obtain the images of the power workers to be retrieved and the image dataset containing the images of the power workers to be retrieved, and further annotate the images in the image dataset from multiple attribute dimensions; S2, using a pre-trained visual language model to generate a structured description of all attribute dimensions in the image of the power worker to be retrieved, and further obtain a structured text query; S3, extract the visual features and text semantic features of each image in the image dataset respectively, and combine the text query and image features of the power worker image to be retrieved to calculate the mixed similarity score between the power worker image to be retrieved and each image in the image dataset; S4. Based on the mixed similarity scores, sort the images from high to low and select the top K images in the image dataset as candidate images. Further combine the text query to calculate the attribute consistency score for each attribute dimension of the candidate image set. The pre-trained visual language model generates feedback instructions based on the attribute consistency score calculation results. S5, based on the text query, feedback instructions and the image of the power worker to be retrieved, infers a new text query and loops through S3-S5 until the loop stops when the loop stopping condition is met. The candidate image set obtained at this time is the similar image of the image to be retrieved in the image dataset.

[0007] Furthermore, in S1, the attribute dimensions include one or more of the following: gender, age group, hairstyle, type of top, color of top, type of bottom, color of bottom, footwear, personal items, and accessories.

[0008] Furthermore, the pre-training process of the visual language model, S2, specifically includes: S201, Obtain a training image dataset containing multiple images of people, and label the images in the image dataset from multiple attribute dimensions; S202, freeze all parameters of the visual language model and introduce an adapter into the attention layer of the visual language model to obtain a visual language model with the adapter; based on the training image dataset, construct pairwise verification training samples, attribute-specific question answering training samples, and correction feedback training samples respectively. S203, extract image features from the training image dataset, input the image features from the training image dataset into the visual language model of the adapter to obtain structured attribute descriptions, further calculate the cross-entropy loss between the structured attribute descriptions of the training image dataset and the corresponding annotations; update the adapter parameters according to the cross-entropy loss until convergence, and obtain the initially trained visual language model; S204, input the pairwise validation training samples, attribute-specific question-answering training samples and correction feedback training samples into the initially trained visual language model in sequence, and calculate the cross-entropy loss corresponding to the pairwise validation training samples, attribute-specific question-answering training samples and correction feedback training samples, and then add them together to obtain the total loss; S205, based on the total loss, update the adapter parameters on the basis of the initially trained visual language model until convergence, then stop the loop and obtain the pre-trained visual language model.

[0009] Furthermore, in S201, the pairwise verification training samples contain multiple image pairs composed of two images each. The visual language model outputs a judgment result based on a preset judgment question and calculates the cross-entropy loss between the result and the actual result of the image pair. The training samples for attribute-specific question answering contain multiple images. The visual language model generates a description of the corresponding attribute dimension in the image based on the query question according to a preset single attribute dimension. The cross-entropy loss is calculated based on the description and the actual attributes of the image. The correction feedback training samples include image pairs consisting of two similar images, erroneous or incomplete structured attribute descriptions corresponding to the similar images, and feedback instructions for the structured attribute descriptions. The visual language model generates feedback instructions based on the image pairs and the corresponding erroneous or incomplete structured attribute descriptions, and further calculates the cross-entropy loss between the generated feedback instructions and the preset feedback instructions.

[0010] Furthermore, in S3, the mixed similarity score is: ; in, This represents the weights used to control the visual and textual modalities. This indicates the image of the power workers to be retrieved. Visual features; Represents each image in the image database Visual features; Indicates the image to be retrieved. Textual semantic features obtained in round iteration; Represents each image in the image database Textual semantic features; Represents the L2 norm. Let represent the mixed similarity score calculated in the t-th iteration.

[0011] Furthermore, in S4, the calculation process for the attribute consistency score of each attribute dimension of the candidate image set is as follows: S401, perform semantic parsing on the text query to obtain atomic attribute value pairs corresponding to different attribute dimensions, and perform deduplication and normalization processing to obtain a structured attribute set composed of all processed atomic attribute value pairs; S402, using the attribute dimension as the unit, calculates the attribute consistency score of the current attribute dimension using the corresponding attribute dimension of all images in the candidate image set.

[0012] Furthermore, the calculation formula for S402 is as follows: ; in, This represents the total number of candidate images in the candidate image set; Represents the i-th image in the candidate image set. ; Indicates the first Each attribute dimension; Indicates the first The attribute values ​​corresponding to each attribute dimension in the structured attribute set; Indicates the attribute consistency score; Represents images in the candidate image set With attribute values ​​in a structured attribute set The probability of a match.

[0013] Furthermore, in step S4, a feedback instruction is generated based on the attribute consistency score calculation result, specifically as follows: First, determine whether the attribute consistency score of each attribute dimension is higher than the attribute consistency threshold. If it is higher than or equal to the attribute consistency threshold, no action is taken. If it is lower than the threshold, the attribute dimension is marked as a matching conflict point. Secondly, using a dedicated visual language model, it is determined whether the attribute dimension corresponding to the matching conflict point is an interfering attribute dimension from the candidate image or an attribute dimension that is not fully described in the text query. If it is an interfering attribute dimension from the candidate image, the dedicated visual language model generates a negative constraint feedback instruction. If it is an attribute dimension that is not fully described in the text query, the dedicated visual language model generates an attribute emphasis feedback instruction.

[0014] Furthermore, in S5, the loop stopping condition means that the current loop reaches the maximum number of loops, or the ratio of the number of intersection images to the number of union images between the candidate image set generated in the previous loop and the candidate image set generated in the current loop is greater than a threshold.

[0015] Secondly, this invention proposes a text-driven power worker re-identification self-correction system to implement the aforementioned text-driven power worker re-identification self-correction method.

[0016] The beneficial effects of this invention are: First, the general visual language model of this invention is customized into a special model, which improves the model's ability to perceive and understand pedestrian identity features.

[0017] Secondly, the closed-loop iterative self-correcting inference architecture constructed in this invention effectively solves the information bottleneck of image-to-text conversion and the error propagation problem of open-loop retrieval, significantly improving the accuracy of person re-identification.

[0018] Finally, the hybrid retrieval system of this invention integrates stable visual anchor features with dynamic semantic query features, effectively preventing semantic drift. This allows the model to maintain high recognition robustness even in complex scenarios such as severe occlusion, cluttered backgrounds, and fabric changes, demonstrating high practical value and promising prospects for widespread adoption. Attached Figure Description

[0019] Figure 1 This is an overall framework diagram of the present invention.

[0020] Figure 2 This is a detailed flowchart of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0022] like Figure 1 and Figure 2As shown, this invention proposes a text-driven self-correction method for re-identifying power workers based on a visual large language model. By customizing and optimizing a general visual language model and constructing a closed-loop iterative self-correction inference architecture, it integrates the dual advantages of visual features and textual semantics to solve problems such as information bottlenecks, error propagation, and semantic drift in person re-identification, significantly improving the accuracy of pedestrian identity matching in complex scenarios. The method includes the following steps: Step 1: Conduct fine-grained attribute alignment training to fine-tune the general visual language model, obtaining a specialized model adapted for the re-identification task of power workers. Obtain a people re-identification dataset (such as Occluded-Duke), and perform structured attribute annotation on pedestrian images within the dataset. The annotation dimensions include gender, estimated age, hairstyle (length / color), upper body clothing (type / color / pattern), lower body clothing (type / color), footwear, and personal items / accessories. Then, extract image features from the images of power workers in the people re-identification dataset using a pedestrian detection model (such as TransReID).

[0023] Freeze the backbone network weights of the general visual language model and add a low-rank adaptive (LoRA) adapter only to the model's attention layer, setting the LoRA's rank to 16, with 1e -5 The LoRA adapter was trained for 3 epochs using the learning rate, and the parameters were updated using mini-batch gradient descent. The specific training steps are as follows: The first stage is attribute alignment training: a fixed visual question-answering prompt template is designed, and the image features of the labeled pedestrian images and the prompt template are input into a general visual language model, so that the model generates a structured attribute description consistent with the labeling dimension. Cross-entropy loss is used as the optimization objective, thereby improving the model's ability to perceive fine-grained attributes and suppressing coarse-grained attribute illusions.

[0024] The second stage is identity verification and feedback generation training: construct three types of training samples: pairwise verification, attribute-specific question answering, and correction feedback synthesis. Calculate the corresponding cross-entropy loss based on the model output and sum them to obtain the total loss. Then, train the adapter parameters through multiple rounds and stages based on the total loss.

[0025] Among them, the pairwise verification training samples are input by the image features of two pedestrian images and a verification question, such as "Are the people in these two images the same person?" The visual language model generates a judgment result based on the image pair, calculates the corresponding cross-entropy loss with the actual verification result of the image pair, and trains the model to output the judgment result and generate the reasoning basis. The pairwise verification training samples are used to train the visual language model's consistency discrimination ability between two images. The attribute-specific question answering training samples are obtained by inputting image features of power workers and querying questions based on single attributes, such as "What is the color of the person's shirt?" The visual language model generates answers based on the corresponding attribute dimension of the question. The cross-entropy loss is calculated based on the model's answer and the image's true attribute in that dimension to train the model to accurately answer specific attributes of pedestrians. The training samples for corrective feedback use hard-negative samples or image pairs that the model easily confuses, along with erroneous or incomplete structured attribute descriptions for each image (pre-defined by the user). The visual language model generates feedback instructions based on the image pairs and their corresponding structured attribute descriptions. The cross-entropy loss is calculated using the model-generated feedback instructions and the pre-defined feedback instructions, thereby training the visual language model to generate feedback instructions that emphasize the differences between image pairs. This enables the model to generate negative constraint-type or attribute-emphasis-type natural language feedback instructions for erroneous or incomplete structured attribute descriptions.

[0026] The trained adapter is fused with the backbone network of a general visual language model to obtain a dedicated visual language model with the ability to re-identify power workers.

[0027] Step 2: Generate initial structured text queries based on a dedicated model First, the images of the power workers to be retrieved are acquired and normalized to a fixed size of 256×128 pixels to eliminate differences in image size and proportion. The image features of the normalized power workers images are extracted using a pedestrian detection model. The image features are then combined with the fixed visual question-and-answer prompt template from step one to form the model input sequence.

[0028] Subsequently, the input sequence is fed into the optimized visual language model to generate a structured description containing multi-dimensional, fine-grained attributes of pedestrians.

[0029] The generated structured description is further formatted, invalid information is removed, and the core identity-identifying attribute dimensions (i.e., annotation dimensions) are retained to form a standardized initial text query. : in, This represents the optimized model; This refers to the pedestrian image being queried; This represents a structured visual question-and-answer prompt template.

[0030] In subsequent iterations, the text query in each iteration It also needs to incorporate the feedback instructions generated in the previous iteration, which can be expressed as follows: in, Indicates the first Iterative text query; Indicates the first Feedback instructions for round iteration.

[0031] Step 3: Obtain the candidate image set using a hybrid retrieval system. First, visual anchor features and textual semantic features are extracted.

[0032] via visual encoder Stable visual features are extracted from the preprocessed query image and used as fixed visual anchors throughout the process. ; via CLIP text encoder Text query Embedding as text semantic features At the same time, each image in the personnel re-identification dataset or the database of real power workers is analyzed. through respectively , Extracting visual features and text semantic features .

[0033] Next, the mixed similarity score is calculated.

[0034] The visual similarity between the query image and each image in the image library is calculated using cosine similarity. and text similarity Further, the mixed similarity score is calculated using convex combinations. : in, This represents the weights used to control the visual and textual modalities, as shown in this specific embodiment. =0.65, to ensure the stability of visual anchor point retrieval; Indicates query image Visual feature vectors; Represents gallery images Visual feature vectors; Indicates the first In the round of iteration, query the image The corresponding text semantic feature vector; Represents gallery images The text semantic feature vector; This represents the L2 norm.

[0035] Finally, the candidate image set is selected. The mixed similarity score between the query image and all images in the image library is calculated. The images in the image library are then sorted from highest to lowest according to the mixed similarity score, and the top K images (in this specific embodiment, the top 20 images) are selected as the candidate image set. The input is then fed into the corrector module (in this specific embodiment, the corrector is the dedicated visual language model obtained in step one) for subsequent attribute deconstruction and consistency verification.

[0036] Step 4: Generate feedback commands using the corrector First, semantic destructuring is performed on the text query. Semantic parsing is performed to break down the overall description into independent atomic attribute value pairs. ,in For attribute dimensions, For the corresponding attribute values; the decomposed atomic attribute value pairs are deduplicated and normalized to form a standardized structured attribute set. This serves as a benchmark for verifying attribute consistency.

[0037] Then, attribute consistency verification is performed. The candidate image set is... Each image in the dataset is matched one-to-one with atomic attribute value pairs in the structured attribute set, and the images in the candidate image set are estimated using a fine-tuned visual language model. With attribute value Probability of matching ; Calculate the attribute consistency score for each attribute dimension. : in, This indicates the size of the candidate image set, i.e., the number of candidate images returned in each round of retrieval; Represents the i-th image in the candidate image set. ; Indicates the first Each attribute dimension; Indicates the first The attribute values ​​corresponding to each attribute dimension.

[0038] A score threshold is set based on the attribute consistency score. The lower the score, the more severe the matching conflict of that attribute dimension. Attribute dimensions below the threshold are marked as matching conflict points.

[0039] Finally, a dedicated visual language model is used to analyze whether the attribute dimension corresponding to the matching conflict point is an interfering attribute dimension from the candidate image or a key attribute dimension that is not fully described in the text query. The interfering features or key features are further extracted to generate feedback instructions: if the conflict point is that the candidate image set has common interfering features, a negative constraint feedback instruction is generated in the form of "exclude candidates with [interfering features]"; if the conflict point is that the text query omits key pedestrian identity features, an attribute emphasis feedback instruction is generated in the form of "prioritize candidates with [key features]". Based on this, the generated feedback instructions Standardization is performed to ensure that it can be recognized and parsed by the inference module (in this specific embodiment, the inference module uses the dedicated visual language model obtained in step one).

[0040] Step 5: Iteratively optimize the text query and repeat the retrieval and verification steps. First, optimize text queries. This involves revising the previous round of text queries. Feedback instructions The query text query is input into the inference module along with the pedestrian image. The inference module integrates the constraint / emphasis information from the feedback instructions to correct and refine the original text query. It retains correct attribute descriptions, deletes incorrect descriptions, adds missing key attributes and constraints, and generates an optimized new text query. : Then, repeat the retrieval process. Submit the new text query. Input the hybrid retrieval tool from step three, repeat the feature extraction, similarity calculation, and candidate set selection process to obtain a new candidate image set. .

[0041] Set the maximum number of iterations. The algorithm checks whether the current iteration number t has reached Tmax. If it has, the iteration terminates directly; otherwise, it further calculates the current candidate image set. Compared with the previous round of candidate image sets The ratio of the number of intersection images to the number of union images is used. If this ratio exceeds a preset threshold, the candidate image set is considered to be stable, and the iteration terminates. After any termination condition is met, the final candidate image set is sorted and output according to the mixed similarity score, which serves as the final result of person re-identification.

[0042] This invention further validates the effectiveness of the proposed method in three typical scenarios of person re-identification, and conducts experiments on corresponding mainstream datasets: (1) Standard Person Re-identification: Standard pedestrian identity matching was performed across non-overlapping cameras. The Market-1501, MSMT17, and CUHK03 datasets were selected, covering pedestrian images from different scenes and perspectives. (2) Occluded Person Re-identification: For complex scenarios where pedestrians' bodies are partially occluded, the Occluded-Duke dataset is selected. Since global appearance cues for pedestrians are unreliable, the focus is on verifying the model's ability to utilize local features. (3) Re-identification of people changing clothes: For the high-difficulty scenario of pedestrians changing clothes, the PRCC and LTCC datasets were selected to verify the model's ability to extract and distinguish non-clothing identity features.

[0043] The specific statistics for each dataset are as follows: MSMT17 contains 126,441 images and 4,101 pedestrian IDs; Market-1501 contains 32,668 images and 1,501 pedestrian IDs; CUHK03 contains 13,164 images and 1,467 pedestrian IDs; Occluded-Duke contains 35,489 images and 1,404 pedestrian IDs; PRCC contains 33,698 images and 221 pedestrian IDs; and LTCC contains 17,119 images and 152 pedestrian IDs.

[0044] The specific experimental setup is as follows: The base model uses InternVL3.5-8B as the backbone of the visual language model for inference and corrector. The visual encoder and text encoder are initialized based on Siglip2-base-patch16-224. In the tuning parameters, the LoRA adapter rank is set to 16, the training epochs are 3, the learning rate is 1e-5, and only the model's attention layer is tuned. The hyperparameters for hybrid similarity calculation are also included in the inference parameters. =0.65, candidate image set size K=20, maximum number of iterations =3, the image input size is normalized to 256×128 pixels; the hardware environment is implemented based on the PyTorch framework, and the model training and inference are completed on 4 NVIDIA A100 GPUs.

[0045] The training phase is divided into two stages: fine-grained attribute alignment and authentication, and feedback generation. Both stages employ mini-batch gradient descent, freezing the backbone weights of the model and updating only the LoRA adapter parameters to avoid overfitting and improve training efficiency. The inference phase adopts a two-stage evaluation strategy: first, visual pre-filtering is used to retain a candidate list with high recall, and then the Top-K candidate images are refined through closed-loop iteration, making the cost of each query independent of the image library size, thus achieving scalable and efficient inference.

[0046] The experimental evaluation metrics adopted the standard evaluation metrics in the field of person re-identification to evaluate the retrieval performance of the model, as follows: (1) Rank-1 accuracy: reflects the probability that the model retrieves the correct identity at the first position.

[0047] in, This indicates the number of samples where the correct identity appears as the first result in the search results; This represents the total number of samples retrieved.

[0048] (2) Average accuracy (mAP): in, This represents the average precision of a single query sample. This indicates the overall ranking quality of the evaluation model, and it is more sensitive to hard negative samples and long-tail retrieval behavior.

[0049] The experimental results are as follows: The comparison results of this method with existing mainstream person re-identification methods on various datasets are shown in Tables 1 and 2: Table 1. Comparison of State-of-the-art (SOTA) methods on standard (Market1501, MSMT17, CUHK03) and closed (occlded-duke) person re-identification datasets. Table 2. Comparison of State-of-the-art (SOTA) methods on standard (Market1501, MSMT17, CUHK03) and occlded-duke person re-identification datasets. It can be seen that the method of the present invention has achieved better results than existing methods in standard re-identification, occlusion re-identification and clothing change re-identification scenarios, indicating that the method has strong adaptability to the identification of power workers in complex field environments.

[0050] On the standard datasets Market-1501, MSMT17, and CUHK03, the proposed method outperforms the comparative methods in both mAP and Rank-1, indicating that it can effectively extract stable identity features of power workers. On the occluded-Duke scenario, the proposed method achieves higher retrieval accuracy, demonstrating its ability to address common issues in power work sites such as equipment occlusion, pose changes, and insufficient local visibility information. On the clothing-changing scenarios LTCC and PRCC, the proposed method also shows significant advantages, indicating that it does not rely on single clothing appearance cues and can leverage fine-grained attribute information of power workers and a closed-loop self-correction mechanism to improve recognition robustness across time periods and tasks.

[0051] In summary, this invention, through text-driven attribute modeling, hybrid retrieval, and self-correction mechanisms, can better adapt to the actual application conditions in power work sites, such as multiple obstructions, complex perspectives, and frequent changes in clothing, thereby improving the accuracy and stability of power worker re-identification.

[0052] Based on the same inventive concept, this invention proposes a text-driven self-correction system for re-identifying power workers, comprising: The data acquisition and annotation module is used to acquire images of power workers to be retrieved and an image dataset containing images of power workers to be retrieved, and further annotate the images in the image dataset from multiple attribute dimensions; The text query generation module is used to generate a structured description of all attribute dimensions in the image of the power worker to be retrieved using a pre-trained visual language model, and further obtain a structured text query. The hybrid similarity score calculation module is used to extract the visual features and text semantic features of each image in the image dataset, and combine the text query and image features of the image of the power worker to be retrieved to calculate the hybrid similarity score between the image of the power worker to be retrieved and each image in the image dataset. The attribute consistency score calculation module is used to sort the images in the image dataset from high to low according to the mixed similarity score, select the top K images in the image dataset as candidate images, and further combine the text query to calculate the attribute consistency score of each attribute dimension of the candidate image set, and generate feedback instructions based on the attribute consistency score calculation results. The iterative loop module is used to obtain new text queries based on text queries, feedback instructions, and images of power workers to be retrieved, until the loop stops when the loop stopping condition is met. The candidate image set obtained at this time is the similar image of the image to be retrieved in the image dataset.

[0053] For the system embodiments, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments; the implementation methods of the modules will not be repeated here. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0054] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0055] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A text-driven self-correction method for re-identifying power workers, characterized in that, include: S1, obtain the images of the power workers to be retrieved and the image dataset containing the images of the power workers to be retrieved, and further annotate the images in the image dataset from multiple attribute dimensions; S2, using a pre-trained visual language model to generate a structured description of all attribute dimensions in the image of the power worker to be retrieved, and further obtain a structured text query; S3, extract the visual features and text semantic features of each image in the image dataset respectively, and combine the text query and image features of the power worker image to be retrieved to calculate the mixed similarity score between the power worker image to be retrieved and each image in the image dataset; S4. Based on the mixed similarity scores, sort the images from high to low and select the top K images in the image dataset as candidate images. Further combine the text query to calculate the attribute consistency score for each attribute dimension of the candidate image set. The pre-trained visual language model generates feedback instructions based on the attribute consistency score calculation results. S5, based on the text query, feedback instructions and the image of the power worker to be retrieved, infers a new text query and loops through S3-S5 until the loop stops when the loop stopping condition is met. The candidate image set obtained at this time is the similar image of the image to be retrieved in the image dataset.

2. The text-driven self-correction method for re-identification of power workers based on claim 1, characterized in that, In S1, the attribute dimensions include one or more of the following: gender, age group, hairstyle, type of top, color of top, type of bottom, color of bottom, footwear, personal items, and accessories.

3. The text-driven self-correction method for re-identification of power workers according to claim 1, characterized in that, The pre-training process of the visual language model, S2, is as follows: S201, Obtain a training image dataset containing multiple images of people, and label the images in the image dataset from multiple attribute dimensions; S202, freeze all parameters of the visual language model and introduce an adapter into the attention layer of the visual language model to obtain a visual language model with the adapter; based on the training image dataset, construct pairwise verification training samples, attribute-specific question answering training samples, and correction feedback training samples respectively. S203, extract image features from the training image dataset, input the image features from the training image dataset into the visual language model of the adapter to obtain structured attribute descriptions, further calculate the cross-entropy loss between the structured attribute descriptions of the training image dataset and the corresponding annotations; update the adapter parameters according to the cross-entropy loss until convergence, and obtain the initially trained visual language model; S204, input the pairwise validation training samples, attribute-specific question-answering training samples and correction feedback training samples into the initially trained visual language model in sequence, and calculate the cross-entropy loss corresponding to the pairwise validation training samples, attribute-specific question-answering training samples and correction feedback training samples, and then add them together to obtain the total loss; S205, based on the total loss, update the adapter parameters on the basis of the initially trained visual language model until convergence, then stop the loop and obtain the pre-trained visual language model.

4. The text-driven self-correction method for re-identification of power workers based on claim 3, characterized in that, In step S201, the pairwise verification training samples contain multiple image pairs composed of two images each. The visual language model outputs a judgment result based on a preset judgment question and calculates the cross-entropy loss between the result and the actual result of the image pair. The training samples for attribute-specific question answering contain multiple images. The visual language model generates a description of the corresponding attribute dimension in the image based on the query question according to a preset single attribute dimension. The cross-entropy loss is calculated based on the description and the actual attributes of the image. The correction feedback training samples include image pairs consisting of two similar images, erroneous or incomplete structured attribute descriptions corresponding to the similar images, and feedback instructions for the structured attribute descriptions. The visual language model generates feedback instructions based on the image pairs and the corresponding erroneous or incomplete structured attribute descriptions, and further calculates the cross-entropy loss between the generated feedback instructions and the preset feedback instructions.

5. The text-driven self-correction method for re-identification of power workers based on claim 1, characterized in that, In S3, the mixed similarity score is: ； in, This represents the weights used to control the visual and textual modalities. This indicates the image of the power workers to be retrieved. Visual features; Represents each image in the image database Visual features; Indicates the image to be retrieved. Textual semantic features obtained in round iteration; Represents each image in the image database Textual semantic features; Represents the L2 norm. Let represent the mixed similarity score calculated in the t-th iteration.

6. The text-driven self-correction method for re-identification of power workers according to claim 1, characterized in that, In step S4, the calculation process for the attribute consistency score of each attribute dimension of the candidate image set is as follows: S401, perform semantic parsing on the text query to obtain atomic attribute value pairs corresponding to different attribute dimensions, and perform deduplication and normalization processing to obtain a structured attribute set composed of all processed atomic attribute value pairs; S402, using the attribute dimension as the unit, calculates the attribute consistency score of the current attribute dimension using the corresponding attribute dimension of all images in the candidate image set.

7. The text-driven self-correction method for re-identification of power workers based on claim 6, characterized in that, The calculation formula for S402 is as follows: ； in, This represents the total number of candidate images in the candidate image set; Represents the i-th image in the candidate image set. ; Indicates the first Each attribute dimension; Indicates the first The attribute values ​​corresponding to each attribute dimension in the structured attribute set; Indicates the attribute consistency score; Represents images in the candidate image set With attribute values ​​in a structured attribute set The probability of a match.

8. The text-driven self-correction method for re-identification of power workers according to claim 1, characterized in that, In step S4, a feedback instruction is generated based on the attribute consistency score calculation result, specifically as follows: First, determine whether the attribute consistency score of each attribute dimension is higher than the attribute consistency threshold. If it is higher than or equal to the attribute consistency threshold, no action is taken. If it is lower than the threshold, the attribute dimension is marked as a matching conflict point. Secondly, using a dedicated visual language model, it is determined whether the attribute dimension corresponding to the matching conflict point is an interfering attribute dimension from the candidate image or an attribute dimension that is not fully described in the text query. If it is an interfering attribute dimension from the candidate image, the dedicated visual language model generates a negative constraint feedback instruction. If it is an attribute dimension that is not fully described in the text query, the dedicated visual language model generates an attribute emphasis feedback instruction.

9. The text-driven self-correction method for re-identification of power workers according to claim 1, characterized in that, In S5, the loop stopping condition is that the current loop reaches the maximum number of loops, or the ratio of the number of intersection images to the number of union images between the candidate image set generated in the previous loop and the candidate image set generated in the current loop is greater than a threshold.

10. A text-driven self-correction system for re-identifying power workers, used to implement the text-driven self-correction method for re-identifying power workers as described in claim 1, characterized in that, include: The data acquisition and annotation module is used to acquire images of power workers to be retrieved and an image dataset containing images of power workers to be retrieved, and further annotate the images in the image dataset from multiple attribute dimensions; The text query generation module is used to generate a structured description of all attribute dimensions in the image of the power worker to be retrieved using a pre-trained visual language model, and further obtain a structured text query. The hybrid similarity score calculation module is used to extract the visual features and text semantic features of each image in the image dataset, and combine the text query and image features of the image of the power worker to be retrieved to calculate the hybrid similarity score between the image of the power worker to be retrieved and each image in the image dataset. The attribute consistency score calculation module is used to sort the images in the image dataset from high to low according to the mixed similarity score, select the top K images in the image dataset as candidate images, and further combine the text query to calculate the attribute consistency score of each attribute dimension of the candidate image set, and generate feedback instructions based on the attribute consistency score calculation results. The iterative loop module is used to obtain new text queries based on text queries, feedback instructions, and images of power workers to be retrieved, until the loop stops when the loop stopping condition is met. The candidate image set obtained at this time is the similar image of the image to be retrieved in the image dataset.

Images