Image coding model training method and device, computer equipment and storage medium

Abstract

The invention relates to an image coding model training method and device, computer equipment and a storage medium. The method comprises the following steps: collecting sample images of a plurality of handwritten signatures; extracting a main body area image of the handwritten signature from the sample image, and performing data enhancement processing on the main body area image to obtain an enhanced image of the handwritten signature; inputting the enhanced image as a guide condition of an image generation process into a diffusion model, and enabling the diffusion model to output a generated image corresponding to the handwritten signature as a training data set; and carrying out model training on a preset base model by using the training data set by adopting a comparative learning method to obtain an image coding model. By adopting the method, the model can have relatively high anti-interference capability on external factors such as color font jitter and background texture change of the signature image, and the attention on the signature font structure characteristics is enhanced, so that the robustness of image coding is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image coding model training method, apparatus, computer equipment and storage medium. Background Art

[0002] Handwritten signature retrieval technology has a wide range of applications in finance, insurance, identity authentication, digital signatures and text recognition. The signature feature extraction method is the core of this technology. The commonly used handwritten signature feature extraction methods mainly include template matching and deep learning models. The template matching method compares the signature to be identified with the pre-stored signature template and calculates the similarity to achieve recognition; the deep learning model uses convolutional neural networks or other types of neural networks to automatically extract high-level features from the signature image and learn the intrinsic pattern of the signature through training.

[0003] Although the existing handwritten signature recognition technology has achieved certain results, there are still some problems. Due to the influence of factors such as lighting, paper curvature, and shooting angle, the curvature, angle, and depth of the signature vary, and the template matching-based method is difficult to achieve high robustness. In addition, the existing deep learning models usually require a large amount of annotated data to achieve good performance, but in practical applications, it is very difficult to obtain a large amount of high-quality annotated data, and the manual annotation process is complicated and error-prone, which limits the training effect of the deep learning model. Summary of the invention

[0004] Based on this, it is necessary to provide an image coding model training method, device, computer equipment and storage medium that can be trained with a small amount of labeled data to address the above technical problems.

[0005] On the one hand, a method for training an image coding model is provided, the method comprising:

[0006] Collect multiple sample images of handwritten signatures;

[0007] Extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature;

[0008] Inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0009] A contrastive learning method is adopted to perform model training on a preset base model using the training data set to obtain an image coding model.

[0010] In one embodiment, extracting the main area image of the handwritten signature from the sample image and performing data enhancement processing on the main area image to obtain an enhanced image of the handwritten signature includes:

[0011] Detecting the position of the handwritten signature from the sample image using a pre-trained handwritten signature detection model, and cropping an image of the main area of ​​the handwritten signature;

[0012] Extracting feature vectors from the main body region image using a feature extraction algorithm, and performing deduplication on the main body region image by calculating similarities between the feature vectors;

[0013] The quality of the deduplicated main area image is screened through a quality estimation model;

[0014] The main body area image obtained after quality screening is subjected to data enhancement processing to obtain an enhanced image of the handwritten signature.

[0015] In one embodiment, the data enhancement processing includes elastic deformation, and the data enhancement processing of the main area image to obtain the enhanced image of the handwritten signature includes:

[0016] Extracting edge features of the handwritten signature from the main area image by an edge detection algorithm to obtain an edge image;

[0017] A random displacement field is created to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural shaking and paper distortion, so as to obtain an enhanced image of the handwritten signature.

[0018] In one embodiment, the creating of the random displacement field to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural shaking and paper distortion to obtain the enhanced image of the handwritten signature includes:

[0019] Performing a random affine transformation on the edge image to obtain a first transformed image;

[0020] Randomly generating a displacement vector for each pixel in the first transformed image, and obtaining a random displacement field according to the displacement vector corresponding to each pixel;

[0021] The random displacement field is smoothed, and the edge image is transformed by the random displacement field obtained after the smoothing process to obtain a second transformed image as an enhanced image of the handwritten signature.

[0022] In one embodiment, the diffusion model includes a ControlNet model, and the step of inputting the enhanced image as a guiding condition of an image generation process into the diffusion model so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set includes:

[0023] Setting a text prompt word, wherein the text prompt word is used to generate a background image of the handwritten signature;

[0024] The enhanced image and the text prompt word are input into the ControlNet model as guiding conditions, so that the ControlNet model guides the diffusion model to fuse the enhanced image with the background image to output a generated image corresponding to the handwritten signature as a training data set.

[0025] In one embodiment, the contrastive learning method is used to perform model training on the preset base model using the training data set, including:

[0026] Constructing positive sample pairs and negative sample pairs according to the training data set, wherein the positive sample pair includes a pair of generated images obtained after the same sample image is subjected to different data enhancement processing, and the negative sample pair includes generated images corresponding to different sample images;

[0027] The loss value is calculated by using a contrastive learning loss function to achieve unsupervised training of the base model.

[0028] In one of the embodiments, after the unsupervised training of the base model is implemented, the method further includes:

[0029] Obtaining a labeled data set by labeling sample images of the handwritten signature, wherein the labeled data set includes positive samples and negative samples, wherein the positive samples include multiple sample images of the same person, and the negative samples include sample images of different persons;

[0030] Constructing a triplet according to the annotated data set, wherein the triplet includes an anchor sample and a positive sample and a negative sample corresponding to the anchor sample;

[0031] The loss value is calculated using a triplet loss function to implement supervised training of the base model and obtain the image coding model.

[0032] On the other hand, an image coding model training device is provided, the device comprising:

[0033] A collection module, used for collecting sample images of multiple handwritten signatures;

[0034] An enhancement module, used for extracting a main body area image of the handwritten signature from the sample image, and performing data enhancement processing on the main body area image to obtain an enhanced image of the handwritten signature;

[0035] A generation module, used for inputting the enhanced image as a guiding condition of an image generation process into a diffusion model, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0036] The training module is used to adopt a contrast learning method and use the training data set to perform model training on a preset base model to obtain an image coding model.

[0037] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0038] Collect multiple sample images of handwritten signatures;

[0039] Extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature;

[0040] Inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0041] A contrastive learning method is adopted to perform model training on a preset base model using the training data set to obtain an image coding model.

[0042] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Collect multiple sample images of handwritten signatures;

[0044] Extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature;

[0045] Inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0046] A contrastive learning method is adopted to perform model training on a preset base model using the training data set to obtain an image coding model.

[0047] The above-mentioned image coding model training method, device, computer equipment and storage medium collect sample images of multiple handwritten signatures, extract the main area images from them and perform data enhancement processing to obtain enhanced images, and then input these enhanced images as guiding conditions into the diffusion model to generate handwritten signature images that simulate real handwriting changes as training data sets, and use the contrastive learning method to use the data set to perform unsupervised training on the preset base model, and finally obtain an image coding model that can effectively extract handwritten signature features. This method not only enables the model to have a high anti-interference ability to external factors such as color and glyph jitter and background texture changes of the signature image during training, but also enhances the model's attention to the signature glyph structure features, thereby improving the robustness of image coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of an image coding model training method in one embodiment;

[0049] Figure 2 A schematic diagram of a flow chart of an image coding model training method in another embodiment;

[0050] Figure 3 It is a schematic diagram of a flow chart of a data enhancement step using elastic deformation in one embodiment;

[0051] Figure 4 is a structural block diagram of an image coding model training device in one embodiment;

[0052] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In one embodiment, Figure 1-3 As shown, a method for training an image coding model is provided, comprising the following steps:

[0055] Step S1, collecting sample images of multiple handwritten signatures.

[0056] The collected handwritten signatures can come from the fields of finance, insurance, identity authentication, digital signatures, text recognition, etc. This application does not limit the data source. For example, in the field of insurance claims, the handwritten signatures of doctors and patients can be collected from discharge certificates and outpatient medical records. Such data often have pain points such as data clutter and lack of labels, and can be used as a data source for model training.

[0057] Step S2, extracting the main area image of the handwritten signature from the sample image, and performing data enhancement processing on the main area image to obtain an enhanced image of the handwritten signature.

[0058] In one embodiment, extracting the main body area image of the handwritten signature from the sample image and performing data enhancement processing on the main body area image to obtain an enhanced image of the handwritten signature includes:

[0059] Step S21, detecting the position of the handwritten signature from the sample image using a pre-trained handwritten signature detection model, and cropping the main area image of the handwritten signature.

[0060] The handwritten signature detection model can use YOLO, DBNet or other detection frameworks, which are not limited here. The sample image is predicted by the loaded detection model, and the model will output the bounding box of each potential handwritten signature main area and the corresponding confidence score. The NMS algorithm can be applied to remove overlapping bounding boxes, or a reasonable confidence threshold can be set to retain only high-confidence bounding boxes, and then the main image area of ​​the handwritten signature can be cropped out according to each retained bounding box and saved.

[0061] Step S22: extracting feature vectors from the main body region image using a feature extraction algorithm, and performing deduplication on the main body region image by calculating the similarity between the feature vectors.

[0062] In order to avoid duplicate signature images in the training data set, it is necessary to deduplicate the extracted handwritten signature main area images. Feature extraction algorithms (such as SIFT, HOG, etc.) can be used to extract the key features of each image. These features can be images, textures, edges, corners, etc., and feature matching technology can be used to identify and remove duplicate images. For each pair of images, the similarity between the feature vectors (such as Euclidean distance, cosine similarity) is calculated. If the feature similarity between the two images exceeds the preset threshold, the two images are considered to be duplicates and one of them is deleted. This prevents the same image or very similar images from appearing multiple times in the data set, avoids overfitting the model to a specific type of image, and improves the generalization ability of the model, ensuring that the model can learn from a wider range of data samples. At the same time, fewer duplicate samples can reduce unnecessary calculations, thereby improving training efficiency.

[0063] Step S23: Perform quality screening on the deduplicated main area image using a quality estimation model.

[0064] In order to ensure the quality of the training data set, it is also necessary to screen the deduplicated images and remove those that do not meet the quality standards. For example, image quality assessment tools (such as blur detection) can be used to detect image clarity, contrast and other indicators. Images that do not meet the quality standards, such as images that are too blurry or improperly exposed, should be removed from the data set.

[0065] Step S24, performing data enhancement processing on the main body area image obtained after quality screening to obtain an enhanced image of the handwritten signature.

[0066] Among them, data enhancement should ensure that the signatures before and after enhancement are handwritten by the same person, and increase the diversity of data as much as possible. Traditional data enhancement methods, such as random image cropping, color jitter enhancement, random rotation and other operations, can only perform overall enhancement at the image level, and it is difficult to simulate local changes such as font jitter and background texture changes. Therefore, this embodiment adopts elastic deformation to deform the image by creating a random displacement field, so that multiple local areas of the signature produce natural distortions, thereby simulating the handwriting changes caused by natural jitter, paper bending and other factors, and further improving the effect of subsequent unsupervised training.

[0067] In one embodiment, the data enhancement processing includes elastic deformation, and the data enhancement processing is performed on the main area image to obtain an enhanced image of the handwritten signature, including:

[0068] Step S241: extracting the edge image of the handwritten signature from the main area image by using an edge detection algorithm.

[0069] Among them, the edge detection algorithm is an image processing technology used to identify points in an image where the brightness changes significantly, that is, edges. Common edge detection algorithms include the Canny algorithm, the Sobel operator, the Prewitt operator, the Laplacian algorithm, etc. In this embodiment, the Canny algorithm is taken as an example. This is a multi-stage edge detection algorithm that uses a Gaussian filter to smooth the grayscale image to remove noise, and then finds the point with the largest intensity gradient as the edge point, and determines the final edge by suppressing non-maximum values ​​and threshold processing, thereby obtaining an edge image of the handwritten signature.

[0070] Step S242, creating a random displacement field to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural shaking and paper distortion, and obtain an enhanced image of the handwritten signature.

[0071] In one embodiment, the creation of a random displacement field performs local deformation processing on the edge image to simulate handwriting changes caused by natural jitter and paper distortion to obtain an enhanced image of the handwritten signature, including: performing a random affine transformation on the edge image to obtain a first transformed image; randomly generating a displacement vector for each pixel in the first transformed image, and obtaining a random displacement field based on the displacement vector corresponding to each pixel; smoothing the random displacement field, and transforming the edge image using the random displacement field obtained after smoothing to obtain a second transformed image as an enhanced image of the handwritten signature.

[0072] Among them, affine transformation includes operations such as rotation, scaling, shearing and translation, which are used to simulate global changes caused by factors such as the paper placement angle and the signer's writing posture when signing. For each pixel point in the first transformed image, a pair of random numbers are randomly generated as the displacements △x and △y in the x and y directions, where the displacements △x and △y are usually random numbers uniformly distributed within a specific range, such as (-1, 1), and these random numbers constitute a two-dimensional array, namely a random displacement field. Then, the generated random displacement field is smoothed and filtered using a Gaussian kernel to make the local deformation more natural, and a control factor α can be used to multiply the random displacement field to adjust the degree of deformation. The smoothed displacement field is applied to the edge image, which means that each pixel point will move to a new position according to the displacement in the displacement field. For points that are not at the original pixel position after movement, a difference method needs to be used to estimate the pixel value of the new position. The pixel value of the new position can be determined using nearest neighbor interpolation, bilinear interpolation or other interpolation methods. After completing the relocation of all pixel points, a second transformed image is obtained as the enhanced image.

[0073] Through the above steps, slight changes in handwriting caused by hand shaking, changes in pen stroke strength, etc. when signing can be simulated, so that the generated data is closer to the actual signature situation.

[0074] Step S3, inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set.

[0075] In one embodiment, the diffusion model includes a ControlNet model, and the enhanced image is input into the diffusion model as a guiding condition of the image generation process, so that the diffusion model outputs the generated image corresponding to the handwritten signature as a training data set, including: setting text prompt words, and the text prompt words are used to generate a background image of the handwritten signature; the enhanced image and the text prompt words are input into the ControlNet model as guiding conditions, so that the ControlNet model guides the diffusion model to fuse the enhanced image with the background image to output the generated image corresponding to the handwritten signature as a training data set.

[0076] Specifically, a series of text prompt words describing the background texture are set to guide the generation of the corresponding background image, thereby improving the diversity of the background texture. By changing the text prompt words and different enhanced images, on the basis of controlling the signatures before and after generation to be the handwriting of the same person, the data distribution of the real scene is simulated as much as possible to improve the diversity of signature data. This makes the model have a high anti-interference ability to external factors such as color and glyph jitter of the signature image, background texture changes, etc., and enhances the model's attention to the structural features of the signature glyphs. Finally, a handwritten signature image training dataset with diverse backgrounds and handwriting changes is formed, thereby improving the robustness of image encoding.

[0077] Step S4, adopting a comparative learning method, using the training data set to perform model training on the preset base model to obtain an image coding model.

[0078] In one embodiment, the contrastive learning method is used to perform model training on a preset base model using the training data set, including: constructing positive sample pairs and negative sample pairs according to the training data set, the positive sample pair including a pair of generated images obtained after the same sample image is subjected to different data enhancement processing, and the negative sample pair including generated images corresponding to different sample images; using a contrastive learning loss function to calculate the loss value to achieve unsupervised training of the base model.

[0079] Specifically, the contrastive learning loss function adopts InfoNCE loss, and its formula is as follows:

[0080]

[0081] Where N is the number of sample images in a batch, q i and k i is the encoding vector of a positive sample pair corresponding to sample image i, k jis the encoding vector of the negative sample corresponding to sample image j, exp is the similarity function, and τ is the temperature coefficient, which is used to adjust the distribution of the similarity score. Specifically, the larger the temperature coefficient, the lower the model's discrimination of negative samples, so that more negative samples can be included. On the contrary, if the temperature coefficient is too small, the model will pay more attention to those particularly difficult negative samples, thereby ignoring other possible positive samples. In this embodiment, τ is 0.07.

[0082] For sample image i, two or more versions of generated images are generated through two different data augmentation processes, forming one or more pairs of positive samples, while the sample image j is also generated after data augmentation as a negative sample. The similarity between positive samples is maximized by calculating the distance between the feature vectors of the two images in the positive sample pair and encouraging this distance to be as small as possible in the loss function, so that the model can recognize that the images obtained after different data augmentation processes are still the same handwritten signature. At the same time, in order to ensure that sample images from different handwritten signatures can be distinguished by the model, the similarity between positive and negative samples is minimized by penalizing this undesirable similarity in the loss function.

[0083] Based on the above training process, the loss function is used for unsupervised training, which not only defines the goal of model optimization, but also provides quantitative indicators and gradient information to help the model learn how to distinguish positive samples from negative samples. In this way, the base model can learn high-quality feature representations in the unsupervised training stage, thus laying a solid foundation for the subsequent supervised fine-tuning stage.

[0084] In one embodiment, after the unsupervised training of the base model is implemented, it also includes: obtaining a labeled data set by labeling sample images of the handwritten signature, the labeled data set includes positive samples and negative samples, the positive samples include multiple sample images of the same person, and the negative samples include sample images of different people; constructing a triplet according to the labeled data set, the triplet includes an anchor sample and positive samples and negative samples corresponding to the anchor sample; using a triplet loss function to calculate the loss value, implement supervised training of the base model, and obtain the image coding model.

[0085] After the unsupervised training stage, the model has learned some basic feature representations. Next, these learned features can be used for supervised fine-tuning to make the model further fit the distribution of the real data set. For example, in this embodiment, triple loss is used for model fine-tuning. It should be noted that in order to prevent the model from overfitting, especially when there is less labeled data, you can choose to freeze the shallow parameters of the model and only allow the deep layers or newly added layers to be updated. This allows the model to retain the basic feature extraction capabilities learned from large-scale unlabeled data, while adapting to specific task requirements by adjusting the upper layers.

[0086] Among them, the formula of the triplet loss function is as follows:

[0087] Triplet Loss = max{d(a i ,p i )-d(a i ,n i )+margin,0};

[0088] Among them, a is the anchor sample, p is the positive sample corresponding to the anchor sample, n is the negative sample corresponding to the anchor sample, d is the distance function used to measure the distance between the anchor sample and the positive sample and the negative sample, and margin is a hyperparameter used to expand the distance between the positive sample and the negative sample to improve the robustness of the model.

[0089] Specifically, a batch of sample images are collected from practical applications, and the handwritten signatures of the same person are used as positive samples, and the handwritten signatures of different people are used as negative samples to form a standard data set. For each anchor sample, positive samples of the same person and negative samples of different people are found to obtain triplets. The loss value of each group of triplets is calculated through the triplet loss function, and back propagation is performed only on the unfrozen weights to calculate the gradient and update the corresponding weights. A suitable optimization algorithm is selected to adjust the model parameters until the model converges or reaches a satisfactory performance level.

[0090] The triplet loss function enables the model to better distinguish samples of different categories by optimizing the distance relationship in the embedding space. It not only improves the classification performance of the model, but also enhances the robustness and generalization ability of the model. After training, the model performance is evaluated on an independent test set to ensure that it has strong generalization ability and can be well applied to handwritten signature recognition in actual scenarios.

[0091] It should be understood that although Figure 1-3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0092] In one embodiment, Figure 4 As shown, a graphics coding model training device is provided, comprising: a collection module, an enhancement module, a generation module and a training module, wherein:

[0093] A collection module, used for collecting sample images of multiple handwritten signatures;

[0094] An enhancement module, used for extracting a main body area image of the handwritten signature from the sample image, and performing data enhancement processing on the main body area image to obtain an enhanced image of the handwritten signature;

[0095] A generation module, used for inputting the enhanced image as a guiding condition of an image generation process into a diffusion model, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0096] The training module is used to adopt a contrast learning method and use the training data set to perform model training on a preset base model to obtain an image coding model.

[0097] In one embodiment, the enhancement module is also used to detect the position of the handwritten signature from the sample image through a pre-trained handwritten signature detection model, and crop out the main area image of the handwritten signature; use a feature extraction algorithm to extract feature vectors in the main area image, and perform deduplication on the main area image by calculating the similarity between the feature vectors; perform quality screening on the deduplicated main area image through a quality estimation model; and perform data enhancement processing on the main area image obtained after quality screening to obtain an enhanced image of the handwritten signature.

[0098] In one embodiment, the enhancement module is also used to extract edge features of the handwritten signature from the main area image through an edge detection algorithm to obtain an edge image; create a random displacement field to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural jitter and paper distortion to obtain an enhanced image of the handwritten signature.

[0099] In one embodiment, the enhancement module is also used to perform a random affine transformation on the edge image to obtain a first transformed image; randomly generate a displacement vector for each pixel in the first transformed image, and obtain a random displacement field based on the displacement vector corresponding to each pixel; smooth the random displacement field, and transform the edge image using the random displacement field obtained after smoothing to obtain a second transformed image as an enhanced image of the handwritten signature.

[0100] In one embodiment, the generation module is also used to set text prompt words, and the text prompt words are used to generate a background image of the handwritten signature; the enhanced image and the text prompt words are input into the ControlNet model as guiding conditions, so that the ControlNet model guides the diffusion model to fuse the enhanced image with the background image to output the generated image corresponding to the handwritten signature as a training data set.

[0101] In one embodiment, the training module is also used to construct positive sample pairs and negative sample pairs based on the training data set, the positive sample pairs include a pair of generated images obtained after the same sample image is subjected to different data enhancement processing, and the negative sample pairs include generated images corresponding to different sample images; the loss value is calculated using a contrastive learning loss function to achieve unsupervised training of the base model.

[0102] In one embodiment, the training module is also used to obtain a labeled data set by annotating sample images of the handwritten signature, the labeled data set including positive samples and negative samples, the positive samples including multiple sample images of the same person, and the negative samples including sample images of different people; construct a triplet according to the labeled data set, the triplet including an anchor sample and positive samples and negative samples corresponding to the anchor sample; use a triplet loss function to calculate the loss value to implement supervised training of the base model to obtain the image coding model.

[0103] For the specific limitations of the graphic coding model training device, please refer to the limitations of the graphic coding model training method above, which will not be repeated here. Each module in the above-mentioned graphic coding model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0104] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training data sets and annotated data sets. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a graphics coding model training method is implemented.

[0105] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0106] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0107] Collect multiple sample images of handwritten signatures;

[0108] Extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature;

[0109] Inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0110] A contrastive learning method is adopted to perform model training on a preset base model using the training data set to obtain an image coding model.

[0111] In one embodiment, when the processor executes the computer program, the following steps are also implemented: detecting the position of the handwritten signature from the sample image through a pre-trained handwritten signature detection model, and cropping the main area image of the handwritten signature; extracting feature vectors in the main area image using a feature extraction algorithm, and deduplicating the main area image by calculating the similarity between the feature vectors; performing quality screening on the deduplicated main area image through a quality estimation model; and performing data enhancement processing on the main area image obtained after quality screening to obtain an enhanced image of the handwritten signature.

[0112] In one embodiment, when the processor executes the computer program, the following steps are also implemented: edge features of the handwritten signature are extracted from the main area image through an edge detection algorithm to obtain an edge image; a random displacement field is created to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural jitter and paper distortion to obtain an enhanced image of the handwritten signature.

[0113] In one embodiment, when the processor executes the computer program, the following steps are also implemented: performing a random affine transformation on the edge image to obtain a first transformed image; randomly generating a displacement vector for each pixel in the first transformed image, and obtaining a random displacement field based on the displacement vector corresponding to each pixel; smoothing the random displacement field, and transforming the edge image using the random displacement field obtained after smoothing to obtain a second transformed image as an enhanced image of the handwritten signature.

[0114] In one embodiment, when the processor executes the computer program, the following steps are also implemented: setting text prompt words, wherein the text prompt words are used to generate a background image of the handwritten signature; inputting the enhanced image and the text prompt words into the ControlNet model as guiding conditions, so that the ControlNet model guides the diffusion model to fuse the enhanced image with the background image to output a generated image corresponding to the handwritten signature as a training data set.

[0115] In one embodiment, when the processor executes the computer program, the following steps are also implemented: constructing positive sample pairs and negative sample pairs according to the training data set, the positive sample pairs including a pair of generated images obtained after the same sample image is subjected to different data enhancement processing, and the negative sample pairs including generated images corresponding to different sample images; using contrastive learning loss function to calculate the loss value to achieve unsupervised training of the base model.

[0116] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a labeled data set by labeling sample images of the handwritten signature, the labeled data set including positive samples and negative samples, the positive samples including multiple sample images of the same person, and the negative samples including sample images of different people; constructing a triplet according to the labeled data set, the triplet including an anchor sample and positive samples and negative samples corresponding to the anchor sample; using a triplet loss function to calculate the loss value, implement supervised training of the base model, and obtain the image coding model.

[0117] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0118] Collect multiple sample images of handwritten signatures;

[0119] Extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature;

[0120] Inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set;

[0121] A contrastive learning method is adopted to perform model training on a preset base model using the training data set to obtain an image coding model.

[0122] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: detecting the position of the handwritten signature from the sample image through a pre-trained handwritten signature detection model, and cropping the main area image of the handwritten signature; extracting feature vectors in the main area image using a feature extraction algorithm, and deduplicating the main area image by calculating the similarity between the feature vectors; performing quality screening on the deduplicated main area image through a quality estimation model; and performing data enhancement processing on the main area image obtained after quality screening to obtain an enhanced image of the handwritten signature.

[0123] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: edge features of the handwritten signature are extracted from the main area image by an edge detection algorithm to obtain an edge image; a random displacement field is created to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural jitter and paper distortion to obtain an enhanced image of the handwritten signature.

[0124] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing a random affine transformation on the edge image to obtain a first transformed image; randomly generating a displacement vector for each pixel in the first transformed image, and obtaining a random displacement field based on the displacement vector corresponding to each pixel; smoothing the random displacement field, and transforming the edge image using the random displacement field obtained after smoothing to obtain a second transformed image as an enhanced image of the handwritten signature.

[0125] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: setting text prompt words, wherein the text prompt words are used to generate a background image of the handwritten signature; inputting the enhanced image and the text prompt words into the ControlNet model as guiding conditions, so that the ControlNet model guides the diffusion model to fuse the enhanced image with the background image to output a generated image corresponding to the handwritten signature as a training data set.

[0126] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: constructing positive sample pairs and negative sample pairs based on the training data set, the positive sample pairs include a pair of generated images obtained after the same sample image is subjected to different data enhancement processing, and the negative sample pairs include generated images corresponding to different sample images; using a contrastive learning loss function to calculate the loss value to achieve unsupervised training of the base model.

[0127] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a labeled data set by labeling sample images of the handwritten signature, the labeled data set including positive samples and negative samples, the positive samples including multiple sample images of the same person, and the negative samples including sample images of different people; constructing a triplet according to the labeled data set, the triplet including an anchor sample and positive samples and negative samples corresponding to the anchor sample; using a triplet loss function to calculate the loss value, thereby realizing supervised training of the base model and obtaining the image coding model.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for training an image coding model, characterized in that: include: Collect multiple sample images of handwritten signatures; Extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature; Inputting the enhanced image into a diffusion model as a guiding condition of an image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set; A contrastive learning method is adopted to perform model training on a preset base model using the training data set to obtain an image coding model.

2. The image coding model training method according to claim 1, characterized in that: The extracting the main body region image of the handwritten signature from the sample image and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature includes: Detecting the position of the handwritten signature from the sample image using a pre-trained handwritten signature detection model, and cropping an image of the main area of ​​the handwritten signature; Extracting feature vectors from the main body region image using a feature extraction algorithm, and performing deduplication on the main body region image by calculating similarities between the feature vectors; The quality of the deduplicated main area image is screened through a quality estimation model; The main body area image obtained after quality screening is subjected to data enhancement processing to obtain an enhanced image of the handwritten signature.

3. The image coding model training method according to claim 1 or 2, characterized in that: The data enhancement processing includes elastic deformation, and the data enhancement processing is performed on the main area image to obtain an enhanced image of the handwritten signature, including: Extracting edge features of the handwritten signature from the main area image by an edge detection algorithm to obtain an edge image; A random displacement field is created to perform local deformation processing on the edge image to simulate the handwriting changes caused by natural shaking and paper distortion, so as to obtain an enhanced image of the handwritten signature.

4. The image coding model training method according to claim 3, characterized in that: The creating of the random displacement field performs local deformation processing on the edge image to simulate the handwriting change caused by natural shaking and paper distortion to obtain the enhanced image of the handwritten signature, including: Performing a random affine transformation on the edge image to obtain a first transformed image; Randomly generating a displacement vector for each pixel in the first transformed image, and obtaining a random displacement field according to the displacement vector corresponding to each pixel; The random displacement field is smoothed, and the edge image is transformed by the random displacement field obtained after the smoothing process to obtain a second transformed image as an enhanced image of the handwritten signature.

5. The image coding model training method according to claim 1, characterized in that: The diffusion model includes a ControlNet model, and the enhanced image is input into the diffusion model as a guiding condition of the image generation process, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set, including: Setting a text prompt word, wherein the text prompt word is used to generate a background image of the handwritten signature; The enhanced image and the text prompt word are input into the ControlNet model as guiding conditions, so that the ControlNet model guides the diffusion model to fuse the enhanced image with the background image to output a generated image corresponding to the handwritten signature as a training data set.

6. The image coding model training method according to claim 1, characterized in that: The method of using the contrastive learning method to perform model training on the preset base model using the training data set includes: Constructing positive sample pairs and negative sample pairs according to the training data set, wherein the positive sample pair includes a pair of generated images obtained after the same sample image is subjected to different data enhancement processing, and the negative sample pair includes generated images corresponding to different sample images; The loss value is calculated by using a contrastive learning loss function to achieve unsupervised training of the base model.

7. The image coding model training method according to claim 6, characterized in that: After the unsupervised training of the base model is implemented, the method further includes: Obtaining a labeled data set by labeling sample images of the handwritten signature, wherein the labeled data set includes positive samples and negative samples, wherein the positive samples include multiple sample images of the same person, and the negative samples include sample images of different persons; Constructing a triplet according to the annotated data set, wherein the triplet includes an anchor sample and a positive sample and a negative sample corresponding to the anchor sample; The loss value is calculated using a triplet loss function to implement supervised training of the base model and obtain the image coding model.

8. An image coding model training device, characterized in that: The device comprises: A collection module, used for collecting sample images of multiple handwritten signatures; an enhancement module, used for extracting a main body region image of the handwritten signature from the sample image, and performing data enhancement processing on the main body region image to obtain an enhanced image of the handwritten signature; A generation module, used for inputting the enhanced image as a guiding condition of the image generation process into a diffusion model, so that the diffusion model outputs a generated image corresponding to the handwritten signature as a training data set; The training module is used to adopt a contrast learning method and use the training data set to perform model training on a preset base model to obtain an image coding model.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images