Model training method and device and storage medium

Abstract

The invention provides a model training method and device and a storage medium, and the method comprises the steps: carrying out the enhancement processing of a text image block, and obtaining a first enhanced image block and a second enhanced image block corresponding to the image block; inputting the first enhanced image block into a first network to obtain a first feature output by the first network, and inputting the second enhanced image block into a second network to obtain a second feature output by the second network; and constructing a loss function used for representing the difference degree between the first feature and the second feature, carrying out back propagation training on the first network and the second network by using the loss function, and obtaining the first encoder used for text image character recognition after training is completed. According to the invention, the model based on comparative learning can be trained without negative samples, and the performance of the model obtained by training is improved. In addition, negative samples are not needed in model training, and the method is friendly to the number of samples.

Description

Technical Field

[0001] The present invention relates to the technical field of neural networks and contrastive learning, and in particular to a model training method, device and storage medium. Background Art

[0002] In neural networks, model learning methods are primarily categorized into supervised learning and unsupervised learning. Supervised learning involves training a model to establish a mapping between input and output given known input data and output annotations. Unsupervised learning involves training a model to capture the structure and relationships between input data given known input data. Supervised learning is commonly used in regression and classification tasks, while unsupervised learning is often used for clustering, dimensionality reduction, anomaly detection, and other tasks. Because collecting labeled data is time-consuming and labor-intensive, supervised learning struggles to achieve good generalization in real-life scenarios where labeled data is scarce. Contrastive learning, a form of unsupervised learning, does not rely on labeled data. Its high degree of freedom in defining positive and negative samples and its excellent performance have made it a key research area within unsupervised learning.

[0003] Without negative samples, contrastive learning has a single goal: to make the features of all positive samples as similar as possible. This creates a learning shortcut, where the model outputs the same features regardless of the input, leading to model collapse. Therefore, existing techniques typically require both positive and negative samples during contrastive learning to prevent the model from learning such shortcuts. Furthermore, contrastive learning algorithms that use negative samples require appropriate negative samples, and model performance is affected by the quantity and quality of negative samples. Summary of the Invention

[0004] At least one embodiment of the present application provides a model training method, device, and storage medium, which can train a model based on contrastive learning in the absence of negative samples, thereby improving the performance of the trained model.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a model training method, comprising:

[0007] Performing enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block;

[0008] Inputting the first enhanced image block into a first network to obtain a first feature output by the first network, and inputting the second enhanced image block into a second network to obtain a second feature output by the second network; wherein the first network includes a first encoder, a first projection layer, and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block, and the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature;

[0009] A loss function is constructed to represent the degree of difference between the first feature and the second feature, and the first network and the second network are trained by backpropagation using the loss function. After the training is completed, the first encoder for text image character recognition is obtained.

[0010] Optionally, before performing enhancement processing on the text image block, the method further includes:

[0011] The text image is preprocessed by data cleaning and / or normalization to obtain a preprocessed text image; wherein the text image block is an image block selected from the preprocessed text image.

[0012] Optionally, the enhancement processing includes at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the enhancement processing used for the first enhanced image block and the second enhanced image block is different.

[0013] Optionally, the first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder;

[0014] The first projection layer and the second projection layer are multi-layer perceptrons (MLPs);

[0015] The predictor is a one-layer or two-layer fully connected layer.

[0016] Optionally, constructing a loss function for representing the degree of difference between the first feature and the second feature includes:

[0017] Performing L2 normalization on the first feature to obtain a sixth feature;

[0018] Performing L2 normalization on the second feature to obtain a seventh feature;

[0019] The Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

[0020] In a second aspect, an embodiment of the present application provides a model training device, comprising:

[0021] An enhancement module, configured to perform enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block;

[0022] A training module is used to input the first enhanced image block into a first network to obtain a first feature output by the first network, and to input the second enhanced image block into a second network to obtain a second feature output by the second network; wherein the first network includes a first encoder, a first projection layer and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block; the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature; construct a loss function for representing the degree of difference between the first feature and the second feature, use the loss function to backpropagate to train the first and second networks, and after the training is completed, obtain the first encoder for text image character recognition.

[0023] Optionally, also include:

[0024] The preprocessing module is used to perform data cleaning and / or normalization preprocessing on the text image before performing enhancement processing on the text image block to obtain a preprocessed text image; wherein the text image block is an image block selected from the preprocessed text image.

[0025] Optionally, the enhancement processing includes at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the enhancement processing used for the first enhanced image block and the second enhanced image block is different.

[0026] Optionally, the first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder;

[0027] The first projection layer and the second projection layer are multi-layer perceptrons (MLPs);

[0028] The predictor is a one-layer or two-layer fully connected layer.

[0029] Optionally, the training module constructs a loss function for representing the degree of difference between the first feature and the second feature, specifically including:

[0030] Performing L2 normalization on the first feature to obtain a sixth feature;

[0031] Performing L2 normalization on the second feature to obtain a seventh feature;

[0032] The Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

[0033] According to the third aspect of the present application, at least one embodiment provides a model training device, comprising a processor, a memory, and a program or instruction stored on the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in any one of the first aspects.

[0034] According to the fourth aspect of the present application, at least one embodiment provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the steps of any method of the first aspect are implemented.

[0035] According to a fifth aspect of the present application, at least one embodiment provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in any one of the first aspects.

[0036] Compared to the prior art, the model training method and apparatus provided in the embodiments of the present application utilize a predictor in a first network to predict the latent space in a second network, accurately clustering data-augmented images together. This allows the encoder to learn a better, more advanced semantic representation, thereby obtaining a more accurate feature representation of the image in text recognition and, in turn, better predicting the text content in the image. Furthermore, the model training method in the embodiments of the present application does not require negative samples, making it sample-friendly. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0038] Figure 1 A flow chart of the model training method according to an embodiment of the present application;

[0039] Figure 2 This is a diagram illustrating an example structure of the first network and the second network according to an embodiment of the present application;

[0040] Figure 3 This is a schematic diagram of the structure of the model training device according to an embodiment of the present application;

[0041] Figure 4 This is another structural diagram of the model training device according to an embodiment of the present application;

[0042] Figure 5 This is another structural schematic diagram of the model training device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.

[0044] It should be understood that references throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment" or "in an embodiment" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. The terms "first," "second," and so on, used in the specification and claims of this application are used to distinguish similar items and are not necessarily used to describe a particular order or sequential sequence. It should be understood that such usage is interchangeable where appropriate, such that the embodiments of the present application described herein can, for example, be implemented in an order other than that illustrated or described herein. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus. The term "and / or" used in the specification and claims refers to at least one of the connected items.

[0045] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0046] The following description provides examples and does not limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. The various examples may appropriately omit, substitute, or add various procedures or components. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0047] In feature space, an image is described by the values ​​of each dimension. If we observe the information of a pair of augmented images in feature space, the relationship between them is not obvious. Therefore, a neural network can be used to project them into a higher-dimensional latent space. In this high-dimensional space, we want the vectors of the two images to be as close as possible, and the distance between these two vectors can be used to represent their relationship.

[0048] In existing contrastive learning techniques, positive and negative pairs are typically used. A positive pair is typically a pair of enhanced images obtained by enhancing an image, while a negative pair is an image obtained by enhancing another image. Existing contrastive learning techniques achieve a closer relationship between the two images in a positive pair by narrowing the distance between the positive pairs and widening the distance between the negative pairs. However, this method requires a large number of negative samples to achieve good results.

[0049] In order to reduce or avoid the dependence of model training on negative samples, an embodiment of the present application provides a model training method, which realizes the training of a model based on contrastive learning without using negative samples and can obtain better model performance.

[0050] Please refer to Figure 1 The model training method provided in the embodiment of the present application includes the following steps:

[0051] Step 11: Perform enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block.

[0052] Here, the text image block can be an image block randomly selected from a text image, and the text image is an image containing text content. Data enhancement is based on existing data. If the amount of data is insufficient for the model, new data can be obtained by making slight changes to the existing data set, such as rotation, shift, flipping and other operations. After feeding into the model, it will be considered as different data. Specifically, the enhancement processing described in step 11 may include but is not limited to at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the first enhanced image block and the second enhanced image block use different enhancement processing.

[0053] For example, we can randomly select an image patch from a text image, resize it to 224×224 by randomly flipping it horizontally, then perform color distortion, including a random sequence of brightness, contrast, saturation, hue adjustments and optional grayscale conversion, and finally perform Gaussian blur and lighting processing on the image patch as follows:

[0054] a) Random Cropping: Randomly select a patch of the text image, for example, uniformly sample from 8% to 100% of the text image, and logarithmically sample the aspect ratio within the range. Then resize the patch to the target size of 224×224 using bicubic interpolation.

[0055] b) Optionally, the image block may be flipped left to right.

[0056] c) Color dithering: The brightness, contrast, saturation, and hue of an image block are uniformly and randomly shifted and applied to all pixels of the same image block. For each image block, the order of these processing is randomly selected.

[0057] d) Color downscaling: Optional grayscale conversion, the output intensity of a pixel (r,g,b) corresponds to its brightness component, calculated as 0.2989r+0.5870g+0.1140b.

[0058] e) Gaussian blur: For a 224×224 image, a 23×23 square Gaussian kernel is used for Gaussian blurring with a uniform sampling standard deviation of [0.1, 2.0].

[0059] f) Lighting: an optional color transformation x→xI x＜0.5 +(1-x)I x≥0.5 The value of a pixel is [0,1].

[0060] Therefore, for a pre-acquired text image set D, image blocks x~D are uniformly sampled from D, and two distributions T and T' are obtained through image enhancement. Image enhancement v = t(x)∈T, v' = t'(x)∈T' is applied. In other words, the input of step 10 is the text image block x, and the output is two enhanced image blocks, such as the first enhanced image block v and the second enhanced image block v'.

[0061] Step 12: input the first enhanced image block into a first network to obtain a first feature output by the first network, and input the second enhanced image block into a second network to obtain a second feature output by the second network.

[0062] Figure 2A structural schematic diagram of the first network and the second network is provided, wherein the first network includes a first encoder 211, a first projection layer 212 and a predictor 213, the first encoder 211 generates a third feature based on the first enhanced image block, the first projection layer 212 performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor 213 generates the first feature based on the fourth feature; the second network includes a second encoder 221 and a second projection layer 222, the second encoder 221 generates a fifth feature based on the second enhanced image block; the second projection layer 222 performs dimensionality reduction processing on the fifth feature to obtain a second feature.

[0063] in addition, Figure 2 Also shown in the figure are: a first enhancement module 210 for enhancing the text image block x to obtain a first enhanced image block v; and a second enhancement module 220 for enhancing the text image block x to obtain a second enhanced image v′.

[0064] The encoder can be a stack of several units, each of which receives a single element of the input sequence, collects information about that element, and propagates it forward. The encoder vector is the final hidden state produced from the encoder part of the model, which serves as the initial hidden state for the decoder part of the model.

[0065] Optionally, the first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder. The first projection layer and the second projection layer are both multi-layer perceptrons (MLPs). The predictor is a one-layer or two-layer fully connected layer.

[0066] by Figure 2 For example, for the first enhanced image v, it passes through the first network, and the first encoder is f θ , the third feature obtained by the first encoder is represented as y θ =f θ (v). For the second enhanced image v′, it passes through the second network, and the second encoder is f ξ , the fifth feature obtained by the second encoder is represented as y′ ξ =f ξ (v′).

[0067] Here, f θ and f ξis a CNN network, where one network is updated based on the parameters of the other network. The second network has the same structure as the first network, but with different parameters, where ξ is the exponential moving average of the first parameter θ. The target decay rate τ∈[0,1] is updated after each training step ξ←τξ+(1-τ)θ. As can be seen, the inputs of the two encoders are v and v′, and the outputs are y θ and y′ ξ .

[0068] The projection layer projects the hidden layer features of the model into another vector space (a layer) of the neural network. In the two projection layers, the third feature y θ After the first projection layer g θ , and get the fourth feature z θ =g θ (y). Fifth feature y′ ξ After the second projection layer g ξ , and get the second feature z′ ξ =g ξ (y′). Here, the projection layer can use a neural network, such as a multi-layer perceptron (MLP) to perform the transformation. A possible structure is as follows: a linear layer, a batch normalization module, a ReLU activation function, and a linear layer. Therefore, the inputs of the two projection layers are y θ and y′ ξ , the outputs are z θ and z′ ξ .

[0069] The predictor can be a function, usually the decision function obtained by the final training of the machine learning problem. The predictor only exists in the first network and consists of one to two fully connected layers. The first feature z is obtained through the projection layer. θ After that, the predictor q in the first network θ , use q θ (z θ ) predict z′ ξ .

[0070] Step 13: construct a loss function for representing the degree of difference between the first feature and the second feature, use the loss function to backpropagate and train the first network and the second network, and after the training is completed, obtain the first encoder for text image character recognition.

[0071] Here, the first feature is subjected to L2 normalization processing to obtain the sixth feature; the second feature is subjected to L2 normalization processing to obtain the seventh feature; then, the Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

[0072] When predicting, the error is calculated using L2 normalization, where the first feature z θ Perform L2 normalization to obtain the sixth feature

[0073]

[0074] For the second feature z′ ξ Perform L2 normalization to obtain the seventh feature

[0075]

[0076] The loss function is a calculation function used to measure the difference between the model's predicted value and the true value. Here, the smaller the loss function, the better the model robustness. The loss function L(θ,ξ) can be expressed as:

[0077]

[0078] η is the learning rate, and the parameters in the first network and the second network are updated by minimizing the objective function until the preset training result conditions are met. The parameter update process can be expressed as:

[0079]

[0080] ξ←τξ+(1-τ)θ

[0081] Through the above steps, after completing the training of the first network and the second network, the embodiment of the present application can obtain a trained first encoder, which can be used for text image character recognition. During text recognition, the image is input into the first encoder, which generates and outputs a feature representation corresponding to the image. This feature representation is then converted into text content in natural language, thereby realizing the text recognition function.

[0082] As can be seen from the above, the embodiments of the present application use the predictor in the first network to predict the latent space in the second network, thereby overcoming the impact of image enhancement on the image and accurately clustering the data-enhanced images together. This actually forces the encoder to learn a better and more advanced semantic expression, thereby obtaining a more accurate feature representation for an image in text recognition, thereby better predicting the text content in the image. In addition, this method uses the predictor to predict the second network, so no negative samples are required, and the model training method is friendly to the number of samples.

[0083] In addition, before step 11 above, the embodiment of the present application may also obtain multiple text images, perform data cleaning and / or normalization preprocessing on the text images, and obtain preprocessed text images; wherein the text image blocks in step 11 above are image blocks selected from the preprocessed text images. Here, data cleaning includes processing such as removing noise. Data normalization processing refers to normalizing text images, which is an optional step. For example, it needs to be applied when text images have different sizes, and the image sizes are unified after operations such as cropping and stretching.

[0084] Based on the above method, the present application also provides a device for implementing the above method. Figure 3 , an embodiment of the present application provides a model training device, comprising:

[0085] An enhancement module 31 is configured to perform enhancement processing on a text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block;

[0086] The training module 32 is used to input the first enhanced image block into the first network to obtain the first feature output by the first network, and to input the second enhanced image block into the second network to obtain the second feature output by the second network; wherein, the first network includes a first encoder, a first projection layer and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block; the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature; construct a loss function for representing the degree of difference between the first feature and the second feature, use the loss function to backpropagate to train the first network and the second network, and after the training is completed, obtain the first encoder for text image character recognition.

[0087] Optional, such as Figure 4 As shown, the above-mentioned model training device also includes:

[0088] The preprocessing module 33 is used to perform data cleaning and / or normalization preprocessing on the text image before performing enhancement processing on the text image block to obtain a preprocessed text image; wherein the text image block is an image block selected from the preprocessed text image.

[0089] Optionally, the enhancement processing includes at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the enhancement processing used for the first enhanced image block and the second enhanced image block is different.

[0090] Optionally, the first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder;

[0091] The first projection layer and the second projection layer are multi-layer perceptrons (MLPs);

[0092] The predictor is a one-layer or two-layer fully connected layer.

[0093] Optionally, the training module constructs a loss function for representing the degree of difference between the first feature and the second feature, specifically including:

[0094] Performing L2 normalization on the first feature to obtain a sixth feature;

[0095] Performing L2 normalization on the second feature to obtain a seventh feature;

[0096] The Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

[0097] It should be noted that the various devices / systems provided in the above embodiments are devices corresponding to the above model training methods, and the implementation methods in the above embodiments are applicable to the embodiments of the devices and can achieve the same technical effects. The above-mentioned device provided in the embodiments of the present application can implement all the method steps implemented in the above method embodiments and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiments in this embodiment will not be specifically described here.

[0098] Please refer to Figure 5 , the embodiment of the present application also provides a hardware structure block diagram of the model training device, such as Figure 5 As shown, the model training device 500 includes:

[0099] Processor 502; and

[0100] a memory 504 in which computer program instructions are stored,

[0101] When the computer program instructions are executed by the processor, the processor 502 is caused to perform the following steps:

[0102] Performing enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block;

[0103] Inputting the first enhanced image block into a first network to obtain a first feature output by the first network, and inputting the second enhanced image block into a second network to obtain a second feature output by the second network; wherein the first network includes a first encoder, a first projection layer, and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block, and the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature;

[0104] A loss function is constructed to represent the degree of difference between the first feature and the second feature, and the first network and the second network are trained by backpropagation using the loss function. After the training is completed, the first encoder for text image character recognition is obtained.

[0105] It should be noted that the various systems provided in the above embodiments are devices corresponding to the above model training methods, and the implementation methods in the above embodiments are all applicable to the embodiments of the devices and can achieve the same technical effects. The above-mentioned device provided in the embodiments of the present application can implement all the method steps implemented in the above-mentioned method embodiments and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiments in this embodiment will not be specifically described here.

[0106] Furthermore, if Figure 5 As shown, the model training device 500 also includes a network interface 501, an input device 503, a hard disk 505, and a display device 506.

[0107] The above-mentioned various interfaces and devices can be interconnected through a bus architecture. The bus architecture can be a bus and bridge that can include any number of interconnected buses. Specifically, various circuits of one or more central processing units (CPUs) and / or graphics processing units (GPUs) represented by processor 502, and one or more memories represented by memory 504 are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits together. It can be understood that the bus architecture is used to achieve connection and communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are all well known in the art and therefore will not be described in detail herein.

[0108] The network interface 501 can be connected to a network (such as the Internet, a local area network, etc.), receive data such as original training samples from the network, and save the received data in the hard disk 505.

[0109] The input device 503 can receive various instructions input by the operator and send them to the processor 502 for execution. The input device 503 can include a keyboard or a pointing device (e.g., a mouse, a trackball, a touchpad, or a touch screen).

[0110] The display device 506 can display the results obtained by the processor 502 executing instructions, such as displaying the model training progress.

[0111] The memory 504 is used to store programs and data necessary for the operation of the operating system, as well as data such as intermediate results during the calculation process of the processor 502.

[0112] It is understood that the memory 504 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. The memory 504 of the apparatus and method described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0113] In some embodiments, the memory 504 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 5041 and application programs 5042 .

[0114] The operating system 5041 includes various system programs, such as a framework layer, a core library layer, and a driver layer, for implementing various basic services and processing hardware-based tasks. The application program 5042 includes various application programs, such as a browser, for implementing various application services. The program implementing the method of the embodiment of the present application can be included in the application program 5042.

[0115] The methods disclosed in the above embodiments of this application can be applied to or implemented by processor 502. Processor 502 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 502 or by software instructions. The above processor 502 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 504, and processor 502 reads the information in memory 504 and, in conjunction with its hardware, completes the steps of the above method.

[0116] It is understood that the embodiments described herein can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof.

[0117] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0118] Specifically, when the computer program is executed by the processor 502, the following steps may be implemented:

[0119] Before performing enhancement processing on the text image block, data cleaning and / or normalization preprocessing is performed on the text image to obtain a preprocessed text image; wherein the text image block is an image block selected from the preprocessed text image.

[0120] Optionally, the enhancement processing includes at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the enhancement processing used for the first enhanced image block and the second enhanced image block is different.

[0121] Optionally, the first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder;

[0122] The first projection layer and the second projection layer are multi-layer perceptrons (MLPs);

[0123] The predictor is a one-layer or two-layer fully connected layer.

[0124] Specifically, when the computer program is executed by the processor 502, the following steps may be implemented:

[0125] Constructing a loss function for representing the degree of difference between the first feature and the second feature, including:

[0126] Performing L2 normalization on the first feature to obtain a sixth feature;

[0127] Performing L2 normalization on the second feature to obtain a seventh feature;

[0128] The Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

[0129] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0130] In some embodiments of the present application, a computer-readable storage medium is further provided, on which a program is stored. When the program is executed by a processor, the following steps are implemented:

[0131] Performing enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block;

[0132] Inputting the first enhanced image block into a first network to obtain a first feature output by the first network, and inputting the second enhanced image block into a second network to obtain a second feature output by the second network; wherein the first network includes a first encoder, a first projection layer, and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block, and the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature;

[0133] A loss function is constructed to represent the degree of difference between the first feature and the second feature, and the first network and the second network are trained by backpropagation using the loss function. After the training is completed, the first encoder for text image character recognition is obtained.

[0134] When this program is executed by the processor, it can implement all the implementation methods of the above-mentioned model training method and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0135] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned model training method embodiment are implemented and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0136] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0137] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0138] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0142] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A model training method, characterized in that: include: Performing enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block; Inputting the first enhanced image block into a first network to obtain a first feature output by the first network, and inputting the second enhanced image block into a second network to obtain a second feature output by the second network; wherein the first network includes a first encoder, a first projection layer, and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block, and the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature; A loss function is constructed to represent the degree of difference between the first feature and the second feature, and the first network and the second network are trained by backpropagation using the loss function. After the training is completed, the first encoder for text image character recognition is obtained.

2. The method according to claim 1, wherein Before performing enhancement processing on the text image block, the method further includes: The text image is preprocessed by data cleaning and / or normalization to obtain a preprocessed text image; wherein the text image block is an image block selected from the preprocessed text image.

3. The method according to claim 1, wherein The enhancement processing includes at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the enhancement processing used for the first enhanced image block and the second enhanced image block is different.

4. The method according to claim 1, wherein The first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder; The first projection layer and the second projection layer are multi-layer perceptrons (MLPs); The predictor is a one-layer or two-layer fully connected layer.

5. The method according to claim 1, wherein Constructing a loss function for representing the degree of difference between the first feature and the second feature, including: Performing L2 normalization on the first feature to obtain a sixth feature; Performing L2 normalization on the second feature to obtain a seventh feature; The Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

6. A model training device, characterized in that: include: An enhancement module, configured to perform enhancement processing on the text image block to obtain a first enhanced image block and a second enhanced image block corresponding to the image block; A training module, configured to input the first enhanced image block into a first network to obtain a first feature output by the first network, and input the second enhanced image block into a second network to obtain a second feature output by the second network; wherein the first network includes a first encoder, a first projection layer, and a predictor, the first encoder generates a third feature based on the first enhanced image block, the first projection layer performs dimensionality reduction processing on the second feature to obtain a fourth feature, and the predictor generates the first feature based on the fourth feature; the second network includes a second encoder and a second projection layer, the second encoder generates a fifth feature based on the second enhanced image block, and the second projection layer performs dimensionality reduction processing on the fifth feature to obtain a second feature; A loss function is constructed to represent the degree of difference between the first feature and the second feature, and the first network and the second network are trained by backpropagation using the loss function. After the training is completed, the first encoder for text image character recognition is obtained.

7. The device according to claim 6, characterized in that Also includes: The preprocessing module is used to perform data cleaning and / or normalization preprocessing on the text image before performing enhancement processing on the text image block to obtain a preprocessed text image; wherein the text image block is an image block selected from the preprocessed text image.

8. The device according to claim 6, wherein The enhancement processing includes at least one of the following processing: adjusting the text image block to a preset size by random horizontal flipping, color distortion processing, Gaussian blur processing, and lighting processing; wherein the enhancement processing used for the first enhanced image block and the second enhanced image block is different.

9. The device according to claim 6, wherein The first encoder and the second encoder are convolutional neural networks (CNNs) with the same structure but different parameters, and the parameters of the second encoder are exponential moving averages of the parameters of the first encoder; The first projection layer and the second projection layer are multi-layer perceptrons (MLPs); The predictor is a one-layer or two-layer fully connected layer.

10. The device according to claim 6, wherein The training module constructs a loss function for representing the degree of difference between the first feature and the second feature, specifically including: Performing L2 normalization on the first feature to obtain a sixth feature; Performing L2 normalization on the second feature to obtain a seventh feature; The Euclidean distance between the sixth feature and the seventh feature is used as the loss function.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the model training method according to any one of claims 1 to 5 are implemented.

12. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 5.

Images