A character recognition method, device, equipment and medium

Abstract

The embodiment of the present application discloses a character recognition method, which is implemented by using a pre-trained character recognition network model when performing character recognition. The character recognition network model is generated by training a first sample image and a plurality of sub-sample images corresponding to the first sample image. The height of each sub-sample image in the plurality of sub-sample images is the same as the height of the first sample image, the width of each sub-sample image is the same, and the width of each sub-sample image is smaller than the width of the first sample image. The present application does not need to manually label the first sample image, and trains the character recognition network model by aligning the local features (the plurality of sub-sample images) and the overall features (the first sample image), thereby reducing the labeling cost and improving the training efficiency. In actual use, the text image is input into the character recognition network model, the features of the text information of the image can be completely extracted, and the output result is obtained according to the features.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a character recognition method and device, equipment and medium. BACKGROUND

[0002] Optical Character Recognition (OCR) refers to the analysis and recognition processing of image files containing text data to obtain the text technology, which is an important aspect in the research and application field of automatic recognition technology.

[0003] Generally, the OCR recognition model is generated by a supervised training method, and in the training process, sample data that has been manually labeled needs to be collected, and then the sample data is used for training. In order to improve the recognition accuracy of the OCR recognition model, a large amount of sample data needs to be collected, which requires a large amount of manpower for manual labeling, increasing the training cost. SUMMARY

[0004] Therefore, the embodiments of the present application provide a character recognition method, device, equipment and medium to train the model using unlabeled sample data and reduce the training cost.

[0005] To achieve the above object, the technical scheme provided by the embodiments of the present application is as follows:

[0006] In a first aspect of the embodiments of the present application, a character recognition method is provided, which comprises:

[0007] Obtaining a text image to be processed, the text image to be processed comprising text information to be recognized;

[0008] Inputting the text image to be processed into a character recognition network model to obtain an output result, the output result comprising the text information to be recognized;

[0009] The character recognition network model is generated by training using training samples, the training samples comprising a first sample image and a plurality of sub-sample images corresponding to the first sample image, the height of each sub-sample image in the plurality of sub-sample images being the same as the height of the first sample image, the width of each sub-sample image in the plurality of sub-sample images being the same, and the width of each sub-sample image being smaller than the width of the first sample image, and the first sample image comprising text information.

[0010] In a second aspect of the embodiments of the present application, a character recognition device is provided, which comprises:

[0011] An acquisition unit is configured to acquire a text image to be processed, the text image to be processed comprising text information to be recognized;

[0012] a processing unit, configured to input the text image to be processed into a character recognition network model, and obtain an output result, the output result comprising the text information to be recognized; wherein the character recognition network model is generated by training using training samples, the training samples comprising a first sample image and a plurality of sub-sample images corresponding to the first sample image, each of the plurality of sub-sample images having a same height as the first sample image, each of the plurality of sub-sample images having a same width, and the width of each of the plurality of sub-sample images being smaller than the width of the first sample image, the first sample image comprising text information.

[0013] In a third aspect of the embodiments of the present application, an electronic device is provided, the device comprising: a processor and a memory;

[0014] the memory, configured to store instructions or a computer program;

[0015] the processor, configured to execute the instructions or the computer program in the memory, so that the electronic device performs the character recognition method in the first aspect.

[0016] In a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, the computer readable storage medium storing instructions, when the instructions are run on a device, causing the device to perform the character recognition method in the first aspect.

[0017] In a fifth aspect of the embodiments of the present application, a computer program product is provided, when the computer program product is run on a computer, causing the computer to perform the character recognition method in the first aspect.

[0018] Therefore, the embodiments of the present application have the following beneficial effects:

[0019] The character recognition network model is generated by training using the first sample image and the plurality of sub-sample images corresponding to the first sample image. Each of the plurality of sub-sample images has the same height as the first sample image, and each of the plurality of sub-sample images has the same width and the width of each of the plurality of sub-sample images is less than the width of the first sample image. That is, when training the character recognition network model, the first sample image does not need to be manually labeled, but the character recognition network model is trained by aligning local features (the plurality of sub-sample images) and overall features (the first sample image), thereby reducing labeling costs and improving training efficiency. In actual use, the text image to be processed is input into the character recognition network model, so that the character recognition network model can completely extract features of the text information of the image, and recognize according to the extracted features to obtain an output result. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0021] Figure 1 A flowchart of a character recognition method provided by an embodiment of the present application;

[0022] Figure 2 A flowchart of training a character recognition network model provided by an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a division operation provided by an embodiment of the present application;

[0024] Figure 4 Another flowchart of training a character recognition network model provided by an embodiment of the present application;

[0025] Figure 5 A structural diagram of a character recognition device provided by an embodiment of the present application;

[0026] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative effort belong to the scope of protection of the present application.

[0028] OCR refers to a technology for analyzing and recognizing an image file containing text information to obtain the text. Generally, an OCR recognition model is generated by a supervised training method. In the training process, sample data that has been manually labeled needs to be collected, and then the sample data is used for training. To improve the recognition accuracy of the OCR recognition model, when a large amount of sample data is collected, a large amount of manpower is needed for manual labeling, increasing the training cost.

[0029] Based on this, the character recognition method provided in the embodiments of the present application is implemented by using a pre-trained character recognition network model. The character recognition network model is generated by training based on a first sample image and a plurality of sub-sample images corresponding to the first sample image. The height of each sub-sample image in the plurality of sub-sample images is the same as the height of the first sample image, and the width of each sub-sample image in the plurality of sub-sample images is the same and smaller than the width of the first sample image. That is, when training the character recognition network model, the first sample image does not need to be manually labeled, but the character recognition network model is trained by aligning the local features (the plurality of sub-sample images) and the overall features (the first sample image), thereby reducing the labeling cost and improving the training efficiency. In actual use, the text image to be processed is input into the character recognition network model, so that the character recognition network model can completely extract the features of the text information in the image, and recognize according to the extracted features to obtain an output result.

[0030] The character recognition method provided in the embodiments of the present application will be described below in conjunction with the drawings. Referring to Figure 1 , Figure 1 The flowchart of the character recognition method provided in the embodiments of the present application.

[0031] The method specifically includes the following steps:

[0032] S101: Obtain a text image to be processed, which includes text information to be recognized.

[0033] The text information to be recognized can be characters to be recognized, including Chinese characters, English words, English letters, numbers, symbols, etc.

[0034] S102: input the text image to be processed into the character recognition network model, and obtain an output result, which includes the text information to be recognized.

[0035] In this embodiment, after obtaining the text image to be processed, the text image to be processed is input into a pre-trained character recognition network model to obtain the text information to be recognized included in the text image to be processed, so as to output the text information to be recognized through processing of the character recognition network model.

[0036] The character recognition network model is trained by using training samples, the training samples include a first sample image and a plurality of sub-sample images corresponding to the first sample image, the height of each sub-sample image in the plurality of sub-sample images is the same as the height of the first sample image, the width of each sub-sample image in the plurality of sub-sample images is the same, and the width of each sub-sample image is less than the width of the first sample image, and the first sample image includes text information.

[0037] In the training of the character recognition network model, the first sample image does not need to be manually labeled, but is trained by aligning the local features (the plurality of sub-sample images) and the overall features (the first sample image), which can reduce the labeling cost and improve the training efficiency.

[0038] After the training of the character recognition network model is completed, the text image to be processed is input into the character recognition network model, so that the character recognition network model can extract features of the text image, and recognize the text information in the text image according to the extracted features, thereby outputting the text information in the text image.

[0039] Therefore, the trained character recognition network model can extract complete features of the text information in the image, recognize the text according to the extracted complete features, obtain an output result, and improve the accuracy of recognition.

[0040] The process of training the character recognition network model of the present application will be described below with reference to the accompanying drawings. Referring to Figure 2 , Figure 2 A flowchart of training the character recognition network model is provided for the embodiments of the present application.

[0041] The method mainly includes the following steps:

[0042] S201: obtain a first sample image and a plurality of sub-sample images corresponding to the first sample image.

[0043] Before training the character recognition network model, first, a training sample for training the initial network model needs to be obtained, the training sample including a first sample image and a plurality of sub-samples corresponding to the first sample image, wherein the first sample image and the plurality of sub-sample images include text information. The manner of obtaining the training sample can be that a first sample image without artificial annotation is obtained, and then the first sample image is divided into a plurality of sub-sample images, and the first sample image and the plurality of sub-sample images corresponding thereto are used to form a training sample, that is, the sub-sample images are part of the first sample image. The height of each sub-sample image is the same as the height of the first sample image, the width of each sub-sample image is less than the width of the first sample image, and the width of each sub-sample image is the same.

[0044] In a possible implementation, the first sample image can be divided into a plurality of sub-sample images in the following manner. Specifically, a division parameter is first determined, and then the first sample image is divided multiple times by using the division parameter, thereby obtaining a plurality of sub-sample images. The ratio of the width of each sub-sample image to the width of the first sample image is equal to the division parameter, and the division parameter is greater than 0 and less than 1. That is, the first sample image is divided into a plurality of sub-sample images with a width equal to the division parameter.

[0045] In the specific implementation of the division operation, for each division operation, a starting position of the division can be first determined in the first sample image, and the first sample image is divided according to the starting position of each division and the division parameter, thereby obtaining a plurality of sub-sample images. Referring to Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a division operation in an embodiment of the present application. In this scenario, the width of the first sample image is width, and the division parameter is K, that is, the width of each sub-sample image is K*width, wherein the value range of K is 75%-95%. For each division operation, a starting position of the division is determined in the first sample image, and then a sub-sample image with a width of K*width is divided. As shown in FIG. 1, the first sample image is divided into sub-sample image 1, sub-sample image 2,..., and sub-sample image n, wherein the height of each of the sub-sample image 1, the sub-sample image 2,..., and the sub-sample image n is the same as that of the first sample image, and the width of each of the sub-sample image 1, the sub-sample image 2,..., and the sub-sample image n is K*width. Figure 3

[0046] It should be noted that the starting position of each division operation can be a fixed width interval or randomly determined, and the present application does not limit this.

[0047] ​S202: input the first sample image and the plurality of sub-sample images into the initial network model respectively, and obtain a first feature vector set and a second feature vector set, wherein the first feature vector set and the second feature vector set have the same dimension, and the feature vectors in the first feature vector set and the second feature vector set are feature vectors of the text information.

[0048] After obtaining the training sample including the first sample image and the plurality of sub-sample images divided from the first sample image, the initial network model can be trained. The first sample image is input into the initial network model to obtain a first feature vector set corresponding to the first sample image. The plurality of sub-sample images divided from the first sample image are input into the initial network model to obtain a second feature vector set, wherein the first feature vector set and the second feature vector set have the same dimension, and the first feature vector set and the second feature vector set are feature vectors of the text information of the sample image.

[0049] It should be noted that the order of inputting the first sample image and the plurality of sub-sample images corresponding to the first sample image into the initial network model is not limited, that is, the first feature vector set corresponding to the first sample image can be obtained first, or the second feature vector set corresponding to the plurality of sub-sample images can be obtained first.

[0050] S203: determine a first loss amount according to the first feature vector set and the second feature vector set.

[0051] After obtaining the first feature vector set and the second feature vector set based on the initial network model, the first loss amount can be determined according to the first feature vector set and the second feature vector set.

[0052] In a possible implementation, the first loss amount between the first feature vector set and the second feature vector set can be calculated by using a contrast loss function. The loss function is usually used to represent the matching degree between samples, and can also be used to train the model for extracting features. Generally, two originally similar training samples are still similar in the feature space after feature extraction by the model; two originally dissimilar samples are still dissimilar in the feature space after feature extraction by the model. When two similar samples are subjected to feature extraction, the loss amount between the two feature vectors calculated by using the loss function should also be small. The loss function can determine the loss amount between the two feature vectors by calculating the distance, such as the Euclidean distance, between the two feature vectors, so as to determine whether the initial network model is trained well.

[0053] S204: Adjust the parameters of the initial network model based on the first loss amount, re-execute the process of inputting the first sample image and the plurality of sub-sample images into the initial network model and the subsequent training process until the first loss amount is less than the first threshold value, and obtain a character recognition network model.

[0054] In the embodiment, the sub-sample images are obtained by dividing the first sample image, and therefore the plurality of sub-sample images have a high similarity with the first sample image. Therefore, after feature extraction by the initial network model, the first feature vector set and the second feature vector set should have a high similarity, i.e., the first loss amount determined based on the first feature vector set and the second feature vector set should be small. Therefore, when the first loss amount is greater than or equal to the first threshold value, it indicates that the similarity between the first feature vector set and the second feature vector set based on the initial network model does not meet the requirements, and the recognition effect of the initial network model after training does not meet the requirements. Therefore, the parameters of the initial network model need to be adjusted, and the initial network model is trained using the first sample image and the corresponding sub-sample images. That is, the process of inputting the first sample image and the plurality of sub-sample images into the initial network model and the subsequent training process is re-executed until the first loss amount determined based on the first feature vector set and the second feature vector set is less than the first threshold value, thereby obtaining a final character recognition network model.

[0055] It should be noted that in the process of training the initial network model at a time, the value of the division parameter is fixed, i.e., the width of each sub-sample image in the same training sample is the same. When the initial network model is trained again using the first sample image and the plurality of sub-sample images corresponding to the first sample image, the value of the division parameter can be different, i.e., the width of the sub-sample image can be different between different training samples. For example, the value range of the division parameter is set to 75%-95%, and the value interval of the division parameter is set to 5%, i.e., in different training samples, the possible values of the division parameter are 75%, 80%, 85%, 90%, and 95%. Training the initial network model using different training samples can improve the accuracy of feature extraction by the character recognition network model, and facilitate subsequent character recognition using the trained character recognition network model.

[0056] In actual application, the character recognition network model obtained by training may not be accurate enough due to insufficient samples for training the initial network model, which is manifested as a large first loss amount between the first feature vector set and the second feature vector set determined based on the character recognition network model. Embodiments of the present application provide a possible implementation manner, which obtains a processed first sample image by performing data enhancement processing on the first sample image, inputs the first sample image processed by data enhancement into the initial network model, and obtains the first feature vector set. The data enhancement processing method includes rotation, flip transformation, noise disturbance, etc., and the embodiments do not limit the specific manner of data enhancement.

[0057] In addition, the first sample image processed by data enhancement can also be divided into multiple sub-sample images, and the initial network model is trained by using the first sample image processed by data enhancement and the multiple sub-sample images corresponding thereto, so as to improve the accuracy of the character recognition network model.

[0058] In the training of the character recognition network model, the embodiments of the present application do not need to manually label the first sample image, but train the character recognition network model by aligning the local features (multiple sub-sample images) and the overall features (first sample image), so that the character recognition network model can extract complete features of the text information of the image, and then the complete text features can be used for character recognition, which can not only reduce the labeling cost and improve the training efficiency, but also lay a foundation for subsequent text recognition and improve the accuracy of recognition.

[0059] The process of training the character recognition network provided in the above embodiments mainly trains the encoder in the character recognition network model, that is, the text image to be processed is input into the character recognition network model, the encoder first pre-processes the text image, including steps such as digitization, geometric transformation, normalization, smoothing, etc., then extracts features from the pre-processed text image, and outputs the feature vector corresponding to the text image through a full connection layer. By training the encoder in the character recognition network model, the features of the text image extracted by the encoder can be more accurate, and the feature vector corresponding to the text image obtained can be more accurate.

[0060] In order to further improve the accuracy of the character recognition network model in recognizing text information, the embodiments of the present application also provide a preferred implementation manner, that is, the decoder in the character recognition network model is trained. The main function of the decoder is to decode the feature vector output by the encoder and recognize the text information corresponding to the feature vector. By training the function of the decoder in the character recognition network model, the accuracy of recognizing the text information can be improved. The process of training the character recognition network model to recognize the text information will be described below with reference to the drawings.

[0061] Referring toFigure 4 , Figure 4 Another flowchart for training a character recognition network model is provided in the embodiments of the present application.

[0062] The method mainly includes the following steps:

[0063] S401: Obtain a second sample image and a label corresponding to the second sample image, wherein the second sample image includes text information, and the label is used to reflect the text information.

[0064] In the embodiments, to further train the decoder of the character recognition network model, a second sample image with a label is obtained, the label reflecting text information of the second sample image, which is used to compare with the text information of the second sample image recognized by the character recognition network model, and then the character recognition network model is trained according to the comparison result.

[0065] S402: Input the second sample image into the character recognition network model to obtain a recognition result, the recognition result including text information.

[0066] The character recognition network model extracts features from the second sample image and recognizes the features to obtain the text information corresponding to the second sample image.

[0067] S403: Determine a second loss based on the recognition result and the label of the second sample image.

[0068] After obtaining the recognition result output by the character recognition network model, the second loss is determined based on the label in the second sample image and the recognition result, wherein the second loss represents the difference between the text information of the second sample image and the text information recognized by the character recognition network model.

[0069] S404: Adjust the parameters of the character recognition network model based on the second loss, and re-execute the process of inputting the second sample image into the character recognition network model and the subsequent training process until the second loss is less than a second threshold.

[0070] When the second loss is large, it indicates that there is a large difference between the text information in the second text image recognized by the character recognition network model and the labeled text information, and the parameters of the character recognition network model need to be adjusted, and the character recognition network model needs to be retrained, that is, the process of inputting the second sample image into the character recognition network model and the subsequent training process is re-executed until the second loss is less than the second threshold.

[0071] By training the decoder in the character recognition network model, the accuracy of the character recognition network model in recognizing text information based on features can be improved.

[0072] When the trained character recognition network model meets the requirements, the character recognition network model can be used for character recognition. That is, the text image to be processed is input into the character recognition network model to obtain an output result, and the output result includes the text information to be recognized.

[0073] In this embodiment, the character recognition network model has been preliminarily trained by the training method shown in FIG. 6, so that the character recognition network model can realize feature extraction and basic recognition function. When further training is performed by using the second sample image, the training can be completed without obtaining a large number of second sample images with labels, thereby reducing the training cost and improving the recognition accuracy. Figure 2

[0074] Based on the above method embodiments, the embodiments of the present application provide a device and equipment for implementing the above method, which will be described below with reference to the accompanying drawings.

[0075] Referring to FIG. 7, Figure 5 Figure 5 FIG. 7 is a structural diagram of a character recognition device provided by an embodiment of the present application. As shown in FIG. 7, the device 500 can include an acquisition unit 501 and a processing unit 502. Figure 5

[0076] The acquisition unit 501 is configured to acquire a text image to be processed, and the text image to be processed includes text information to be recognized.

[0077] The processing unit 502 is configured to input the text image to be processed into a character recognition network model to obtain an output result, and the output result includes the text information to be recognized. The character recognition network model is generated by training using training samples, and the training samples include a first sample image and a plurality of sub-sample images corresponding to the first sample image. The height of each sub-sample image in the plurality of sub-sample images is the same as the height of the first sample image, the width of each sub-sample image in the plurality of sub-sample images is the same, and the width of each sub-sample image is smaller than the width of the first sample image. The first sample image includes text information.

[0078] ​​​In a specific implementation, the processing unit 502 is specifically configured to input the first sample image and the plurality of sub-sample images into an initial network model respectively, obtain a first feature vector set and a second feature vector set, the dimension of the first feature vector set is the same as the dimension of the second feature vector set, and the feature vectors in the first feature vector set and the feature vectors in the second feature vector set are feature vectors of the text information; determine a first loss amount according to the first feature vector set and the second feature vector set, and adjust parameters of the initial network model based on the first loss amount, re-execute the inputting of the first sample image and the plurality of sub-sample images into the initial network model and subsequent training processes until the first loss amount is less than a first threshold value, and obtain the character recognition network model.

[0079] In a specific implementation, the processing unit 502 is specifically configured to determine a division parameter, and divide the first sample image multiple times by using the division parameter to obtain the plurality of sub-sample images, wherein a ratio of the width of each sub-sample image in the plurality of sub-sample images to the width of the sample image is equal to the division parameter, and the division parameter is greater than 0 and less than 1.

[0080] In a specific implementation, the processing unit 502 is specifically configured to, for each division operation, determine a starting position of division in the first sample image; and divide the first sample image according to the starting position and the division parameter to obtain the plurality of sub-sample images.

[0081] In a specific implementation, the value range of the division parameter is 75%-95%.

[0082] In a specific implementation, the obtaining unit 501 is further configured to obtain a second sample image and a label corresponding to the second sample image, the second sample image includes text information, and the label is used to reflect the text information.

[0083] The processing unit 502 is further configured to input the second sample image into the character recognition network model to obtain a recognition result, the recognition result includes the text information; determine a second loss amount based on the recognition result and the label of the second sample image, and adjust parameters of the character recognition network model based on the second loss amount, re-execute the inputting of the second sample image into the character recognition network model and subsequent training processes until the second loss amount is less than a second threshold value.

[0084] In a specific implementation, the processing unit 502 is specifically configured to perform data enhancement processing on the first sample image to obtain a processed first sample image; and input the processed first sample image into the initial network model to obtain the first feature vector set.

[0085] In a specific implementation, the processing unit 502 is specifically configured to calculate a first loss amount between the first feature vector set and the second feature vector set by using a contrast loss function.

[0086] It should be noted that the implementation of each unit in this embodiment can refer to the related description in the above method embodiments, and this embodiment will not be described here again.

[0087] Referring to Figure 6 , a structural diagram of an electronic device 600 suitable for implementing the embodiments of the present application is shown. The terminal device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant, personal digital assistant), PADs (portable android devices, tablet computers), PMPs (Portable Media Player, portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. Figure 6 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0088] As shown in Figure 6 , the electronic device 600 can include a processing device (such as a central processor, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded into a random access memory (RAM) 603 from a storage device 608. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0089] In general, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 608 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 609. The communication devices 609 can allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it is understood that all of the illustrated devices are not required to implement or be present. More or fewer devices can alternatively be implemented or present.

[0090] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication devices 609, or installed from the storage devices 608, or installed from the ROM 602. When the computer program is executed by the processing devices 601, the above-mentioned functions defined in the methods of embodiments of the present application are performed.

[0091] The electronic device provided by the embodiments of the present application and the group member adding method provided by the above-mentioned embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above-mentioned embodiments, and the present embodiment has the same beneficial effects as the above-mentioned embodiments.

[0092] The embodiments of the present application provide a computer readable medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method according to any one of the above-mentioned embodiments.

[0093] It should be noted that the computer-readable medium in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium that can send, propagate or transmit the program for use by or in connection with an instruction execution system, apparatus or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including, but not limited to, a wire, cable, optical fiber, RF (radio frequency), or any suitable combination of the above.

[0094] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0095] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and be not assembled into the electronic device.

[0096] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method described above.

[0097] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0098] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0099] The units involved in the embodiments described in the present application can be implemented by software, or can be implemented by hardware. Among them, the name of the unit / module does not constitute a limitation to the unit itself in some cases. For example, the voice data acquisition module can also be described as a "data acquisition module".

[0100] The functions described in the above description above can be performed at least in part by one or more hardware logic components. For example, non-limiting examples of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0101] In the context of this application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] According to one or more embodiments of the present application, a character recognition method is provided, which can include:

[0103] obtaining a text image to be processed, the text image to be processed including text information to be recognized;

[0104] inputting the text image to be processed into a character recognition network model to obtain an output result, the output result including the text information to be recognized;

[0105] wherein the character recognition network model is generated by training using training samples, the training samples including a first sample image and a plurality of sub-sample images corresponding to the first sample image, each of the plurality of sub-sample images having a same height as the first sample image, each of the plurality of sub-sample images having a same width, the width of each of the plurality of sub-sample images being smaller than the width of the first sample image, and the first sample image including text information.

[0106] According to one or more embodiments of the present application, a training process of the character recognition network model includes:

[0107] inputting the first sample image and the plurality of sub-sample images into an initial network model respectively to obtain a first feature vector set and a second feature vector set, the first feature vector set having a same dimension as the second feature vector set, and each feature vector in the first feature vector set and each feature vector in the second feature vector set being a feature vector of the text information;

[0108] determine a first loss amount according to the first feature vector set and the second feature vector set, and adjust parameters of the initial network model based on the first loss amount, re-execute the process of inputting the first sample image and the plurality of sub-sample images into the initial network model and the subsequent training process until the first loss amount is less than a first threshold value, and obtain the character recognition network model.

[0109] According to one or more embodiments of the present application, the process of obtaining the plurality of sub-sample images comprises:

[0110] determine a division parameter, and divide the first sample image multiple times by using the division parameter to obtain the plurality of sub-sample images, wherein a ratio of a width of each sub-sample image in the plurality of sub-sample images to a width of the sample image is equal to the division parameter, and the division parameter is greater than 0 and less than 1.

[0111] According to one or more embodiments of the present application, the process of dividing the sample image multiple times by using the division parameter to obtain the plurality of sub-sample images comprises:

[0112] for each division operation, determine a starting position of division in the first sample image;

[0113] divide the first sample image according to the starting position and the division parameter to obtain the plurality of sub-sample images.

[0114] According to one or more embodiments of the present application, the division parameter is in a range of 75% to 95%.

[0115] According to one or more embodiments of the present application, the method further comprises:

[0116] obtain a second sample image and a label corresponding to the second sample image, wherein the second sample image comprises text information, and the label is used to reflect the text information;

[0117] input the second sample image into the character recognition network model to obtain a recognition result, wherein the recognition result comprises the text information;

[0118] determine a second loss amount based on the recognition result and the label of the second sample image, and adjust parameters of the character recognition network model based on the second loss amount, re-execute the process of inputting the second sample image into the character recognition network model and the subsequent training process until the second loss amount is less than a second threshold value.

[0119] According to one or more embodiments of the present application, the process of inputting the first sample image into the initial network model to obtain a first feature vector set comprises:

[0120] perform data augmentation processing on the first sample image to obtain a processed first sample image;

[0121] input the processed first sample image into the initial network model to obtain the first feature vector set.

[0122] According to one or more embodiments of the present application, the determining a first loss amount according to the first feature vector set and the second feature vector set comprises:

[0123] calculating the first loss amount between the first feature vector set and the second feature vector set by using a contrastive loss function.

[0124] According to one or more embodiments of the present application, a character recognition device is provided, which can comprise:

[0125] an obtaining unit configured to obtain a text image to be processed, the text image to be processed comprising text information to be recognized;

[0126] a processing unit configured to input the text image to be processed into a character recognition network model to obtain an output result, the output result comprising the text information to be recognized; wherein the character recognition network model is generated by training using training samples, the training samples comprising a first sample image and a plurality of sub-sample images corresponding to the first sample image, each sub-sample image in the plurality of sub-sample images having a same height as the first sample image, each sub-sample image in the plurality of sub-sample images having a same width, and the width of each sub-sample image being smaller than the width of the first sample image, the first sample image comprising text information.

[0127] In one or more embodiments of the present application, the processing unit is specifically configured to input the first sample image and the plurality of sub-sample images into an initial network model respectively to obtain a first feature vector set and a second feature vector set, the first feature vector set having a same dimension as the second feature vector set, the feature vectors in the first feature vector set and the feature vectors in the second feature vector set being feature vectors of the text information; determine a first loss amount according to the first feature vector set and the second feature vector set, and adjust parameters of the initial network model based on the first loss amount, re-execute the inputting the first sample image and the plurality of sub-sample images into the initial network model and subsequent training processes until the first loss amount is smaller than a first threshold value, and obtain the character recognition network model.

[0128] In one or more embodiments of the present application, the processing unit is specifically configured to determine a division parameter, and divide the first sample image multiple times using the division parameter to obtain the plurality of sub-sample images, wherein a ratio of a width of each sub-sample image in the plurality of sub-sample images to a width of the sample image is equal to the division parameter, and the division parameter is greater than 0 and less than 1.

[0129] In one or more embodiments of the present application, the processing unit is specifically configured to, for each division operation, determine a starting position of division in the first sample image; and divide the first sample image according to the starting position and the division parameter to obtain the plurality of sub-sample images.

[0130] In one or more embodiments of the present application, the division parameter is in a range of 75% to 95%.

[0131] In one or more embodiments of the present application, the obtaining unit is further configured to obtain a second sample image and a label corresponding to the second sample image, wherein the second sample image comprises text information, and the label is used to reflect the text information.

[0132] The processing unit is further configured to input the second sample image into the character recognition network model to obtain a recognition result, wherein the recognition result comprises the text information; determine a second loss based on the recognition result and the label of the second sample image; adjust parameters of the character recognition network model based on the second loss; and re-perform inputting the second sample image into the character recognition network model and subsequent training processes until the second loss is less than a second threshold.

[0133] In one or more embodiments of the present application, the processing unit is specifically configured to perform data enhancement processing on the first sample image to obtain a processed first sample image; and input the processed first sample image into the initial network model to obtain the first feature vector set.

[0134] In one or more embodiments of the present application, the processing unit is specifically configured to calculate a first loss between the first feature vector set and the second feature vector set using a contrastive loss function.

[0135] According to one or more embodiments of the present application, an electronic device is provided, which comprises a processor and a memory.

[0136] The memory is configured to store instructions or computer programs.

[0137] The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device performs the character recognition method.

[0138] According to one or more embodiments of the present application, a computer readable storage medium is provided, in which instructions are stored, when the instructions are run on a device, cause the device to perform the character recognition method.

[0139] It should be noted that the various embodiments described in the specification are presented using a progression of embodiments, each presenting one or more aspects of the embodiments described in the specification. Each of the various embodiments can be provided independently, all combinations thereof, or any other combination of one or more of the various embodiments described in the specification. The embodiments are presented using the following terminology: an aspect, one aspect, an implementation, one implementation, an embodiment, one embodiment, and the like, which are used interchangeably herein. Each of the various embodiments can be provided independently, all combinations thereof, or any other combination of one or more of the various embodiments described in the specification.

[0140] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases of only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0141] It should also be noted that in this document, relationship terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0142] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and

[0143] The above description of disclosed embodiments is intended to be illustrative and not restrictive. Many embodiments of the application will be apparent to those of skill in the art upon reviewing the above description. The scope of the application should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the appended claims, along with their full scope of equivalents.

Claims

1. A character recognition method, characterized in that, The method includes: Acquire a text image to be processed, wherein the text image to be processed includes text information to be recognized; The text image to be processed is input into a character recognition network model to obtain an output result, the output result including the text information to be recognized; The character recognition network model is generated by training samples, which include a first sample image and multiple sub-sample images corresponding to the first sample image. The height of each sub-sample image is the same as the height of the first sample image, the width of each sub-sample image is the same, and the width of each sub-sample image is less than the width of the first sample image. The first sample image includes text information. The character recognition network model is generated by aligning the local features corresponding to the multiple sub-sample images with the overall features corresponding to the first sample image. The process of acquiring the multiple sub-sample images includes: Determine the segmentation parameters, and use the segmentation parameters to segment the first sample image multiple times to obtain the plurality of sub-sample images. The ratio of the width of each sub-sample image to the width of the first sample image is equal to the segmentation parameters, and the segmentation parameters are greater than 0 and less than 1.

2. The method according to claim 1, characterized in that, The training process of the character recognition network model includes: The first sample image and the plurality of sub-sample images are respectively input into the initial network model to obtain a first feature vector set and a second feature vector set. The dimension of the first feature vector set is the same as the dimension of the second feature vector set. The feature vectors in the first feature vector set and the feature vectors in the second feature vector set are the feature vectors of the text information. A first loss is determined based on the first feature vector set and the second feature vector set, and the parameters of the initial network model are adjusted based on the first loss. The process of inputting the first sample image and the multiple sub-sample images into the initial network model and subsequent training is repeated until the first loss is less than the first threshold, thereby obtaining the character recognition network model.

3. The method according to claim 1, characterized in that, The step of dividing the sample image multiple times using the division parameters to obtain the multiple sub-sample images includes: For each segmentation operation, the starting position of the segmentation is determined in the first sample image; The first sample image is divided according to the starting position and the division parameters to obtain the plurality of sub-sample images.

4. The method according to claim 1 or 3, characterized in that, The value range of the division parameter is between 75% and 95%.

5. The method according to claim 2, characterized in that, The method further includes: Acquire a second sample image and its corresponding annotation, wherein the second sample image includes text information and the annotation is used to reflect the text information; The second sample image is input into the character recognition network model to obtain a recognition result, which includes the text information. Based on the recognition result and the annotation of the second sample image, a second loss is determined, and the parameters of the character recognition network model are adjusted based on the second loss. The process of inputting the second sample image into the character recognition network model and subsequent training is repeated until the second loss is less than the second threshold.

6. The method according to claim 2, characterized in that, The step of inputting the first sample image into the initial network model to obtain the first feature vector set includes: The first sample image is subjected to data augmentation processing to obtain the processed first sample image; The processed first sample image is input into the initial network model to obtain the first feature vector set.

7. The method according to claim 2, characterized in that, Determining the first loss amount based on the first feature vector set and the second feature vector set includes: The first loss between the first feature vector set and the second feature vector set is calculated using the contrastive loss function.

8. A character recognition device, characterized in that, The device includes: The acquisition unit is used to acquire a text image to be processed, wherein the text image to be processed includes text information to be recognized; A processing unit is configured to input the text image to be processed into a character recognition network model and obtain an output result, the output result including the text information to be recognized; wherein, the character recognition network model is generated by training using training samples, the training samples including a first sample image and multiple sub-sample images corresponding to the first sample image, the height of each sub-sample image being the same as the height of the first sample image, the width of each sub-sample image being the same, and the width of each sub-sample image being less than the width of the first sample image, the first sample image including text information, and the character recognition network model being generated by aligning the local features corresponding to the multiple sub-sample images with the overall features corresponding to the first sample image; The process of acquiring the multiple sub-sample images includes: Determine the segmentation parameters, and use the segmentation parameters to segment the first sample image multiple times to obtain the plurality of sub-sample images. The ratio of the width of each sub-sample image to the width of the first sample image is equal to the segmentation parameters, and the segmentation parameters are greater than 0 and less than 1.

9. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the character recognition method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on the device, cause the device to perform the character recognition method according to any one of claims 1-7.

Images