Character recognition method, device, intelligent food collection cabinet, electronic device and storage medium

Abstract

The present application provides a character recognition method, device, intelligent food collection cabinet, electronic device, and storage medium. The character recognition method includes: converting an image to be recognized into a standard format image; determining a character region within the standard format image; and identifying a character string within the character region based on a lightweight character recognition neural network. The present application simplifies the complexity and content of images requiring neural network processing. Therefore, a lightweight character recognition neural network can be used to quickly and accurately implement character recognition, achieving a faster response speed. Furthermore, the use of a lightweight character recognition neural network significantly reduces the computing performance requirements of the device.

Description

Technical Field

[0001] The present application relates to the technical field of optical character recognition, and in particular to a character recognition method, device, intelligent food collection cabinet, electronic device and storage medium. Background Art

[0002] Optical character recognition (OCR) is a technology that identifies text in images and extracts usable character information. It is increasingly used in fields such as computer vision and artificial intelligence. Currently, neural networks are the mainstream tool for OCR. Applying neural networks to OCR can effectively improve the accuracy and efficiency of character recognition.

[0003] However, in order to obtain higher recognition accuracy and adapt to various image conditions, the existing neural networks used for optical character recognition are designed to be relatively large and complex, requiring the use of high-performance computers or dedicated servers for calculations, with high implementation costs and poor recognition timeliness. This makes it difficult to apply such large and complex neural networks in some real application scenarios such as mobile or embedded devices. First, the model is too large and faces problems such as insufficient memory. Second, these scenarios require low latency, or fast response speed, which the above neural networks are difficult to meet. For example, if an optical character recognition function is added to a smart food collection cabinet to identify the food collection code, there will be requirements for low computing performance and fast response of the device to meet the customer's user experience.

[0004] Therefore, there is an urgent need to provide a character recognition solution that has low requirements on device computing performance and fast response. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a character recognition method, device, intelligent food collection cabinet, electronic device and storage medium to solve the problem that the current character recognition method cannot take into account the accuracy, efficiency and cost.

[0006] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0007] The first aspect of the present application provides a character recognition method, comprising:

[0008] Convert the image to be recognized into a standard format image;

[0009] determining a character area in the standard format image;

[0010] The character string in the character area is recognized based on a lightweight character recognition neural network.

[0011] In some modified implementations of the first aspect of the present application, converting the image to be recognized into a standard format image includes:

[0012] Convert the image to be recognized into a standard format image that conforms to a preset color mode and / or a preset image size.

[0013] In some modified implementations of the first aspect of the present application, determining the character area in the standard format image includes:

[0014] A lightweight text detection neural network is used to determine the character area in the standard format image.

[0015] In some modified implementations of the first aspect of the present application, the lightweight text detection neural network includes:

[0016] PSENet is a progressive scaling network that uses the lightweight neural network MobileNet as the backbone network.

[0017] In some modified implementations of the first aspect of the present application, the identifying the character string in the character area based on the lightweight character recognition neural network includes:

[0018] extracting a character region image including the character region from the standard format image through affine transformation;

[0019] Based on a lightweight character recognition neural network, a character string in the character area is recognized from the character area image.

[0020] In some modified implementations of the first aspect of the present application, the lightweight character recognition neural network includes:

[0021] Lightweight Convolutional Recurrent Neural Networks.

[0022] In some modified implementations of the first aspect of the present application, the output layer of the lightweight convolutional recurrent neural network adopts local sensitive hashing (LSH) encoding.

[0023] In some modified implementations of the first aspect of the present application, the character string recognized based on the lightweight character recognition neural network includes multiple;

[0024] The method further comprises:

[0025] A target character string that meets the constraints of the current application scenario is obtained by screening the multiple character strings.

[0026] In some modified implementations of the first aspect of the present application, the step of screening the target character string from the plurality of character strings to obtain a target character string that meets the constraints of the current application scenario includes:

[0027] At least one of regular expression matching, background information matching, and confidence matching is used to match the target character string that meets the constraints of the current application scenario from the multiple character strings.

[0028] A second aspect of the present application provides a character recognition device, comprising:

[0029] A standard format conversion module, used for converting the image to be recognized into a standard format image;

[0030] A character region determination module, configured to determine a character region in the standard format image;

[0031] The character string extraction module is used to identify the character string in the character area based on a lightweight character recognition neural network.

[0032] In some modified implementations of the second aspect of the present application, the standard format conversion module includes:

[0033] The standard format conversion unit is used to convert the image to be recognized into a standard format image that conforms to a preset color mode and / or a preset image size.

[0034] In some modified implementations of the second aspect of the present application, the character region determination module includes:

[0035] The character region determining unit is used to determine the character region in the standard format image by using a lightweight text detection neural network.

[0036] In some modified implementations of the second aspect of the present application, the lightweight text detection neural network includes:

[0037] PSENet is a progressive scaling network that uses the lightweight neural network MobileNet as the backbone network.

[0038] In some modified implementations of the second aspect of the present application, the string extraction module includes:

[0039] a character region image extraction unit, configured to extract a character region image containing the character region from the standard format image through affine transformation;

[0040] The character string extraction unit is configured to recognize a character string in the character region from the character region image based on a lightweight character recognition neural network.

[0041] In some modified implementations of the second aspect of the present application, the lightweight character recognition neural network includes:

[0042] Lightweight Convolutional Recurrent Neural Networks.

[0043] In some modified implementations of the second aspect of the present application, the output layer of the lightweight convolutional recurrent neural network adopts local sensitive hashing (LSH) encoding.

[0044] In some modified implementations of the second aspect of the present application, the character string recognized based on the lightweight character recognition neural network includes multiple;

[0045] The device further comprises:

[0046] The target character string screening module is used to screen the target character string that meets the constraints of the current application scenario from the multiple character strings.

[0047] In some modified implementations of the second aspect of the present application, the target string screening module includes:

[0048] The target character string screening unit is used to adopt at least one of regular expression matching, background information matching and confidence matching to match a target character string that meets the constraints of the current application scenario from the multiple character strings.

[0049] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect of the present application.

[0050] A fourth aspect of the present application provides a computer-readable storage medium having computer-readable instructions stored thereon, which can be executed by a processor to implement the method described in the first aspect of the present application.

[0051] The fifth aspect of the present application provides an intelligent food collection cabinet, comprising: a cabinet body, and an image acquisition device and a main control device provided on the cabinet body; wherein,

[0052] The image acquisition device is connected to the main control device;

[0053] The image acquisition device is used to capture an image of the meal voucher to generate an image to be recognized, and send the image to be recognized to the main control device;

[0054] The main control device is used to identify the character string from the image to be identified using the method described in the first aspect of this application to determine the meal pickup code recorded on the meal pickup voucher, and control the opening of the cabinet door of the designated meal pickup position in the cabinet according to the meal pickup code.

[0055] The purpose of the embodiments of the present application is to provide a character recognition method, device, intelligent food collection cabinet, electronic device and storage medium. First, the image to be recognized is converted into a standard format image, then the character area in the standard format image is determined, and then the character string in the character area is recognized based on a lightweight character recognition neural network. By converting the image to be recognized into a standard format image, the image to be recognized by the character recognition neural network can be made simpler and more consistent. By determining the character area and then inputting it into the neural network for recognition, the data processing volume of the neural network can be effectively reduced, the performance requirements of the computing device can be reduced, and the computing overhead can be reduced. Therefore, the above scheme can effectively reduce the requirements for the character recognition neural network, so that it does not have to deal with and process complex and irregular images. Therefore, a lightweight character recognition neural network can be used to implement character recognition. Overall, since the complexity and content of the image to be processed by the neural network are simplified in advance, the lightweight character recognition neural network can be used to quickly and accurately implement the character recognition function with a faster response speed. In addition, since the lightweight character recognition neural network is used, the computing performance requirements of the device can be greatly reduced, so that mobile devices, embedded devices, etc. can also implement character recognition, which is easy to promote and implement and has broad application prospects.

[0056] The second aspect of this application provides a character recognition device, the third aspect provides an electronic device, the fourth aspect provides a computer-readable storage medium, and the fifth aspect provides an intelligent food collection cabinet. They are based on the same inventive concept as the character recognition method provided in the first aspect of this application and have at least the same beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0058] Figure 1 Schematically illustrates a flow chart of a character recognition method provided by some embodiments of the present application;

[0059] Figure 2 A schematic diagram schematically illustrates a standard format image provided by some embodiments of the present application;

[0060] Figure 3 A schematic diagram schematically illustrates a character area image provided by some embodiments of the present application;

[0061] Figure 4Schematically illustrates a calculation diagram of a matrix multiplication based on a One-Hot encoding method provided in some embodiments of the present application;

[0062] Figure 5 Schematically illustrates a calculation diagram of a matrix multiplication based on an LSH encoding method provided in some embodiments of the present application;

[0063] Figure 6 A schematic diagram of a character recognition device provided in some embodiments of the present application is schematically shown;

[0064] Figure 7 A schematic diagram schematically illustrates an electronic device provided by some embodiments of the present application;

[0065] Figure 8 A schematic diagram of a computer-readable storage medium provided in some embodiments of the present application is schematically shown. DETAILED DESCRIPTION

[0066] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0067] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0068] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in the examples of this application and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0069] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0070] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0071] In addition, the terms "first" and "second" are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0072] The embodiments of the present application provide a character recognition method, device, smart food collection cabinet, electronic device and storage medium, which are exemplarily described below with reference to the accompanying drawings.

[0073] Please refer to Figure 1 , which schematically shows a flow chart of a character recognition method provided by some embodiments of the present application, such as Figure 1 As shown, a character recognition method may include the following steps:

[0074] Step S101: converting the image to be recognized into a standard format image.

[0075] Among them, the above-mentioned image to be recognized can be an image containing characters to be recognized collected by an image acquisition device such as a camera or a scanner, and can be in any format such as jpg or tif, which is not limited in the embodiment of the present application.

[0076] Considering that the sizes, color modes, etc. of the images to be recognized acquired by different image acquisition devices are not the same, the diversity of the images to be recognized will inevitably bring higher requirements to the subsequent neural network used for character recognition. For example, a large number of diverse training samples are required to train the neural network, and the neural network needs to have more layers and higher complexity. These will affect the realization of lightweight character recognition neural network. Therefore, in the embodiment of the present application, by converting the image to be recognized into a standard format image, the simplification of the image to be recognized can be achieved, so that the subsequent images that need to be recognized by the character recognition neural network are simpler and more consistent, and the requirements for the character recognition neural network are reduced. For example, the subsequent character recognition neural network only needs to use a relatively small number of standard format images for training, and it can be achieved with fewer layers and lower network depth, which helps to achieve the lightweight character recognition neural network, so that the above-mentioned standard format images can be recognized by using a lightweight character recognition neural network.

[0077] In practical applications, the above-mentioned conversion of the image to be recognized into a standard format image may include:

[0078] Convert the image to be recognized into a standard format image that conforms to a preset color mode and / or a preset image size.

[0079] The above-mentioned standard format image may refer to conforming to a preset color mode, conforming to a preset image size, or conforming to both, thereby achieving simplification and uniformity of the image to be identified.

[0080] The embodiment of the present application does not limit the above-mentioned preset color mode and preset image size. Those skilled in the art can flexibly set them according to actual needs. For example, the above-mentioned preset color mode can adopt RGB mode, or CMYK mode, HSB mode, Lab color mode, etc. As long as a unified standard is set, it can achieve the purpose of the embodiment of the present application. In some examples, the image to be identified can be uniformly converted into a 3-channel RGB image. Since images in RGB color mode are more common in actual applications, after setting it as a standard format image, there is no need to perform color mode conversion on the image to be identified in the original RGB mode, and only the image to be identified in the non-RGB mode needs to be converted into a color mode. As a whole, the workload of color mode conversion in this step can be effectively reduced.

[0081] In addition, the embodiments of the present application do not limit the above-mentioned preset image size. The above-mentioned preset image size may include requirements for the height and width of the image, so as to uniformly process the image to be identified into an image of fixed height and width, thereby achieving the consistency of the image to be identified. Among them, the above-mentioned preset image size may include requirements for specific pixel values ​​and ratio requirements, which are not limited in the embodiments of the present application.

[0082] In some examples, in order to maintain the image ratio, when converting the image to be recognized into a standard format image that meets the preset image size, the image content can be filled. For example, if the preset image size ratio is 4:3 and the ratio of the image to be recognized is 16:9, the height of the image to be recognized can be filled to convert the image to be recognized into a 4:3 standard format image.

[0083] Step S102: Determine the character area in the standard format image.

[0084] Considering that characters may only exist in a partial area of ​​the image to be recognized, in order to avoid character recognition in non-character areas and improve the efficiency of subsequent character recognition, it is necessary to first determine and extract the character area in the standard format image. Specifically, in some embodiments, the above-mentioned determination of the character area in the standard format image includes:

[0085] A lightweight text detection neural network is used to determine the character area in the standard format image.

[0086] Since step S101 has standardized the image to be recognized, thus achieving simplification and consistency of the image to be recognized, this step can use a lightweight text detection neural network to determine the character area in the above-mentioned standard format image.

[0087] Among them, the embodiments of the present application do not limit the specific implementation method of the above-mentioned lightweight text detection neural network. For example, it can adopt the progressive scale expansion network (PSENet), pixel aggregation network (PANNet), differentiable binarization network (DBNet) DBNet and other text detection neural networks provided in the prior art, and implement them after lightweight processing. Among them, the above-mentioned lightweight processing can be achieved by replacing the backbone network (BN) in the above-mentioned text detection neural network with a lightweight neural network (such as MobileNet, SqueezeNet, ShuffleNet, NasNet, MnasNet, EfficientNet, etc.). It can be specifically implemented with reference to the lightweight processing technology for neural networks provided by the prior art, which will not be repeated here.

[0088] In some specific examples, the lightweight text detection neural network described above can include a progressive scaling network (PSENet) using the lightweight MobileNet neural network as its backbone. This lightweight text detection neural network offers the advantages of high detection accuracy and speed, effectively improving the overall character recognition accuracy and response speed of this solution.

[0089] The recognized character area is generally a rectangle, which specifically includes coordinate information of the four vertices of the rectangle.

[0090] It should be noted that the character region identified in this step is not limited to one, and may be multiple. In particular, multiple character regions need to undergo character recognition in subsequent steps.

[0091] Step S103: Recognize the character string in the character area based on the lightweight character recognition neural network.

[0092] After the character region is identified, the lightweight character recognition neural network can be used to identify the string in the character region. Specifically, in some embodiments, the lightweight character recognition neural network-based identification of the string in the character region includes:

[0093] extracting a character region image including the character region from the standard format image through affine transformation;

[0094] Based on a lightweight character recognition neural network, a character string in the character area is recognized from the character area image.

[0095] Affected by the acquisition angle of the image to be recognized, the text in the above-mentioned image to be recognized may have problems such as deformity and skewness. Since the above-mentioned steps did not perform orthodontic operations, the text in the above-mentioned standard format image still has problems such as deformity and skewness. In order to reduce the requirements for the lightweight character recognition neural network and improve the recognition efficiency of the subsequent lightweight character recognition neural network, in this step, first, the character area image containing the character area is extracted from the above-mentioned standard format image through affine transformation, and then the lightweight character recognition neural network is used to recognize the character string in the above-mentioned character area. In this way, the character area image that the lightweight character recognition neural network needs to face and process can be relatively simple and consistent, which can effectively improve the recognition accuracy and efficiency, and ensure that the lightweight character recognition neural network can accurately and efficiently realize the recognition of the character string in the above-mentioned character area image.

[0096] For details, please refer to Figure 2 and Figure 3 To understand, Figure 2 The following schematically illustrates a standard format image provided by some embodiments of the present application. Figure 3 A schematic diagram of a character area image provided by some embodiments of the present application is schematically shown, such as Figure 2 、 Figure 3 As shown, this embodiment can directly use affine transformation to extract the character area image from the standard format image based on the vertex coordinates of the character area circumscribed rectangle. For example, first, the vertex coordinates of the source character area circumscribed rectangle and the vertex coordinates of the target rectangular area in the standard format image are set. The vertex coordinates of the target rectangle can be directly set to (0, 0) in the upper left corner. In this way, after subsequent transformation, the character area image can be directly extracted from the standard format image at one time. Then, the affine transformation matrix is ​​calculated and the matrix is ​​used to extract the character area image.

[0097] In the above embodiment, the above-mentioned lightweight character recognition neural network can adopt a lightweight convolutional recurrent neural network, wherein the lightweight convolutional recurrent neural network can be obtained by performing lightweight processing on the convolutional recurrent neural network (CRNN), and the above-mentioned lightweight processing can be achieved by adopting local sensitive hashing LSH encoding for the output layer of the CRNN.

[0098] Specifically, the output layer of the traditional CRNN adopts the One-Hot output encoding method. Since the scale of the Chinese and English character sets is generally large (for example, 10,000), the output vector dimension of the One-Hot encoding method must also match it, which leads to a large number of parameters of the classification matrix W of the last fully connected layer of the CRNN model, and the amount of matrix multiplication calculation is also large (please refer to Figure 4 To understand,), which ultimately affects the overall scale and reasoning speed of the CRNN model. Therefore, in the embodiment of the present application, the locality-sensitive hashing (LSH) output encoding method is used in the output layer of CRNN to replace the above-mentioned One-Hot output encoding method. The LSH output encoding method uses the maximum interval loss during training, binarizes the logit during reasoning, and then calculates the Hamming distance with the LSH encoding of all characters. The character with the closest distance is the prediction result. Analogous to the example of the above-mentioned One-Hot encoding method, after adopting LSH encoding, the dimension of the classification matrix W is reduced from (10000*128) to (256*128), a reduction of 97.4%, which can greatly reduce the computational amount of matrix multiplication (please refer to Figure 5 contrast Figure 4 to understand).

[0099] This embodiment adopts an efficient local sensitive hashing (LSH) encoding output method to replace the traditional one-hot output encoding method, which can significantly reduce the scale of the character recognition model and improve the inference speed without losing accuracy.

[0100] Through the above implementation, the lightweight CRNN can be achieved. Since CRNN has the advantages of high recognition accuracy and fast speed, the lightweight CRNN also inherits the above advantages. Therefore, the use of lightweight CRNN to realize the recognition of character strings in the above character area image can effectively improve the accuracy and speed of character recognition, while reducing the requirements for the performance of the computing device.

[0101] Based on any of the foregoing embodiments, in some modified embodiments, the character string recognized based on the lightweight character recognition neural network includes multiple;

[0102] The method further comprises:

[0103] A target character string that meets the constraints of the current application scenario is obtained by screening the multiple character strings.

[0104] It is easy to understand that the image to be identified may contain multiple character strings, but some of the character strings do not need to be extracted. Therefore, this embodiment can further filter out the target character string that meets the constraints of the current application scenario from the multiple character strings to meet the actual needs of the application scenario.

[0105] In some variations of the above embodiment, the step of filtering the target character string from the plurality of character strings to obtain a character string that meets the constraints of the current application scenario includes:

[0106] At least one of regular expression matching, background information matching, and confidence matching is used to match the target character string that meets the constraints of the current application scenario from the multiple character strings.

[0107] For example, in the application scenario of a smart tableware cabinet recognizing the meal pickup code in a meal pickup voucher, first, each single character string identified in step S103 is further matched with a regular expression. For example, the first character in the smart tableware cabinet item needs to be an uppercase character, and the next four characters need to be numbers. Only the strings that meet the conditions are screened out as candidate strings. Then, the candidate strings are further screened based on the constraint conditions. For example, if there is only one valid character (meal pickup code) that meets the requirements on the same meal pickup voucher in the smart tableware cabinet, when there are multiple candidate strings, they are screened based on the environmental information of the candidate strings (i.e., background information matching) and the confidence information of the character recognition results (i.e., confidence matching).

[0108] In the above implementation, the candidate character strings are screened and judged by using a regular expression or other method, which can complete the processing very quickly and accurately, and can be customized according to actual needs. Thus, the target character string that meets the needs of the actual application scenario is screened, such as the meal pickup code.

[0109] The at least one character recognition method provided in the embodiment of the present application first converts the image to be recognized into a standard format image, then determines the character area in the standard format image, and then recognizes the character string in the character area based on a lightweight character recognition neural network. By converting the image to be recognized into a standard format image, the image to be recognized by the character recognition neural network can be made simpler and more consistent. By determining the character area and then inputting it into the neural network for recognition, the data processing volume of the neural network can be effectively reduced, the performance requirements of the computing device can be reduced, and the computing overhead can be reduced. Therefore, the above scheme can effectively reduce the requirements for the character recognition neural network, so that it does not have to face and process complex and irregular images. Therefore, a lightweight character recognition neural network can be used to achieve character recognition. Overall, since the complexity and content of the image to be processed by the neural network are simplified in advance, the lightweight character recognition neural network can be used to quickly and accurately realize the character recognition function with a faster response speed. Moreover, since a lightweight character recognition neural network is used, the requirements for the computing performance of the device can be greatly reduced, so that mobile devices, embedded devices, etc. can also achieve character recognition, which is easy to promote and implement and has broad application prospects.

[0110] Furthermore, to meet the requirements of real-time character recognition, the character recognition method provided in any of the aforementioned embodiments of this application not only utilizes lightweight and fast neural network models as much as possible when designing them, but also performs specialized acceleration and compression processing on the models, including model quantization, pruning, and distillation. This effectively improves character recognition efficiency and accuracy, and is more applicable to devices with limited computing power, such as mobile devices and embedded systems.

[0111] In the above embodiment, a character recognition method is provided, and correspondingly, the present application also provides a character recognition device. The character recognition device provided in the embodiment of the present application can implement the above character recognition method, and the character recognition device can be implemented by software, hardware, or a combination of software and hardware. For example, the character recognition device can include integrated or separate functional modules or units to perform the corresponding steps in the above methods. Please refer to Figure 6 , which schematically illustrates a schematic diagram of a character recognition device provided by some embodiments of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment. The device embodiment described below is merely illustrative.

[0112] like Figure 6 As shown, the character recognition device 10, used in the server, may include:

[0113] The standard format conversion module 101 is used to convert the image to be recognized into a standard format image;

[0114] A character region determining module 102 is configured to determine a character region in the standard format image;

[0115] The character string extraction module 103 is configured to recognize the character string in the character area based on a lightweight character recognition neural network.

[0116] In some modified implementations of the embodiments of the present application, the standard format conversion module 101 includes:

[0117] The standard format conversion unit is used to convert the image to be recognized into a standard format image that conforms to a preset color mode and / or a preset image size.

[0118] In some modified implementations of the embodiments of the present application, the character region determination module 102 includes:

[0119] The character region determining unit is used to determine the character region in the standard format image by using a lightweight text detection neural network.

[0120] In some variations of the embodiments of the present application, the lightweight text detection neural network includes:

[0121] PSENet is a progressive scaling network that uses the lightweight neural network MobileNet as the backbone network.

[0122] In some modified implementations of the embodiments of the present application, the character string extraction module 103 includes:

[0123] a character region image extraction unit, configured to extract a character region image containing the character region from the standard format image through affine transformation;

[0124] The character string extraction unit is configured to recognize a character string in the character region from the character region image based on a lightweight character recognition neural network.

[0125] In some variations of the embodiments of the present application, the lightweight character recognition neural network includes:

[0126] Lightweight Convolutional Recurrent Neural Networks.

[0127] In some modified implementations of the embodiments of the present application, the output layer of the lightweight convolutional recurrent neural network adopts local sensitive hashing (LSH) encoding.

[0128] In some modified implementations of the embodiments of the present application, the character string recognized based on the lightweight character recognition neural network includes multiple;

[0129] The device 10 further includes:

[0130] The target character string screening module is used to screen the target character string that meets the constraints of the current application scenario from the multiple character strings.

[0131] In some modified implementations of the embodiments of the present application, the target string screening module includes:

[0132] The target character string screening unit is used to adopt at least one of regular expression matching, background information matching and confidence matching to match a target character string that meets the constraints of the current application scenario from the multiple character strings.

[0133] The character recognition device 10 provided in the embodiment of the present application is based on the same inventive concept as the character recognition method provided in the aforementioned embodiment of the present application and has the same beneficial effects, which will not be described in detail here.

[0134] An embodiment of the present application also provides an electronic device corresponding to the character recognition method provided in the above embodiment. The electronic device can be any device with data processing capabilities to execute the above character recognition method.

[0135] Please refer to Figure 7 , which schematically shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 7 As shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, and the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and when the processor 200 runs the computer program, it executes the character recognition method provided in any of the aforementioned embodiments of the present application.

[0136] The memory 201 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element and at least one other network element are connected via at least one communication interface 203 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0137] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs. The processor 200 executes the programs after receiving execution instructions. The character recognition method disclosed in any of the aforementioned embodiments of the present application may be applied to the processor 200 or implemented by the processor 200.

[0138] The processor 200 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 200 or by software instructions. The above processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 201 , and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0139] The electronic device provided in the embodiment of the present application and the character recognition method provided in the aforementioned embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0140] The present application also provides a computer-readable medium corresponding to the character recognition method provided in the above embodiment. Figure 8The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the character recognition method provided by any of the aforementioned embodiments is executed.

[0141] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0142] The computer-readable storage medium provided by the above-mentioned embodiment of the present application and the character recognition method provided by the above-mentioned embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0143] An embodiment of the present application also provides an intelligent food collection cabinet corresponding to the character recognition method provided in the aforementioned embodiment, comprising: a cabinet body, and an image acquisition device and a main control device provided on the cabinet body; wherein the image acquisition device is connected to the main control device; the image acquisition device is used to capture an image of the food collection voucher to generate an image to be recognized, and send the image to be recognized to the main control device; the main control device is used to use the character recognition method provided in any of the aforementioned embodiments to recognize a character string from the image to be recognized to determine the food collection code recorded on the food collection voucher, and control the opening of the cabinet door of the designated food collection position in the cabinet body according to the food collection code.

[0144] The smart food collection cabinet provided in the above-mentioned embodiment of the present application and the character recognition method provided in the above-mentioned embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0145] It should be noted that the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0146] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0150] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and description of the present application.

Claims

1. A character recognition method, characterized in that: include: Convert the image to be recognized into a standard format image; determining a character area in the standard format image; Recognizing the character string in the character region based on a lightweight character recognition neural network, wherein the recognizing the character string in the character region based on the lightweight character recognition neural network comprises extracting a character region image containing the character region from the standard format image through an affine transformation, and recognizing the character string in the character region from the character region image based on the lightweight character recognition neural network; The lightweight character recognition neural network includes: a lightweight convolutional recursive neural network; The step of converting the image to be recognized into a standard format image includes: Convert the image to be recognized into a standard format image that conforms to the preset color mode and preset image size.

2. The method according to claim 1, characterized in that The step of converting the image to be recognized into a standard format image includes: Convert the image to be recognized into a standard format image that conforms to a preset color mode or preset image size.

3. The method according to claim 1, characterized in that Determining the character area in the standard format image includes: A lightweight text detection neural network is used to determine the character area in the standard format image.

4. The method according to claim 3, characterized in that The lightweight text detection neural network includes: PSENet is a progressive scaling network that uses the lightweight neural network MobileNet as the backbone network.

5. The method according to claim 1, wherein The output layer of the lightweight convolutional recurrent neural network adopts local sensitive hashing (LSH) encoding.

6. The method according to claim 1, characterized in that The character strings identified based on the lightweight character recognition neural network include multiple, The method further comprises: A target character string that meets the constraints of the current application scenario is obtained by screening the multiple character strings.

7. The method according to claim 6, characterized in that The step of filtering the target character string from the plurality of character strings to obtain the target character string that meets the constraints of the current application scenario includes: At least one of regular expression matching, background information matching, and confidence matching is used to match the target character string that meets the constraints of the current application scenario from the multiple character strings.

8. A character recognition device, characterized in that: include: A standard format conversion module, used for converting the image to be recognized into a standard format image; A character region determination module, configured to determine a character region in the standard format image; a character string extraction module, configured to identify a character string in the character region based on a lightweight character recognition neural network, wherein the character string extraction module extracts a character region image containing the character region from the standard format image through an affine transformation, and identifies the character string in the character region from the character region image based on the lightweight character recognition neural network; The lightweight character recognition neural network includes: a lightweight convolutional recursive neural network; The step of converting the image to be recognized into a standard format image includes: Convert the image to be recognized into a standard format image that conforms to the preset color mode and preset image size.

9. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 7.

11. An intelligent food collection cabinet, characterized in that: include: A cabinet, and an image acquisition device and a main control device arranged on the cabinet; wherein, The image acquisition device is connected to the main control device; The image acquisition device is used to capture an image of the meal voucher to generate an image to be recognized, and send the image to be recognized to the main control device; The main control device is used to use the method described in any one of claims 1 to 7 to identify the character string from the image to be identified to determine the meal pickup code recorded on the meal pickup voucher, and control the opening of the cabinet door of the designated meal pickup position in the cabinet according to the meal pickup code.

Images