Method, device, electronic device and storage medium for parsing font protocol message

Abstract

The embodiments of the present application provide a method, device, electronic device and storage medium for parsing a font protocol message. The electronic device can generate a protocol image based on the order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; intercept the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized part of the protocol image; splice the first image and the second image according to the arrangement indicated by the font protocol to obtain the image to be recognized; recognize the text in the image to be recognized and obtain the text recognition result as the parsing result of the unrecognized part. Since the protocol image includes parts corresponding to private data and public data, the electronic device can obtain the parsing result of the public data by parsing the protocol image. In this way, the public data in the font protocol message including private data can be parsed without connecting to a third-party device.

Description

Technical Field

[0001] The present application relates to the technical field of protocol parsing, and in particular to a method, device, electronic device and storage medium for parsing font protocol messages. Background Art

[0002] The font protocol is a common protocol widely used in industries like fire protection. According to this protocol, when encoding the text characters to be transmitted, the text character image corresponding to the public data to be transmitted is split into two parts, with the upper and lower parts of the text being composed in the order of the lower part of the first line, the upper part of the first line, the lower part of the second line, the upper part of the second line, etc., or the upper part of the first line, the lower part of the first line, the upper part of the second line, the lower part of the second line, etc., and private data is inserted in the middle or at both ends of the message.

[0003] Then, the upper or lower half of each character corresponding to the public data in the composed message is encoded according to the preset encoding rules based on the pixel values ​​of its pixels. Each character generally occupies 16×16 pixels. When encoding, the pixels corresponding to the upper or lower half of each character are encoded according to the preset encoding rules based on the pixel values ​​of the 8 pixels in each column.

[0004] Since the message includes private data, it needs to be connected to a third-party device to parse it, otherwise the public data will not be able to be parsed. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method, device, electronic device, and storage medium for parsing font protocol messages, so as to parse public data in font protocol messages including private data without accessing third-party devices. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for parsing a font protocol message, the method comprising:

[0007] Get the font protocol message;

[0008] Generate a protocol image according to the font protocol according to the arrangement order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte;

[0009] extracting a first image corresponding to the first byte to the second byte and a second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, wherein the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image;

[0010] splicing the first image and the second image according to the arrangement of the split text character images indicated by the font protocol to obtain an image to be recognized;

[0011] Performing text recognition on the image to be recognized to obtain a text recognition result as a parsing result of the unrecognized portion.

[0012] In a second aspect, an embodiment of the present application provides a device for parsing a font protocol message, the device comprising:

[0013] Message acquisition module, used to obtain the font protocol message;

[0014] An image generation module, configured to generate a protocol image according to the font protocol based on the order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte;

[0015] an image interception module, configured to intercept, from the unrecognized portion of the protocol image, a first image corresponding to the first byte to the second byte and a second image corresponding to the second byte to the third byte, respectively, wherein the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image;

[0016] an image stitching module, configured to stitch the first image and the second image together according to the arrangement of the split text character images indicated by the font protocol, to obtain an image to be recognized;

[0017] The text recognition module is used to perform text recognition on the image to be recognized and obtain a text recognition result as the analysis result of the unrecognized part.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0019] Memory for storing computer programs;

[0020] The processor is configured to implement any of the methods described in the first aspect above when executing a program stored in the memory.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the methods described in the first aspect above.

[0022] Beneficial effects of the embodiments of the present application:

[0023] In the solution provided by the embodiment of the present application, the electronic device can obtain a font protocol message; generate a protocol image according to the font protocol based on the order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; intercept the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized part of the protocol image, wherein the first byte and the second byte may be the same or different, and the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image; splice the first image and the second image according to the arrangement of the split text character image indicated by the font protocol to obtain the image to be recognized; perform text recognition on the image to be recognized to obtain a text recognition result as the parsing result of the unrecognized part. Since the font protocol message includes public data and private data, the protocol image generated based on the font protocol message includes both the part corresponding to the private data and the part corresponding to the public data. The electronic device can perform image interception, image splicing and text recognition on the protocol image to obtain the parsing result corresponding to the public data. In this way, even if the private data cannot be parsed, it will not affect the parsing of the public data. Therefore, the public data in the font protocol message including the private data can be parsed without accessing a third-party device.

[0024] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0026] Figure 1 A flowchart of a method for parsing a font protocol message provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of a protocol image provided in an embodiment of the present application;

[0028] Figure 3 Another schematic diagram of the protocol image provided in the embodiment of the present application;

[0029] Figure 4 Based on Figure 2 A schematic diagram of an image to be recognized of the protocol image shown;

[0030] Figure 5 Based on Figure 2Another schematic diagram of the image to be recognized of the protocol image shown;

[0031] Figure 6 Based on Figure 3 A schematic diagram of an image to be recognized of the protocol image shown;

[0032] Figure 7 Based on Figure 1 A flow chart of a method for determining the first value and the second value of the illustrated embodiment;

[0033] Figure 8 for Figure 7 A specific flow chart of step S701 in the embodiment shown;

[0034] Figure 9 Based on Figure 1 A first flow chart of a method for determining a text recognition result according to the illustrated embodiment;

[0035] Figure 10 A schematic diagram of a reference image provided in an embodiment of the present application;

[0036] Figure 11 Based on Figure 1 A first flow chart of an alternative result determination method of the illustrated embodiment;

[0037] Figure 12 Based on Figure 1 A second flow chart of a method for determining a text recognition result according to the illustrated embodiment;

[0038] Figure 13 Based on Figure 1 A schematic diagram of a first process of parsing a font protocol message according to the embodiment shown;

[0039] Figure 14 Based on Figure 1 A second flow chart of an alternative result determination method of the illustrated embodiment;

[0040] Figure 15 Based on Figure 1 A third flow chart of the method for determining the text recognition result of the illustrated embodiment;

[0041] Figure 16 Based on Figure 1 A schematic diagram of a second process for parsing a font protocol message according to the embodiment shown;

[0042] Figure 17 Based on Figure 1 A third flow chart of an alternative result determination method of the illustrated embodiment;

[0043] Figure 18 Based on Figure 1A fourth flow chart of a method for determining a text recognition result according to the illustrated embodiment;

[0044] Figure 19 Based on Figure 1 The embodiment shown is a flow chart for determining a text recognition result corresponding to an image to be recognized;

[0045] Figure 20 Based on Figure 1 A flow chart of parsing a font protocol message in the embodiment shown

[0046] Figure 21 A schematic diagram of the structure of a device for parsing a font protocol message provided in an embodiment of the present application;

[0047] Figure 22 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0049] In order to parse public data in a font protocol message including private data without accessing a third-party device, embodiments of the present application provide a font protocol message parsing method, apparatus, electronic device, computer-readable storage medium, and computer program product. The following first introduces a font protocol message parsing method provided in embodiments of the present application.

[0050] like Figure 1 As shown, a method for parsing a font protocol message, the method comprising:

[0051] S101, obtaining a font protocol message;

[0052] S102, generating a protocol image according to the font protocol according to the arrangement order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte;

[0053] S103, intercepting a first image corresponding to the first byte to the second byte and a second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image;

[0054] The first byte is the same as the second byte, and the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image.

[0055] S104, splicing the first image and the second image according to the arrangement of the split text character images indicated by the font protocol to obtain an image to be recognized;

[0056] S105 , performing text recognition on the image to be recognized to obtain a text recognition result as a result of analyzing the unrecognized portion.

[0057] It can be seen that in the embodiment of the present application, the electronic device can obtain a font protocol message; generate a protocol image according to the font protocol based on the arrangement order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; intercept the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized part of the protocol image, wherein the first byte and the second byte may be the same or different, and the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image; splice the first image and the second image according to the arrangement of the split text character image indicated by the font protocol to obtain the image to be recognized; perform text recognition on the image to be recognized to obtain a text recognition result as the parsing result of the unrecognized part. Since the font protocol message includes public data and private data, the protocol image generated based on the font protocol message includes both the part corresponding to the private data and the part corresponding to the public data. The electronic device can perform image interception, image splicing and text recognition on the protocol image to obtain the parsing result corresponding to the public data. In this way, even if the private data cannot be parsed, it will not affect the parsing of the public data. Therefore, the public data in the font protocol message including the private data can be parsed without accessing a third-party device.

[0058] The method for parsing the font protocol message provided in the embodiment of the present application can be applied to any electronic device that needs to parse the font protocol message, such as fire protection font protocol parsing equipment, early warning font protocol parsing equipment, etc., and is not specifically limited here.

[0059] Font protocol messages typically include private data and public data. The public data is encoded by splitting the text character image corresponding to the public data into two parts, with the message then being composed in the following order: upper part of the first line, lower part of the first line, upper part of the second line, lower part of the second line... or upper part of the first line, lower part of the first line, upper part of the second line, lower part of the second line...

[0060] After receiving the message, it can be encoded according to the preset encoding rules based on the pixel values ​​of each character in the public data, with a column of pixels as the unit. Typically, each character occupies 16×16 pixels, and the upper and lower halves of the character obtained by splitting the character each occupy 8×16 pixels.

[0061] Since the pixel value corresponding to each pixel can be either 0 or 1, there are a total of 2 to the power of 8, or 256 possible pixel values ​​for 8 pixels. Furthermore, the largest two-digit hexadecimal number, FF, is 255 in decimal. Counting the hexadecimal number 00, it can be concluded that a two-digit hexadecimal number can represent a maximum of 256 possible values. In this way, a correspondence between two hexadecimal numbers and the pixel values ​​of a column of 8 pixels can be pre-set. Based on this correspondence, a two-digit hexadecimal number can be used to represent the pixel values ​​of a column of 8 pixels.

[0062] In step S101, the electronic device may obtain a font protocol message. The font protocol message includes private data and public data. The font protocol message is composed of bytes, with each byte storing two hexadecimal digits. Each two hexadecimal digits stored in a byte can represent the pixel values ​​of a column of eight pixels.

[0063] For example, the font protocol message can be: 1B 4B F8 00 70F0 90 10 10 30 30 00 20 30 1010 10F0 E0 0030 60C0 80 00 00 00 00C0 E0 30 10 30E0 C0 00 20 30 10 10 10F0 E000 30 60C0 80 00 0000 00C0 E0 30 10 30E0 C0 00 00 00 00 10F0 F0 10 00 00 0000 00 00 00 00 00 00 10 10 00. The font protocol message contains 80 bytes in total. Since 16 bytes can represent a half-height character, 80 bytes can form 5 half-height characters. The font protocol message can form at most 2 complete characters.

[0064] Since the font protocol message is obtained by splitting, arranging and encoding the text character image corresponding to the public data, in order to parse the font protocol message, the electronic device can determine the pixel value of each column of pixels in the text character image indicated by each byte according to the arrangement order of the bytes included in the font protocol message and the correspondence between the bytes and the pixel values ​​of the pixels specified by the font protocol, and generate a protocol image based on the pixel value of each column of pixels, that is, execute step S102.

[0065] Since the protocol image is generated based on the pixel values ​​of 8 pixels in each column, the height of the protocol image is 8 pixels.

[0066] For example, the schematic diagram of the protocol image generated according to the font protocol message can be as follows: Figure 2 and Figure 3 shown. Figure 2 and Figure 3 The height of each is 8 pixels. Figure 2 There are five identifiers 201 in total, and the image between two adjacent identifiers 201 corresponds to the upper part or the lower part of a line of public data in the message. Figure 3 There are 2 identifiers 301 in total.

[0067] Because the font protocol message includes an identifier that marks the beginning or end of the upper or lower portion of a line, the intercepted portion can only be the upper or lower portion of a line if the initial and final bytes of the intercepted portion are identical. If the initial and final bytes of a portion are identical and both are identifiers, the portion is the upper or lower portion of a line.

[0068] In this way, the electronic device can intercept the first image corresponding to the first byte to the second byte, and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, that is, execute step S103. The first byte and the second byte can be the same or different. In the case where the first byte and the second byte are the same, it helps to improve the efficiency of image interception, and thus helps to improve the overall efficiency of protocol parsing.

[0069] Since the widths of the upper portion and the lower portion corresponding to the same line of text character images are equal, the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image.

[0070] For example, if the first byte and second byte corresponding to the first image are the second and 51st bytes in the font protocol message, respectively, then the number of bytes corresponding to the first image is 50. Since the first and second images have the same number of bytes, the number of bytes corresponding to the second image is also 50. The third byte should be the 100th byte in the font protocol message.

[0071] The unrecognized portion may be a portion of the protocol image that has not undergone text recognition. For example, if the protocol image contains content corresponding to the upper portion of the first line, the lower portion of the first line, the upper portion of the second line, the lower portion of the second line, the upper portion of the third line, and the lower portion of the third line of the message, and the upper portion of the first line and the lower portion of the first line have been recognized, then the unrecognized portion is the portion of the protocol image corresponding to the upper portion of the second line, the lower portion of the second line, the upper portion of the third line, and the lower portion of the third line.

[0072] If, during the encoding process, the split text character images are arranged in the order of the upper portion of the first row, the lower portion of the first row, the upper portion of the second row, and the lower portion of the second row, i.e., the split text character images are arranged with the upper portion of a row in front and the lower portion in the back, then during the decoding process, the electronic device can place the first image in front at the top and the second image in the back at the bottom, and splice the first and second images to obtain the image to be recognized.

[0073] Accordingly, if the split text character images are arranged in a row with the lower part in front and the upper part in the back, then during the decoding process, the electronic device can place the first image at the bottom and the second image at the top, and splice the first and second images to obtain the image to be recognized.

[0074] Based on the above description, the electronic device can splice the first image and the second image according to the arrangement of the split text character images indicated by the font protocol to obtain the image to be recognized, that is, execute step S104. Among them, the first image and the second image have the same width and height of 8 pixels.

[0075] Since the image to be identified is obtained by splicing the first image and the second image in a vertical relationship, the width of the image to be identified is the same as the width of the first image and the width of the second image, and the height of the image to be identified is twice the height of the first image and twice the height of the second image, that is, 16 pixels.

[0076] In the case where no private data is inserted into the font protocol message, the first image and the second image are spliced ​​together to obtain a schematic diagram of the image to be recognized. Figure 2 、 Figure 4 as well as Figure 5 A schematic diagram of the protocol image can be shown as Figure 2 As shown, in Figure 2, from the left, the portion between the first marker 201 and the second marker 201 is the first image 202, and the portion between the second marker 201 and the third marker 201 is the second image 203. The first image 202 and the second image 203 are spliced ​​in the order of the first image 202 at the bottom and the second image 203 at the top, and the following is obtained: Figure 4 The first image to be recognized is shown.

[0077] The portion between the third marker 201 and the fourth marker 201 is the third image 204, and the portion between the fourth marker 201 and the fifth marker 201 is the fourth image 205. The third image 204 and the fourth image 205 are spliced ​​in the order of the third image 204 at the bottom and the fourth image 205 at the top, and the following is obtained: Figure 5 The second image to be recognized is shown.

[0078] In the case of inserting private data into the font protocol message, the first image and the second image are spliced ​​together to obtain a schematic diagram of the image to be recognized. Figure 3 as well as Figure 6 A schematic diagram of the protocol image can be shown as Figure 3 As shown, in Figure 3 Counting from the left, the portion to the left of first identifier 301 is fifth image 302. The portion between first and second identifiers 301 includes both protocol images corresponding to public data and images corresponding to private data. Sixth image 303 is a protocol image corresponding to public data with private data inserted, and seventh image 304 is an image corresponding to private data.

[0079] The portion to the right of the second marker 301 is the eighth image 305. Since the sixth image 303 and the seventh image 304 are sandwiched between the two markers 301, the sixth image 303 and the seventh image 304 can be used as the first image, and the eighth image 305 can be used as the first half of the second image. In this way, the first half of the first image and the first half of the second image, that is, the sixth image 303 and the eighth image 305, can be spliced ​​in the order of the sixth image 303 at the bottom and the eighth image 305 at the top, to obtain the following: Figure 6 The third image to be recognized is shown.

[0080] Because the image to be recognized is obtained by splicing the first image and the second image, the first image and the second image may correspond to the upper and lower halves of a line of text in the message, respectively. Therefore, the image to be recognized may contain complete text content. To determine the text content contained in the image to be recognized, the electronic device may perform text recognition on the image to be recognized, thereby obtaining a text recognition result as the parsing result of the unrecognized portion, i.e., executing step S105.

[0081] For example, if the unrecognized portion corresponds to the upper portion of the third line and the lower portion of the third line of the message in the protocol image, and the text recognition result is "Shengli Building 1st Floor Hand Report", then the electronic device can determine the parsing result of the upper portion of the third line and the lower portion of the third line of the message in the protocol image as "Shengli Building 1st Floor Hand Report".

[0082] The schematic diagram of the image to be recognized can be as follows Figure 4-Figure 6 As shown, Figure 4 The first image to be identified is subjected to text recognition, and the text recognition result "23 / 03 / 04 17:50:44 2-86 manual fault" can be obtained. Figure 5 The second image to be identified is subjected to text recognition, and the text recognition result "fire hydrant on the 2nd floor of Heping Building" can be obtained. Figure 6 The third image to be recognized is subjected to text recognition, and a text recognition result “loop point extended description L1” can be obtained. Figure 6 In the third image to be recognized, private data is inserted between adjacent characters.

[0083] As can be seen, in the embodiments of the present application, since the font protocol message includes public data and private data, the protocol image generated based on the font protocol message includes both the portion corresponding to the private data and the portion corresponding to the public data. The electronic device can perform image capture, image splicing, and text recognition on the protocol image to obtain the parsing result corresponding to the public data. In this way, even if the private data cannot be parsed, it does not affect the parsing of the public data. Therefore, the public data in the font protocol message including the private data can be parsed without connecting to a third-party device.

[0084] As an implementation method of the present application, Figure 7 As shown, before the step of respectively intercepting the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, the method may further include:

[0085] S701, determining a first value and a second value according to a preset determination rule;

[0086] The first value and the second value are used to identify the byte order of bytes in the font protocol message.

[0087] To determine the first byte and the second byte in the font protocol message, the electronic device may determine the first value and the second value according to a preset determination rule, and then compare whether the byte corresponding to the first value and the byte corresponding to the second value in the font protocol message are the same. If the byte corresponding to the first value and the byte corresponding to the second value are the same, the electronic device may determine the byte corresponding to the first value and the byte corresponding to the second value as the first byte and the second byte, respectively.

[0088] For example, if the font protocol message is 1B 4B F8 00 70F0 90 10 10 30 30 00 20 30 10 101B F0 E0 0030 60C0 80 00 00 00 00C0 E0 30 10 30E0, and the first value is 1 and the second value is 5, the electronic device can determine that the byte corresponding to the first value is 1B and the byte corresponding to the second value is 70.

[0089] The preset determination rule may be to set the second value to a different value while the first value is fixed. After the second value has taken all values, the first value is changed, the first value is fixed to the changed value, and the second value is set to a different value, and this process is repeated.

[0090] For example, the font protocol message includes 100 bytes. First, the first value can be fixed to 1, and the second value can be changed from 2 to 100. Then, the first value can be fixed to 2, and the second value can be changed from 3 to 300, and so on.

[0091] S702, determining whether the byte corresponding to the first value in the font protocol message is the same as the byte corresponding to the second value; if the byte corresponding to the first value in the font protocol message is the same as the byte corresponding to the second value, executing step S703, and returning to executing step S701 until the determined first value and second value meet a preset condition; if the byte corresponding to the first value in the font protocol message is different from the byte corresponding to the second value, returning to the step of determining the first value and second value according to a preset determination rule until the determined first value and second value meet the preset condition;

[0092] The first value and the second value corresponding to the preset condition indicate that all bytes of the character module protocol message have been traversed.

[0093] Since the first byte is the same as the second byte, in order to determine the first byte and the second byte, after determining the first value and the second value, the electronic device can determine whether the byte corresponding to the first value in the font protocol message is the same as the byte corresponding to the second value.

[0094] If the byte corresponding to the first value is the same as the byte corresponding to the second value, the electronic device may determine the byte corresponding to the first value and the byte corresponding to the second value as the first byte and the second byte, respectively, and return to step S701. If the byte corresponding to the first value is different from the byte corresponding to the second value, the electronic device may directly return to step S701.

[0095] Continuing with the example in step S701, since the byte corresponding to the first value in the Zimo protocol message is different from the byte corresponding to the second value, the electronic device can return to step S701 and redetermine the first and second values ​​according to the preset determination rule. If the first and second values ​​are 1 and 17, respectively, then the byte corresponding to the first value and the byte corresponding to the second value in the Zimo protocol message are both 1B, and the electronic device can proceed to step S703.

[0096] S703, determine the byte corresponding to the first value as the first byte, determine the byte corresponding to the second value as the second byte, and execute the steps of respectively intercepting the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized part of the protocol image.

[0097] If the byte corresponding to the first value in the font protocol message is the same as the byte corresponding to the second value, it means that the byte corresponding to the first value and the byte corresponding to the second value may both be identifiers. Furthermore, the byte corresponding to the first value to the byte corresponding to the second value may be the upper half or lower half of a line in the public data.

[0098] Then, the electronic device can determine the byte corresponding to the first value as the first byte, and the byte corresponding to the second value as the second byte. And return to execute step S103, that is, intercept the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image. In this way, after splicing the first image and the second image to obtain the image to be recognized, the electronic device can perform text recognition on the image to be recognized and then determine the analysis result.

[0099] Since the first byte and the second byte are identical, the first byte and the second byte may not be identifiers. In this case, the font protocol message may not be parsed based on the image to be recognized. In order to determine all first and second values ​​that meet the condition that the byte corresponding to the first value and the byte corresponding to the second value are identical, and to successfully parse the parsing result corresponding to the unrecognized portion, the electronic device may return to step S701 until the determined first and second values ​​meet the preset condition.

[0100] To efficiently determine the first and second values, a pre-set condition can be pre-determined. The first and second values ​​corresponding to the pre-set condition indicate that all bytes of the character module protocol message have been traversed. Thus, if the pre-set condition is met, it indicates that the first and second values ​​have been traversed, and the loop to determine the first and second values ​​can be stopped.

[0101] In one embodiment, the preset condition may be: the first value is the sum of its initial value and the number of pixels corresponding to the length of a word in the text character image, and the second value is the number of bytes contained in the font protocol message.

[0102] For example, the number of bytes contained in the font protocol message is 80, the initial value of the first value is 1, and the initial value of the second value is 2. When the first value is fixed at 1, the second value can be changed from 2 to 80. Then, the first value can be updated to the current value plus 1, that is, 2. When the first value is fixed at 2, the second value can be changed from 3 to 80... and so on. If the length of a word in the text character image corresponds to the number of pixels of 16, then when the first value is 17 and the second value is 80, the preset condition is met.

[0103] In another embodiment, the preset condition may be that the number of times the first value changes reaches a preset number.

[0104] Since the identifier corresponding to the start position of a row in public data is usually located at the beginning of the message, the preset condition can be that the first value changes a preset number of times. For example, if the preset number is 17, the electronic device can terminate the loop when the first value changes 17 times. This indicates that the traversal of the cases in which the byte corresponding to the first value in the font protocol message may be the identifier has been completed.

[0105] When the byte corresponding to the first value in the font protocol message is different from the byte corresponding to the second value, the electronic device can return to the step of determining the first value and the second value according to the preset determination rule until the determined first value and second value meet the preset conditions.

[0106] If the byte corresponding to the first value is different from the byte corresponding to the second value, it means that the byte corresponding to the first value and the byte corresponding to the second value cannot both be identifiers. Furthermore, the byte corresponding to the first value to the byte corresponding to the second value cannot be the upper half or lower half of a row of public data. The electronic device can return to step S701, i.e., re-determine the first value and the second value, until the determined first value and second value meet the preset condition.

[0107] It can be seen that in the embodiment of the present application, the electronic device can determine the first value and the second value according to the preset determination rule, wherein the first value and the second value are used to identify the byte order of bytes in the font protocol message; determine whether the byte corresponding to the first value in the font protocol message is the same as the byte corresponding to the second value; if the byte corresponding to the first value is the same as the byte corresponding to the second value, determine the byte corresponding to the first value as the first byte, determine the byte corresponding to the second value as the second byte, and perform the steps of intercepting the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, respectively, and return to the step of determining the first value and the second value according to the preset determination rule until the determined first value and the second value meet the preset condition; if the byte corresponding to the first value is different from the byte corresponding to the second value, return to the step of determining the first value and the second value according to the preset determination rule until the determined first value and the second value meet the preset condition. Since the first byte and the second byte are the same, in order to determine two identical bytes in the font protocol message, the first byte and the second byte are further determined. The electronic device can determine the first value and the second value according to the preset determination rule, and compare whether the byte corresponding to the first value in the font protocol message is the same as the byte corresponding to the second value. If they are the same, the electronic device can intercept the first image corresponding to the first byte to the second byte, and the second image corresponding to the second byte to the third byte from the unidentified part of the protocol image. Since there may be other possible first bytes and second bytes, the electronic device can return to the step of determining the first value and the second value according to the preset determination rules. If they are different, it means that the first byte and the second byte cannot be identifiers at the same time, so the electronic device can directly return to the step of determining the first value and the second value according to the preset determination rules. In this way, the first value and the second value can be determined quickly and accurately, thereby improving the efficiency and accuracy of determining the first byte and the second byte.

[0108] As an implementation method of the present application, Figure 8 As shown, the step of determining the first value and the second value according to the preset determination rule may include:

[0109] S801, determining whether the byte sequence corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, and determining whether the second value is greater than the number of bytes included in the font protocol message; if the byte sequence corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, executing step S802; if the second value is greater than the number of bytes included in the font protocol message, executing step S803;

[0110] To update the first value and the second value, and thereby determine the first byte and the second byte, the electronic device may determine whether the byte sequence corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message. If the byte sequence corresponding to the third byte is less than the number of bytes included in the font protocol message, it indicates that the byte sequence corresponding to the third byte can still be increased, that is, the second value can still be increased.

[0111] Furthermore, the electronic device may determine whether the second value is greater than the number of bytes included in the font protocol message. If the second value is greater than the number of bytes included in the font protocol message, indicating that the second value cannot be further increased, the electronic device may adjust the first value so that the first value and the second value meet the limit on the number of bytes included in the font protocol message.

[0112] S802, updating the second value to the current value plus 1, and keeping the first value unchanged;

[0113] The first value and the second value have preset initial values.

[0114] When the byte sequence corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, the electronic device may update the second value to the current value plus 1 and keep the first value unchanged. The preset initial value of the first value may be 1, and the preset initial value of the second value may be 2.

[0115] For example, if the number of bytes included in the font protocol message is 50, the first value is 1, and the second value is 2, then the byte order corresponding to the third byte indicated by the first value and the second value is 3, which is less than the number of bytes included in the font protocol message. Therefore, the electronic device can keep the first value unchanged and update the second value to the current value plus 1, that is, the first value and the second value are 1 and 3 respectively.

[0116] According to the above method for updating the first and second values, the first value can be maintained at 1, and the second value can be varied from 2 to 24. There are 23 combinations of first and second values. When the second value is 25, the third value is 50. At this point, the third value is no longer less than the number of bytes included in the font protocol message.

[0117] As an implementation, the first and second values ​​can be represented by letters. Specifically, if the first value is m and the second value is n, then the byte order corresponding to the third byte is m+2n. The preset initial values ​​of m and n can be 1 and 2, respectively. When the first value is fixed, the second value can be increased in a cyclic manner to obtain more values.

[0118] For example, if the number of bytes included in the font protocol message is 50, m=1, and n=2, then, when m=1, the value of n can be 2, 3, 4, ..., 24. If n=25, then m+2n=51, and m+2n is greater than the number of bytes included in the font protocol message, which no longer satisfies the condition that the byte sequence corresponding to the third byte indicated by the first and second values ​​is less than the number of bytes included in the font protocol message.

[0119] S803: Update the first value to the current value plus 1, and update the second value to the sum of the updated first value and the preset value.

[0120] The preset value is determined based on the number of pixels corresponding to the text character image indicated by the font protocol.

[0121] If the second value is greater than the number of bytes included in the font protocol message, it means that when the first value is fixed, the second value has taken all possible values. Therefore, the electronic device can update the first value to the current value plus 1, and update the second value to the sum of the updated first value and the preset value.

[0122] Since the width of a character in a text character image is usually 16 pixels, and there is at least one character between two adjacent markers, the preset value can be 16 pixels corresponding to one character or 32 pixels corresponding to two characters, etc. Of course, the preset value can also be 0.

[0123] For example, if the number of bytes included in the font protocol message is 50, the preset value is 16, the first value is 1, and the second value is 51, this means that while the first value is fixed at 1, the second value has been increased from the initial value of 2 to 50. At this point, all possible combinations of first and second values ​​corresponding to the first value of 1 have been completed. Therefore, the electronic device can update the first value to 2 and update the second value to the sum of the updated first value and the preset value, i.e., 17.

[0124] As an embodiment, the first value and the second value can be represented by letters. Specifically, the first value is m, the second value is n, and the preset value is p. If n is greater than the number of bytes included in the word module protocol message, then m=m+1, and n=m+p.

[0125] For example, if the number of bytes in the Zimo protocol message is 50, m = 1, n = 51, and p = 16, then m = 2, n = 18. In the following loop, since the values ​​of m and n satisfy the condition that the byte order corresponding to the third byte indicated by the first and second values ​​is less than the number of bytes in the Zimo protocol message, in the following loop, m = 2, n = 18, 19, etc. are fixed.

[0126] It can be seen that in the embodiment of the present application, the electronic device can determine whether the byte order corresponding to the third byte indicated by the first and second values ​​is less than the number of bytes included in the font protocol message, and determine whether the second value is greater than the number of bytes included in the font protocol message; if the byte order corresponding to the third byte indicated by the first and second values ​​is less than the number of bytes included in the font protocol message, update the second value to the current value plus 1, and keep the first value unchanged, wherein the first and second values ​​have preset initial values; if the second value is greater than the number of bytes included in the font protocol message, update the first value to the current value plus 1, and update the second value to the sum of the updated first value and the preset value, wherein the preset value is determined based on the number of pixels corresponding to the text character image indicated by the font protocol. Since the electronic device can update the values ​​of the first and second values ​​according to the preset determination rules, and then determine the corresponding bytes from the font protocol message based on the updated first and second values, and determine whether the determined bytes are the same. Therefore, the electronic device can quickly and accurately determine the first and second values.

[0127] As an implementation of an embodiment of the present application, the update frequency of the first value may not be greater than a preset frequency, wherein the preset frequency may be determined based on the number of pixels corresponding to the text character image indicated by the font protocol.

[0128] Since the identifier corresponding to the starting position of a row in public data is usually located at the beginning of the message, the number of updates of the first value may not exceed the preset number. In other words, the termination condition of the loop of obtaining the first and second values ​​in a cyclic manner may be that the number of updates of the first value reaches the preset number.

[0129] Typically, the identifier is located within the range of the number of pixels corresponding to a word in the front portion of the message in the text character image, ie, 16 pixels, and the preset number may be 16.

[0130] For example, the preset number of times is 16, the initial values ​​of the first and second values ​​are 1 and 2 respectively, and the number of bytes contained in the font protocol message is 35. Then, the first value is fixed at 1, and the second value is increased from 2 to 36. Next, the first value is fixed at 2, and the second value is increased from 3 to 36... and so on. After each update of the first value, the first value is fixed and the second value is set to take values ​​between the updated first value and the number of bytes contained in the font protocol message. When the updated first value is 17, the number of updates to the first value reaches the preset number, i.e., 16 times. The electronic device can then stop the cycle of changing the first and second values.

[0131] It can be seen that in the embodiment of the present application, the number of updates of the first numerical value may be no more than a preset number of times. The preset number of times may be determined based on the number of pixels corresponding to the text character image indicated by the font protocol. Since the identifier corresponding to the starting position of a line in public data is usually located at the front of the message, for example, the identifier is usually located within the range of the number of pixels corresponding to a word in the text character image at the front of the message, the electronic device can set the number of updates of the first numerical value to be no more than a preset number of times. In this way, when the number of updates of the first numerical value reaches the preset number of times, the cycle of taking the first numerical value and the second numerical value will be terminated, which can improve the efficiency of determining the first quantity and the second quantity that meet the condition that the first byte and the second byte are equal.

[0132] As an implementation of an embodiment of the present application, after the step of performing text recognition on the image to be recognized and obtaining a text recognition result as the analysis result of the unrecognized portion, the method may further include:

[0133] In the case where the byte order of the third byte is less than the number of bytes included in the font protocol message, the first value is updated to the target value as the initial value, and the step of determining the first value and the second value according to the preset determination rule is returned.

[0134] The target value is determined based on the byte order of the third byte and a preset error tolerance.

[0135] Since the third byte is the last byte currently recognized in the font protocol message, if after the text recognition is performed on the image to be recognized, the byte sequence of the third byte is less than the number of bytes included in the font protocol message, it means that there are still unrecognized parts in the font protocol message. Therefore, the electronic device can update the first value to the target value, use the target value as the initial value of the first value, and return to execute the step of taking the first value and the second value through the loop.

[0136] Since the third byte is the last byte of the previous parsing, the new round of parsing should make the target value equal to the byte order of the third byte, that is, starting the parsing from the byte order indicated by the third byte. Since message loss may occur during the parsing of the font protocol message, to accommodate message loss, the electronic device can determine the target value as the difference between the target value and a preset error tolerance. In this way, in the new round of parsing, the electronic device can re-parse the preset error tolerance bytes at the end of the previous round of parsing.

[0137] For example, if the font protocol message contains 125 bytes, the byte sequence of the third byte is 63, and the preset error tolerance is 8, then the target value can be the difference between the byte sequence of the third byte and the preset error tolerance, that is, 55. In this way, the electronic device can use 55 as the initial value for the next round of parsing the first value and return to the step of determining the first and second values ​​according to the preset determination rule.

[0138] After obtaining the parsing result corresponding to the unrecognized portion, which may be the portion of the protocol image where text recognition has not been performed, a new round of parsing can be started from the beginning of the unrecognized portion by determining the target value based on the byte order of the third byte and the preset error tolerance.

[0139] For example, if the contents contained in the protocol image correspond to the upper part of the first line, the lower part of the first line, the upper part of the second line, the lower part of the second line, the upper part of the third line, and the lower part of the third line of the message, and the upper part of the first line and the lower part of the first line have been identified, then the unidentified part is the part corresponding to the upper part of the second line, the lower part of the second line, the upper part of the third line, and the lower part of the third line in the protocol image.

[0140] If the byte at the end of the lower portion of the first line, that is, the third byte, has a byte sequence of 50 and the preset error tolerance is 10, then the electronic device can set the target value as the difference between the byte sequence of the third byte and the preset error tolerance, that is, 40. In this way, a new round of parsing can start from the unrecognized portion, that is, from the position before the upper portion of the second line by the preset error tolerance.

[0141] It can be seen that in the embodiment of the present application, when the byte order of the third byte is less than the number of bytes included in the font protocol message, the first numerical value is updated to the target value as the initial value, and the step of determining the first numerical value and the second numerical value according to the preset determination rule is returned. Among them, the target value is determined based on the byte order of the third byte and the preset fault tolerance. Since the third byte is the end byte of the previous parsing, the new round of parsing should make the target value equal to the byte order of the third byte, that is, the parsing starts from the byte order indicated by the third byte. Since the message may be lost during the parsing of the font protocol message, in order to be compatible with the loss of the message, the electronic device can determine the target value as the difference between the target value and the preset fault tolerance. In this way, in the new round of parsing, the electronic device can re-parse the preset fault tolerance bytes at the end of the previous round of parsing, which can ensure that the initial value of the first numerical value is accurate and compatible in the new round of parsing.

[0142] As an implementation method of the present application, the above-mentioned image to be identified may be multiple. In this case, if Figure 9 As shown, the above step of performing text recognition on the image to be recognized and obtaining a text recognition result may include:

[0143] S901, performing text recognition on each image to be recognized using a text recognition model to obtain a recognition result corresponding to each image to be recognized and a confidence level of each text included in the recognition result;

[0144] If the first byte and the second byte corresponding to the image to be identified are the same and are both identifiers, it means that the image to be identified is composed of the upper and lower halves of the text character image corresponding to the same row of public data. Then, text recognition is performed on the image to be identified, and the text recognition result obtained is the parsing result.

[0145] When there are multiple images to be identified, in order to determine the analysis results based on the multiple images to be identified, the electronic device can perform text recognition on each image to be identified through a text recognition model to obtain the recognition result corresponding to each image to be identified and the confidence level of each text included in the recognition result.

[0146] In one embodiment, each time the electronic device obtains an image to be recognized, it can perform text recognition on the image to be recognized, obtaining a recognition result corresponding to the image to be recognized and a confidence level for each text included in the recognition result. Next, the process returns to the step of determining the first value and the second value according to a preset determination rule.

[0147] S902, determining candidate results from the recognition results based on the text included in the reference information;

[0148] The reference information is a reference image and / or a preset font library, the reference image is an image of a display interface corresponding to the font protocol message, and the preset font library is a font library for a scene corresponding to the font protocol message.

[0149] If the recognition result has a high degree of similarity to the text included in the reference information, it indicates that the text included in the recognition result is common in the scene corresponding to the font protocol message, and the recognition result can be determined to be the parsing result corresponding to the font protocol message. To determine the parsing result corresponding to the unrecognized portion from multiple recognition results, the electronic device can determine an alternative result from the recognition results based on the text included in the reference information.

[0150] The reference information includes a reference image and a preset font library. The reference image can be an image of a display interface corresponding to the font protocol message. For example, the display interface can be displayed on a display screen of an operating console. The reference image can be as follows: Figure 10 As shown, Figure 10 The image is taken of the operation console. The text content displayed on the display screen of the operation console is "First alarm: 23 / 03 / 04 17:41 3-30 Manual report Shengli Building 1st floor Manual report 0001 23 / 03 / 01 17:41 3-30 Manual report Shengli Building 1st floor Manual report Fire alarm: 0001 Linkage: 0000 Feedback: 0000", which is the text included in the reference information.

[0151] The preset font library may be a font library for the scene corresponding to the font protocol message. For example, if the scene corresponding to the font protocol message is a fire scene, the preset font library may include: fire alarm, fault, and fire hydrant.

[0152] In one embodiment, if the reference information is a reference image, the electronic device may perform text recognition on the reference image to obtain text displayed on the display interface of the reference image. The electronic device may then calculate the similarity between each recognition result and the text displayed on the display interface, and select a preset number of recognition results with the greatest similarity as candidate results.

[0153] For example, if there are five recognition results, namely recognition results 1 to recognition results 5, and the preset number is two, the similarities between recognition results 1 to recognition results 5 and the text in the display interface are 10%, 56%, 32%, 83%, and 92%, respectively. Among them, recognition results 4 and recognition results 5 are the two recognition results with the greatest similarity, so the electronic device can determine recognition results 4 and recognition results 5 as candidate results.

[0154] In another embodiment, if the reference information is a preset character library, the electronic device may calculate the number of hits between the characters included in each recognition result and the characters included in the preset character library. The hit count is the number of characters included in the recognition result that are identical to characters included in the preset character library. Furthermore, the electronic device may determine the recognition results with the largest number of hits as candidate results.

[0155] For example, if there are five recognition results, namely recognition result 1 to recognition result 5, the preset number is two. The number of hit words corresponding to recognition results 1 to recognition results 5 are 6, 2, 1, 3, and 7 respectively. Among them, recognition results 1 and recognition results 5 are the two recognition results with the largest number of corresponding hit words. Therefore, the electronic device can determine recognition results 1 and recognition results 5 as candidate results.

[0156] S903: Determine, from the candidate results, a text recognition result corresponding to the image to be recognized based on the confidence level.

[0157] Since the text confidence can reflect the degree of certainty that the characters in the image to be recognized are the recognition results, the higher the confidence, the more standard the characters in the image to be recognized. Therefore, the electronic device can determine the text recognition result corresponding to the image to be recognized from the alternative results based on the confidence.

[0158] For example, if there are two candidate results, namely candidate 1 and candidate 2. Candidate 1 includes six characters, and the corresponding confidence levels are 90%, 89%, 92%, 93%, 87%, and 94%, respectively. Candidate 2 includes five characters, and the corresponding confidence levels are 60%, 61%, 56%, 67%, 57%, and 54%, respectively. This indicates that the characters in candidate 2 are less consistent with the recognition result, so the electronic device can determine candidate 1 as the character recognition result corresponding to the image to be recognized.

[0159] As can be seen, in embodiments of the present application, an electronic device can perform text recognition on each image to be recognized using a text recognition model, obtain a recognition result corresponding to each image to be recognized and a confidence level for each text included in the recognition result; determine an alternative result from the recognition result based on text included in reference information, wherein the reference information is a reference image and / or a preset character library, the reference image is an image of a display interface corresponding to a font protocol message, and the preset character library is a character library related to the scene corresponding to the font protocol message; and determine the text recognition result corresponding to the image to be recognized from the alternative results based on the confidence level. If there are multiple images to be recognized, in order to determine a parsing result based on the multiple images to be recognized, the electronic device can perform text recognition on each image to be recognized using a text recognition model. If the recognition result has a high degree of similarity with the text included in the reference information, it indicates that the text included in the recognition result is relatively common in the scene corresponding to the font protocol message. Therefore, the electronic device can determine an alternative result from the recognition result based on the text included in the reference information. Since the text confidence level reflects the degree of certainty that the character in the image to be recognized is the recognition result, the electronic device can determine the text recognition result corresponding to the image to be recognized from the alternative results based on the confidence level. In this way, when there are multiple images to be recognized, the text recognition result corresponding to the image to be recognized can be determined quickly and accurately from the multiple images to be recognized.

[0160] As an implementation method of the embodiment of the present application, the above-mentioned reference information can be a reference image. In this case, Figure 11 As shown, the step of determining the candidate result from the recognition result based on the text included in the reference information may include:

[0161] S1101, performing text recognition on the reference image to obtain a reference result;

[0162] In the case where the reference information is a reference image, in order to determine the text information contained in the reference image, the electronic device may perform text recognition on the reference image to obtain a reference result.

[0163] S1102, calculating the similarity between each recognition result and the reference result, and determining a preset number of recognition results with the largest corresponding similarities as candidate results.

[0164] After obtaining a reference result, the electronic device may calculate the similarity between each recognition result and the reference result in order to determine, from among the multiple recognition results, a recognition result that has a higher degree of match with the text information included in the reference result. Thus, based on the similarity, the electronic device may determine the degree of match between each of the multiple recognition results and the text information included in the reference result.

[0165] After obtaining the similarity between each recognition result and the reference result, the electronic device may determine a preset number of recognition results with the greatest corresponding similarity, and use the preset number of recognition results as candidate results.

[0166] For example, if there are six recognition results, namely recognition results 1 to 6, and the preset number is three, the similarities between recognition results 1 to 6 and the reference result are 21%, 89%, 53%, 94%, 32%, and 76%, respectively. Among them, recognition results 2, 4, and 6 are the three recognition results with the greatest similarity. Therefore, the electronic device can determine recognition results 2, 4, and 6 as candidate results.

[0167] In one embodiment, Figure 12 As shown, the step of determining the text recognition result corresponding to the image to be recognized from the candidate results based on the confidence level may include:

[0168] S1201, for each candidate result, determining the confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result;

[0169] Since the text confidence can reflect the degree of certainty that the characters in the image are the recognition results, after determining a preset number of alternative results, in order to finally determine the text recognition result corresponding to the image to be recognized from the alternative results, the electronic device can determine the confidence corresponding to each alternative result based on the confidence corresponding to each text in the alternative result.

[0170] In one implementation, the electronic device may use the average of the confidences corresponding to each character in the candidate results as the confidence corresponding to the candidate result.

[0171] For example, there are three candidate results, namely, candidate 1 through candidate 3. Taking candidate 1 as an example, the process of calculating the confidence level corresponding to candidate 1 is described. If candidate 1 contains five characters, and the confidence levels corresponding to these five characters are 50%, 73%, 23%, 44%, and 80%, respectively, then the average of these five confidence levels can be calculated as 54%, and thus the confidence level corresponding to candidate 1 can be determined to be 54%.

[0172] S1202: Determine a text recognition result of the image to be recognized from the candidate results according to the confidence level corresponding to each candidate result.

[0173] After determining the confidence levels corresponding to a preset number of candidate results, the electronic device may determine the text recognition result of the image to be recognized from the candidate results according to the confidence levels corresponding to the candidate results.

[0174] In one embodiment, the electronic device may select the candidate result with the highest confidence from among the candidate results as the text recognition result of the image to be recognized.

[0175] Continuing with the example in step S1201, the confidence level corresponding to alternative result 1 is 54%. If the confidence levels corresponding to alternative result 2 and alternative result 3 are 78% and 43% respectively, then the electronic device can determine the alternative result with the highest confidence level, i.e., alternative result 2, as the text recognition result of the image to be recognized.

[0176] As an implementation method, a flow chart of parsing a font protocol message using a reference image as reference information can be shown as follows: Figure 13 As shown, the following steps may be specifically included:

[0177] S1301, generating an agreement image from the font agreement text;

[0178] The font protocol text 1311 may be a font protocol message. To parse the font protocol text 1311, the electronic device may generate a protocol image based on the pixel values ​​of a column of eight pixels indicated by the two hexadecimal digits stored in each byte of the font protocol text 1311, and the preset correspondence between the two hexadecimal digits and the pixel values ​​of the eight pixels.

[0179] S1302, segmenting and combining the protocol images;

[0180] Because the public data in the font protocol text 1311 is encoded in the order of the upper portion of the first line, the lower portion of the first line, the upper portion of the second line, the lower portion of the second line, etc., or the lower portion of the first line, the upper portion of the first line, the lower portion of the second line, the upper portion of the second line, etc., the electronic device can segment the protocol image and combine the segmented images according to the arrangement specified by the font protocol text 1311 to obtain multiple images to be recognized.

[0181] S1303, performing text recognition on the image to be recognized;

[0182] After obtaining a plurality of images to be recognized, in order to determine the text information contained in each image to be recognized, the electronic device may perform text recognition on each image to be recognized, and then obtain a recognition result corresponding to each image to be recognized.

[0183] S1304, performing text recognition on the reference image;

[0184] The electronic device may perform text recognition on the reference image 1312 to obtain reference characters contained in the reference image 1312. The reference characters may be one line or multiple lines.

[0185] S1305, performing similarity scoring on multiple recognition results;

[0186] The electronic device can calculate the similarity between each recognition result and the reference character, and then perform a similarity score on each recognition result.

[0187] S1306, selecting the candidate result with the greatest similarity and combining the corresponding texts;

[0188] The electronic device may determine the candidate result with the greatest similarity from the multiple candidate results, and then combine the text contained in the candidate result.

[0189] S1307, determining a character string corresponding to the true meaning of the font protocol text;

[0190] After combining the characters included in the candidate results with the greatest similarity, the electronic device may use the combined characters as a character string corresponding to the true meaning of the font protocol text.

[0191] S1308, wrap and regenerate the protocol image.

[0192] Since the font protocol text 1311 may contain multiple lines of text, after determining the character string corresponding to the current line, the electronic device may wrap the line and generate the protocol image corresponding to the next line. Next, the electronic device may return to step S1302 to perform subsequent processing on the protocol image corresponding to the next line.

[0193] As can be seen, in an embodiment of the present application, an electronic device can perform text recognition on a reference image to obtain a reference result; calculate the similarity between each recognition result and the reference result, and determine a preset number of recognition results with the greatest similarity as candidate results; for each candidate result, determine the confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result; and determine the text recognition result for the image to be recognized from each candidate result based on the confidence level corresponding to each candidate result. Because the reference image may contain text information encoded by a font protocol message, the electronic device can perform text recognition on the reference image and calculate the similarity between each recognition result and the reference result. If the similarity is high, it indicates that the text information contained in the recognition result is likely text information encoded by a font protocol message, and the electronic device can select the preset number of reference results with the greatest similarity as candidate results. Furthermore, the confidence level corresponding to each candidate result is calculated, and based on the confidence level corresponding to each candidate result, the text recognition result for the image to be recognized is determined. In this way, the text recognition result corresponding to the image to be recognized can be quickly and accurately determined from multiple candidate results.

[0194] As an implementation method of the present application, the above reference information may be a preset character library. Figure 14 As shown, the step of determining the candidate result from the recognition result based on the text included in the reference information may include:

[0195] S1401, calculating the number of hits between the characters included in each recognition result and the characters included in the preset character library;

[0196] The number of hit characters is the number of characters in the recognition result that are identical to characters in the preset character library. A high number of hit characters indicates that the scene corresponding to the font protocol message appearing in the recognition result contains a large number of characters, and the recognition result is likely to be the text recognition result corresponding to the image to be recognized. Therefore, the electronic device can determine the number of hit characters between each recognition result and the characters in the preset character library.

[0197] S1402: Determine the recognition result with the largest number of hit words as the candidate result.

[0198] Since the recognition result with a higher number of hit words may be the text recognition result corresponding to the font protocol message, the electronic device may determine the recognition result with the largest number of hit words as the candidate result.

[0199] If there are multiple recognition results with the most hit words, then the number of candidate results is multiple. If there is only one recognition result with the most hit words, then the number of candidate results is only one.

[0200] In one embodiment, Figure 15 As shown, the step of determining the text recognition result corresponding to the image to be recognized from the candidate results based on the confidence level may include:

[0201] S1501, determine the number of candidate results; if the number of candidate results is one, execute step S1502; if the number of candidate results is multiple, execute step S1503;

[0202] If there are multiple candidate results, the electronic device can determine the text recognition result corresponding to the image to be recognized from the multiple candidate results. If there is only one candidate result, the electronic device can use the candidate result as the text recognition result corresponding to the image to be recognized. Therefore, the electronic device can determine the number of candidate results.

[0203] S1502, determining the text recognition result of the image to be recognized by using the candidate results;

[0204] When the number of candidate results is one, the electronic device may determine the candidate result as the text recognition result corresponding to the image to be recognized.

[0205] S1503, for each candidate result, determining the confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result;

[0206] Since the text confidence can reflect the degree of certainty that the characters in the image are the recognition results, when there are multiple alternative results, in order to finally determine the text recognition result corresponding to the image to be recognized from the alternative results, the electronic device can determine the confidence corresponding to each alternative result based on the confidence corresponding to each text in the alternative result.

[0207] S1504: Determine a text recognition result of the image to be recognized from the candidate results according to the confidence level corresponding to each candidate result.

[0208] After determining the confidence levels corresponding to a preset number of candidate results, the electronic device can determine, based on the confidence levels corresponding to the candidate results, a text recognition result for the image to be recognized from the candidate results. In one embodiment, the electronic device can select the candidate result with the highest confidence level from the candidate results as the text recognition result for the image to be recognized.

[0209] As an implementation method, the flow chart of parsing the font protocol message with the preset font library as reference information can be as follows: Figure 16 As shown, the following steps may be specifically included:

[0210] S1601, generating an agreement image from the font agreement text;

[0211] The font protocol text 1611 may be a font protocol message. To parse the font protocol text 1611, the electronic device may generate a protocol image based on the pixel values ​​of a column of eight pixels indicated by the two hexadecimal digits stored in each byte of the font protocol text 1611, and the preset correspondence between the two hexadecimal digits and the pixel values ​​of the eight pixels.

[0212] The font agreement text 1611 includes public data and private data. That is, even if the font agreement text 1611 contains private data of three parties, the electronic device will generate an agreement image based on the private data.

[0213] S1602, segmenting and combining the protocol images;

[0214] Because the encoding method corresponding to the public data in the font protocol text 1611 is the sequence of the upper portion of the first line, the lower portion of the first line, the upper portion of the second line, the lower portion of the second line, etc., or the sequence of the lower portion of the first line, the upper portion of the first line, the lower portion of the second line, the upper portion of the second line, etc., the electronic device can segment the protocol image and combine the segmented images according to the arrangement specified by the font protocol text 1611 to obtain multiple images to be recognized.

[0215] S1603, performing text recognition on the image to be recognized;

[0216] After obtaining a plurality of images to be recognized, in order to determine the text information contained in each image to be recognized, the electronic device may perform text recognition on each image to be recognized, and then obtain a recognition result corresponding to each image to be recognized.

[0217] S1604, scoring the number of hit words for the multiple recognition results based on the preset character library;

[0218] The number of hit characters is the number of characters in the recognition result that are identical to characters in the preset character library 1612. A high number of hit characters indicates that the scene corresponding to the font protocol message appearing in the recognition result contains a large number of characters, and the recognition result is likely to be the text recognition result corresponding to the image to be recognized. Therefore, the electronic device can determine the number of hit characters between each recognition result and the characters in the preset character library 1612.

[0219] S1605, selecting the recognition result with the largest number of hit words and combining the corresponding words;

[0220] The electronic device may determine the recognition result with the largest number of hit characters from the plurality of recognition results, and then combine the characters included in the recognition result.

[0221] S1606, determining a character string corresponding to the true meaning of the font protocol text;

[0222] After combining the characters included in the candidate results with the greatest similarity, the electronic device may use the combined characters as a character string corresponding to the true meaning of the font protocol text.

[0223] S1607, wrap and regenerate the protocol image.

[0224] Since the font protocol text 1611 may contain multiple lines of text, after determining the character string corresponding to the current line, the electronic device may wrap the line and generate the protocol image corresponding to the next line. Next, the electronic device may return to step S1602 to perform subsequent processing on the protocol image corresponding to the next line.

[0225] As can be seen, in an embodiment of the present application, the electronic device can calculate the number of hits between the text included in each recognition result and the text included in the preset character library; determine the recognition result with the largest number of hits as an alternative result; if there is one alternative result, determine the alternative result as the text recognition result of the image to be recognized; if there are multiple alternative results, for each alternative result, determine the confidence level corresponding to each text in the alternative result based on the confidence level corresponding to the alternative result; and determine the text recognition result of the image to be recognized from each alternative result based on the confidence level corresponding to each alternative result. If the number of hits is large, it means that there are many texts in the scene corresponding to the font protocol message appearing in the recognition result, and the recognition result is more likely to be the text recognition result corresponding to the image to be recognized. Therefore, the electronic device can determine the number of hits between the text included in each recognition result and the text included in the preset character library. Furthermore, the recognition result with the largest number of hits is determined as the alternative result. If there is only one alternative result, the electronic device can determine the alternative result as the text recognition result corresponding to the image to be recognized. If there are multiple candidate results, in order to determine the text recognition result corresponding to the image to be recognized from the candidate results, the electronic device can determine the confidence level corresponding to each candidate result based on the confidence level corresponding to each character in the candidate result. In this way, the text recognition result corresponding to the image to be recognized can be determined quickly and accurately from the multiple candidate results.

[0226] As an implementation method of the embodiment of the present application, the above-mentioned reference information can be a reference image and a preset font library. In this case, Figure 17 As shown, the step of determining the candidate result from the recognition result based on the text included in the reference information may include:

[0227] S1701, performing text recognition on the reference image to obtain a reference result;

[0228] In the case where the reference information is a reference image, in order to determine the text information contained in the reference image, the electronic device may perform text recognition on the reference image to obtain a reference result.

[0229] S1702, calculating the similarity between each recognition result and the reference result, and determining a preset number of recognition results with the largest corresponding similarities as first candidate results;

[0230] After obtaining a reference result, the electronic device may calculate the similarity between each recognition result and the reference result in order to determine, from among the multiple recognition results, a recognition result that has a higher degree of match with the text information included in the reference result. Thus, based on the similarity, the electronic device may determine the degree of match between each of the multiple recognition results and the text information included in the reference result.

[0231] After obtaining the similarity between each recognition result and the reference result, the electronic device may determine a preset number of recognition results with the greatest corresponding similarity, and use the preset number of recognition results as first candidate results.

[0232] S1703, calculating the number of hits between the characters included in each first candidate result and the characters included in the preset character library;

[0233] If there are a large number of matching characters, it indicates that the scene corresponding to the font protocol message appearing in the recognition result contains a large number of characters, and the recognition result is likely to be the character recognition result corresponding to the image to be recognized. Therefore, the electronic device can determine the number of matching characters between the characters included in each first candidate result and the characters included in the preset font library.

[0234] S1704: Determine the first candidate result with the largest number of matching words as the second candidate result.

[0235] Since the recognition result with a higher number of hit words may be the text recognition result corresponding to the font protocol message, the electronic device may determine the first candidate result with the largest number of hit words as the second candidate result.

[0236] If there are multiple first candidate results with the most hit words, then there are multiple second candidate results. If there is one first candidate result with the most hit words, then there is one second candidate result.

[0237] In one embodiment, Figure 18 As shown, the step of determining the text recognition result corresponding to the image to be recognized from the candidate results based on the confidence level may include:

[0238] S1801, determining the number of the second candidate results; if the number of the second candidate results is one, executing step S1802; if the number of the second candidate results is multiple, executing step S1803;

[0239] If there are multiple second candidate results, the electronic device can determine the text recognition result corresponding to the image to be recognized from the multiple second candidate results. If there is only one second candidate result, the electronic device can use the second candidate result as the text recognition result corresponding to the image to be recognized. Therefore, the electronic device can determine the number of second candidate results.

[0240] S1802, determining the text recognition result of the image to be recognized by using the second candidate result;

[0241] When the number of the second candidate result is one, the electronic device may determine the second candidate result as the text recognition result corresponding to the image to be recognized.

[0242] When the number of the second candidate result is one, the electronic device may determine the second candidate result as the text recognition result corresponding to the image to be recognized.

[0243] S1803, for each second candidate result, determining the confidence level corresponding to the second candidate result based on the confidence level corresponding to each character in the second candidate result;

[0244] Since the text confidence can reflect the degree of certainty that the characters in the image are the recognition results, when there are multiple second alternative results, in order to finally determine the text recognition result corresponding to the image to be recognized from the second alternative results, the electronic device can determine the confidence corresponding to each second alternative result based on the confidence corresponding to each text in the second alternative result.

[0245] S1804 : Determine a text recognition result of the image to be recognized from the second candidate results according to the confidence level corresponding to each second candidate result.

[0246] After determining the confidence levels corresponding to a preset number of second candidate results, the electronic device can determine, based on the confidence levels corresponding to the second candidate results, a text recognition result for the image to be recognized from the second candidate results. In one embodiment, the electronic device can select the second candidate result with the highest confidence level from the second candidate results as the text recognition result for the image to be recognized.

[0247] In one embodiment, the flowchart for determining the text recognition result corresponding to the image to be recognized using the reference image and the preset character library as reference information can be as follows: Figure 19 As shown, the following steps may be specifically included:

[0248] S1901, performing text recognition on a reference image to obtain a text recognition result corresponding to the reference image;

[0249] The electronic device can obtain reference image 1911 and perform text recognition on the reference image 1911. The text recognition result obtained includes words that appear frequently in the scene corresponding to the font protocol message. In this way, the text recognition result corresponding to the image to be recognized can be determined based on the text recognition result corresponding to the reference image 1911.

[0250] Specifically, the electronic device may use RCNN (Region-based Convolutional Neural Network) to determine the region containing text information in the reference image, and then perform TCT (Text Character Recognition) on the region containing text information.

[0251] S1902, performing text recognition on each image to be recognized using a text recognition model to obtain a recognition result corresponding to each image to be recognized and a confidence level of each text included in the recognition result;

[0252] After obtaining multiple images to be recognized, the electronic device can use RCNN to determine the areas in each image that contain text information, and then use the text recognition model to perform TCT on the areas containing text information. In this way, the recognition result corresponding to each image to be recognized and the confidence level of each text included in the recognition result can be obtained.

[0253] S1903, performing similarity scoring on the multiple recognition results based on the similarity between the recognition results and the text recognition results corresponding to the reference image, and determining candidate results based on the similarity scores;

[0254] The electronic device can calculate the similarity between each recognition result and the text recognition result corresponding to the reference image 1911, and then assign a similarity score to each recognition result. Then, a preset number of recognition results with the highest similarity are selected as candidate results.

[0255] S1904, determining the number of hits between the characters included in each candidate result and the characters included in the preset character library, and calculating the average confidence value of the number of hits corresponding to each candidate result;

[0256] The number of hits is the number of characters in the candidate results that are identical to characters in the preset character library 1912. A high number of hits indicates that the scene corresponding to the font protocol message that appears in the candidate result contains a large number of characters, and the candidate result is likely to be a candidate for the characters corresponding to the image to be recognized. Therefore, the electronic device can determine the number of hits between the characters in each candidate result and the characters in the preset character library 1912.

[0257] Then, the electronic device can use the confidence mean corresponding to each character output by the character recognition model as the confidence corresponding to the candidate result.

[0258] S1905: Using the candidate result with the highest corresponding confidence as the text recognition result corresponding to the image to be recognized.

[0259] After determining the confidence levels corresponding to a preset number of candidate results, the electronic device may determine the text recognition result of the image to be recognized from the candidate results according to the confidence levels corresponding to the candidate results.

[0260] In one embodiment, after determining the candidate result with the highest confidence, the electronic device can convert the candidate result into UTF8 (Unicode Transformation Format-8, 8-bit variable length encoding) code so that the electronic device can store the candidate result. In this way, the text recognition result of the image to be recognized is also in the format of UTF8 code.

[0261] It can be seen that in the embodiment of the present application, the electronic device can perform text recognition on the reference image to obtain a reference result; calculate the similarity between each recognition result and the reference result, and determine a preset number of recognition results with the largest similarity as the first candidate result; calculate the number of hits between the text included in each first candidate result and the text included in the preset character library; determine the first candidate result with the largest number of hits as the second candidate result; if the number of second candidate results is one, determine the text recognition result of the image to be recognized by the second candidate result; if the number of second candidate results is multiple, for each second candidate result, based on the confidence corresponding to each character in the second candidate result, determine the confidence corresponding to the second candidate result; according to the confidence corresponding to each second candidate result, determine the text recognition result of the image to be recognized from each second candidate result. Since the reference image may contain text information encoded by the font protocol message, the electronic device can perform text recognition on the reference image and then calculate the similarity between each recognition result and the reference result. If the similarity is high, it indicates that the text information contained in the recognition result is likely the text information encoded in the font protocol message. The electronic device then selects a preset number of reference results with the greatest similarity as the first candidate results. The electronic device then determines the number of hits between the text included in each first candidate result and the text included in the preset font library. Furthermore, the recognition result with the largest number of hits is determined as the second candidate result. In this way, the text recognition result corresponding to the image to be recognized can be quickly and accurately determined from multiple candidate results.

[0262] As an implementation method, a flow chart of electronic equipment parsing a font protocol message can be as follows: Figure 20 As shown, the following steps may be specifically included:

[0263] S2001, obtaining a font protocol message;

[0264] The electronic device may obtain the font protocol message to parse the font protocol message.

[0265] S2002, generating a protocol image by taking one byte in the font protocol message as a column of pixels;

[0266] The height of the protocol image is 8 pixels.

[0267] The electronic device can generate a protocol image according to the pixel values ​​of a column of 8 pixels corresponding to the two hexadecimal digits stored in each byte in the font protocol message. Since the component unit of the protocol image is a column of 8 pixels, the height of the protocol image is 8 pixels.

[0268] S2003, if m+2n is less than the length of the font protocol message, keep the first value m unchanged and update the second value n to the current value plus 1; if the second value n is greater than the length of the font protocol message, update the second value n to n+16 and update the first value m to the current value plus 1;

[0269] The preset initial values ​​of the first value m and the second value n are 1 and 17 respectively, and the number of updates of the first value m is no more than 16 times.

[0270] If m+2n is less than the length of the word module protocol message, it means that the value of n can continue to increase. In this case, the value of m can be kept unchanged and the value of n can be updated to the current value plus 1.

[0271] If the second value n is greater than the length of the font protocol message, it indicates that n is too large to correspond to a byte in the font protocol message. Therefore, the value of m can be updated to the current value plus 1, and the value of n can be updated to m+16. Since the width of a character in a text character image is 16 pixels, if the first and second bytes corresponding to m and n are both identifiers, then the distance between m and n is at least the width of a character, that is, 16 pixels. Therefore, updating the value of n to m+16 can improve the efficiency of determining the first and second bytes.

[0272] S2004, taking the mth byte and the nth byte in the font protocol message;

[0273] In order to determine whether the mth byte and the nth byte in the font protocol message are the same, and then determine the mth byte and the nth byte that may be identifiers at the same time, the electronic device can take the mth byte and the nth byte in the font protocol message.

[0274] S2005, determine whether the mth byte and the nth byte are the same; if they are the same, execute step S2006; if they are different, return to execute step S2003;

[0275] In order to determine that the mth byte and the nth byte may both be identifiers, the electronic device may determine whether the mth byte and the nth byte are the same.

[0276] S2006, intercepting the first image corresponding to the mth byte to the nth byte and the second image corresponding to the nth byte to the 2n-mth byte from the unrecognized portion of the protocol image;

[0277] If the mth byte and the nth byte are the same, the electronic device may intercept the first image corresponding to the mth byte to the nth byte and the second image corresponding to the nth byte to the 2n-mth byte from the protocol image. The first image and the second image have the same width and are both 8 pixels high.

[0278] S2007, according to the arrangement of the split text character images indicated by the font protocol, the first image and the second image are spliced ​​together to obtain an image to be recognized;

[0279] The electronic device may splice the first image and the second image according to the arrangement of the split text character images indicated by the font protocol to obtain the image to be recognized. For example, the arrangement may be the first image on top and the second image on the bottom. Alternatively, the arrangement may be the first image on the bottom and the second image on the bottom.

[0280] The height of the spliced ​​image to be recognized is 16 pixels.

[0281] S2008, determining a text recognition result corresponding to the image to be recognized based on the text information contained in the image to be recognized, and using the text recognition result as the parsing result of the kth row;

[0282] Here, the initial value of k is 1.

[0283] The electronic device can determine the text recognition result corresponding to the image to be recognized based on the text information contained in the image to be recognized. Since the font protocol message may contain multiple lines of data, the electronic device can use the text recognition result as the parsing result of the kth line.

[0284] S2009, determine whether m+2n+32 is less than the width of the protocol image; if yes, execute step S2010; if not, execute step S2011;

[0285] In order to determine whether the font protocol message has been parsed, the electronic device may determine whether m+2n+32 is smaller than the width of the protocol image.

[0286] S2010, update the first value m to m=m+2n-8, and update k to the current value plus 1.

[0287] After determining the parsing result for row k, the electronic device can update the value of m to m+2n-8. That is, starting from the end of row k, the character protocol message corresponding to row k+1 is parsed. Here, 8 is the error tolerance. The electronic device can then return to step S2003.

[0288] S2011, parsing completed.

[0289] If m+2n+32 is greater than the width of the protocol image, this means that in the current parsing round, the distance between the last byte in the second image and the last byte in the font protocol message is less than 32 pixels, that is, the pixel width occupied by the two characters in the character image. In this case, the number of remaining bytes is less than 32, and the font protocol message can be considered parsed.

[0290] It should be noted that in the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0291] Corresponding to the above-mentioned method for parsing the font protocol message, an embodiment of the present application further provides a device for parsing the font protocol message. The device for parsing the font protocol message provided in the embodiment of the present application is introduced below.

[0292] like Figure 21 As shown, a parsing device for a font protocol message, the device comprising:

[0293] The message acquisition module 2101 is used to obtain the font protocol message;

[0294] An image generation module 2102 is configured to generate a protocol image according to the font protocol based on the order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte;

[0295] An image interception module 2103 is configured to intercept, from the unrecognized portion of the protocol image, a first image corresponding to a first byte to a second byte and a second image corresponding to a second byte to a third byte, respectively, wherein the first byte is the same as the second byte, and the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image;

[0296] An image stitching module 2104 is configured to stitch the first image and the second image together according to the arrangement of the split text character images indicated by the font protocol to obtain an image to be recognized;

[0297] The text recognition module 2105 is used to perform text recognition on the image to be recognized and obtain a text recognition result as the analysis result of the unrecognized part.

[0298] It can be seen that in the embodiment of the present application, the electronic device can obtain a font protocol message; generate a protocol image according to the font protocol based on the arrangement order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; intercept the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized part of the protocol image, wherein the first byte is the same as the second byte, and the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image; splice the first image and the second image according to the arrangement of the split text character image indicated by the font protocol to obtain the image to be recognized; perform text recognition on the image to be recognized to obtain a text recognition result as the parsing result of the unrecognized part. Since the font protocol message includes public data and private data, the protocol image generated based on the font protocol message includes both the part corresponding to the private data and the part corresponding to the public data. The electronic device can perform image interception, image splicing and text recognition on the protocol image to obtain the parsing result corresponding to the public data. In this way, even if the private data cannot be parsed, it will not affect the parsing of the public data. Therefore, the public data in the font protocol message including the private data can be parsed without accessing a third-party device.

[0299] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0300] a value determination module, configured to determine a first value and a second value according to a preset determination rule, wherein the first value and the second value are used to identify a byte order of bytes in the font protocol message;

[0301] a byte determination module, configured to determine whether the byte corresponding to the first value and the byte corresponding to the second value in the font protocol message are the same;

[0302] a byte determination module, configured to, if the byte corresponding to the first value is the same as the byte corresponding to the second value, determine the byte corresponding to the first value as the first byte, determine the byte corresponding to the second value as the second byte, execute the steps of respectively intercepting the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, and return to the step of determining the first value and the second value according to the preset determination rule until the determined first value and second value meet a preset condition, wherein the first value and second value corresponding to the preset condition indicate that all bytes of the font protocol message have been traversed;

[0303] The return module is used to return to the step of determining the first value and the second value according to the preset determination rule if the byte corresponding to the first value is different from the byte corresponding to the second value, until the determined first value and second value meet the preset condition.

[0304] As an implementation of an embodiment of the present application, the above-mentioned value determination module may include:

[0305] a quantity determination submodule, configured to determine whether a byte sequence corresponding to a third byte indicated by the first and second values ​​is less than the number of bytes included in the font protocol message, and to determine whether the second value is greater than the number of bytes included in the font protocol message;

[0306] a second value updating submodule, configured to update the second value to a current value plus 1, and keep the first value unchanged, if the byte order corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, wherein the first value and the second value have preset initial values;

[0307] The first numerical value updating submodule is used to update the first numerical value to the current value plus 1, and update the second numerical value to the sum of the updated first numerical value and a preset value if the second numerical value is greater than the number of bytes included in the font protocol message, wherein the preset value is determined based on the number of pixels corresponding to the text character image indicated by the font protocol.

[0308] As an implementation of an embodiment of the present application, the update frequency of the above-mentioned first value is no more than a preset frequency, wherein the preset frequency is determined based on the number of pixels corresponding to the text character image indicated by the font protocol.

[0309] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0310] A target value update module is used to update the first value to a target value as an initial value when the byte order of the third byte is less than the number of bytes included in the font protocol message, and return to the step of determining the first value and the second value according to a preset determination rule, wherein the target value is determined based on the byte order of the third byte and a preset error tolerance.

[0311] As an implementation method of the embodiment of the present application, there are multiple images to be recognized;

[0312] The text recognition module 2105 may include:

[0313] A confidence determination submodule, configured to perform text recognition on each image to be recognized using a text recognition model, and obtain a recognition result corresponding to each image to be recognized and a confidence level of each text included in the recognition result;

[0314] an alternative result determination submodule, configured to determine an alternative result from the recognition result based on text included in reference information, wherein the reference information is a reference image and / or a preset font library, the reference image is an image of a display interface corresponding to the font protocol message, and the preset font library is a font library for the scene corresponding to the font protocol message;

[0315] The recognition result determination submodule is used to determine the text recognition result corresponding to the image to be recognized from the candidate results according to the confidence level.

[0316] As an implementation manner of the embodiment of the present application, the above-mentioned reference information is a reference image;

[0317] The above-mentioned alternative result determination submodule may include:

[0318] A reference result acquisition unit, configured to perform text recognition on the reference image to obtain a reference result;

[0319] A first candidate result determination unit is configured to calculate the similarity between each recognition result and the reference result, and determine a preset number of recognition results with the largest corresponding similarities as candidate results;

[0320] The above-mentioned recognition result determination submodule may include:

[0321] a first confidence determination unit, configured to determine, for each candidate result, a confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result;

[0322] The first recognition result determination unit is configured to determine a text recognition result of the image to be recognized from the candidate results according to the confidence level corresponding to each candidate result.

[0323] As an implementation method of the embodiment of the present application, the above-mentioned reference information is a preset font library;

[0324] The above-mentioned alternative result determination submodule may include:

[0325] A first hit word count calculation unit, configured to calculate the hit word count between the characters included in each recognition result and the characters included in the preset character library;

[0326] A second candidate result determination unit is used to determine the corresponding recognition result with the largest number of hit words as the candidate result;

[0327] The above-mentioned recognition result determination submodule may include:

[0328] a first result determination unit, configured to determine the text recognition result of the image to be recognized by using the candidate result if the number of the candidate result is one;

[0329] a confidence determination unit, configured to determine, for each candidate result, a confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result, if there are multiple candidate results;

[0330] The second result determination unit is configured to determine the text recognition result of the image to be recognized from the candidate results according to the confidence level corresponding to each candidate result.

[0331] As an implementation method of the embodiment of the present application, the reference information may be a reference image and a preset font library;

[0332] The above-mentioned alternative result determination submodule may include:

[0333] A reference result acquisition unit, configured to perform text recognition on the reference image to obtain a reference result;

[0334] a third candidate result determination unit, configured to calculate the similarity between each recognition result and the reference result, and determine a preset number of recognition results with the largest corresponding similarities as first candidate results;

[0335] A second hit word number calculation unit, used for calculating the hit word number between the characters included in each first candidate result and the characters included in the preset character library;

[0336] A fourth candidate result determination unit, configured to determine the first candidate result with the largest number of hit words as the second candidate result;

[0337] The above-mentioned recognition result determination submodule may include:

[0338] a second recognition result determining unit, configured to determine the text recognition result of the image to be recognized using the second candidate result if the number of the second candidate result is one;

[0339] an alternative result confidence determination unit, configured to determine, for each second alternative result, a confidence corresponding to the second alternative result based on the confidence corresponding to each character in the second alternative result, if there are multiple second alternative results;

[0340] The third recognition result determination unit is configured to determine the text recognition result of the image to be recognized from the second candidate results according to the confidence level corresponding to each second candidate result.

[0341] The present application also provides an electronic device, such as Figure 22 As shown, including:

[0342] Memory 2201, used for storing computer programs;

[0343] The processor 2202 is configured to implement the method for parsing the font protocol message described in any of the above embodiments when executing the program stored in the memory 2201.

[0344] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 2202, the communication interface, and the memory 2201 communicate with each other via the communication bus.

[0345] It can be seen that in the embodiment of the present application, the electronic device can obtain a font protocol message; generate a protocol image according to the font protocol based on the arrangement order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; intercept the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized part of the protocol image, wherein the first byte is the same as the second byte, and the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image; splice the first image and the second image according to the arrangement of the split text character image indicated by the font protocol to obtain the image to be recognized; perform text recognition on the image to be recognized to obtain a text recognition result as the parsing result of the unrecognized part. Since the font protocol message includes public data and private data, the protocol image generated based on the font protocol message includes both the part corresponding to the private data and the part corresponding to the public data. The electronic device can perform image interception, image splicing and text recognition on the protocol image to obtain the parsing result corresponding to the public data. In this way, even if the private data cannot be parsed, it will not affect the parsing of the public data. Therefore, the public data in the font protocol message including the private data can be parsed without accessing a third-party device.

[0346] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0347] The communication interface is used for communication between the above electronic device and other devices.

[0348] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0349] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0350] In another embodiment provided by the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for parsing any of the font protocol messages are implemented.

[0351] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the method for parsing the font protocol message of any one of the above embodiments.

[0352] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0353] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0354] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0355] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A method for parsing a font protocol message, characterized in that: The method comprises: Get the font protocol message; Generate a protocol image according to the font protocol according to the arrangement order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; extracting a first image corresponding to the first byte to the second byte and a second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, wherein the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image; the first byte and the second byte are two bytes in different positions included in the font protocol message, and the first byte and the second byte are the same; splicing the first image and the second image according to the arrangement of the split text character images indicated by the font protocol to obtain an image to be recognized; Performing text recognition on the image to be recognized to obtain a text recognition result as a parsing result of the unrecognized portion.

2. The method according to claim 1, characterized in that Before the step of respectively intercepting a first image corresponding to the first byte to the second byte and a second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, the method further includes: Determine a first value and a second value according to a preset determination rule, wherein the first value and the second value are used to identify a byte order of bytes in the font protocol message; determining whether a byte corresponding to the first value and a byte corresponding to the second value in the font protocol message are the same; If the byte corresponding to the first value is the same as the byte corresponding to the second value, the byte corresponding to the first value is determined as the first byte, the byte corresponding to the second value is determined as the second byte, and the steps of intercepting the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image are performed, and the process returns to the step of determining the first value and the second value according to the preset determination rule until the determined first value and the second value meet a preset condition, wherein the first value and the second value corresponding to the preset condition indicate that all bytes of the font protocol message have been traversed. If the byte corresponding to the first value is different from the byte corresponding to the second value, return to the step of determining the first value and the second value according to the preset determination rule until the determined first value and second value meet the preset condition.

3. The method according to claim 2, characterized in that The step of determining the first value and the second value according to a preset determination rule includes: determining whether a byte sequence corresponding to a third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, and determining whether the second value is greater than the number of bytes included in the font protocol message; If the byte order corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, update the second value to the current value plus 1, and keep the first value unchanged, wherein the first value and the second value have preset initial values; If the second value is greater than the number of bytes included in the font protocol message, update the first value to the current value plus 1, and update the second value to the sum of the updated first value and a preset value, wherein the preset value is determined based on the number of pixels corresponding to the text character image indicated by the font protocol.

4. The method according to claim 3, characterized in that The first value is updated no more than a preset number of times, wherein the preset number of times is determined based on the number of pixels corresponding to the text character image indicated by the font protocol.

5. The method according to any one of claims 2 to 4, characterized in that: After the step of performing text recognition on the image to be recognized and obtaining a text recognition result as the analysis result of the unrecognized portion, the method further includes: In a case where the byte order of the third byte is less than the number of bytes included in the font protocol message, the first value is updated to a target value as an initial value, and the step of determining the first value and the second value according to a preset determination rule is returned, wherein the target value is determined based on the byte order of the third byte and a preset error tolerance.

6. The method according to any one of claims 1 to 4, characterized in that There are multiple images to be identified; The step of performing text recognition on the image to be recognized to obtain a text recognition result includes: Performing text recognition on each image to be recognized using a text recognition model to obtain a recognition result corresponding to each image to be recognized and a confidence level of each text included in the recognition result; Determining an alternative result from the recognition result based on text included in reference information, wherein the reference information is a reference image and / or a preset font library, the reference image is an image of a display interface corresponding to the font protocol message, and the preset font library is a font library for a scene corresponding to the font protocol message; According to the confidence level, a text recognition result corresponding to the image to be recognized is determined from the candidate results.

7. The method according to claim 6, characterized in that The reference information is a reference image; The step of determining candidate results from the recognition results based on the text included in the reference information includes: Performing text recognition on the reference image to obtain a reference result; Calculating the similarity between each recognition result and the reference result, and determining a preset number of recognition results with the largest corresponding similarities as candidate results; The step of determining the text recognition result corresponding to the image to be recognized from the candidate results according to the confidence level includes: For each candidate result, based on the confidence level corresponding to each character in the candidate result, determine the confidence level corresponding to the candidate result; According to the confidence level corresponding to each candidate result, a text recognition result of the image to be recognized is determined from the candidate results.

8. The method according to claim 6, characterized in that The reference information is a preset font library; The step of determining candidate results from the recognition results based on the text included in the reference information includes: Calculate the number of hits between the characters included in each recognition result and the characters included in the preset character library; The recognition result with the largest number of hit words is determined as the candidate result; The step of determining the text recognition result corresponding to the image to be recognized from the candidate results according to the confidence level includes: If the number of the candidate results is one, determining the text recognition result of the image to be recognized by using the candidate result; and / or If there are multiple candidate results, for each candidate result, determine the confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result; According to the confidence level corresponding to each candidate result, a text recognition result of the image to be recognized is determined from the candidate results.

9. The method according to claim 6, characterized in that The reference information is a reference image and a preset font library; The step of determining candidate results from the recognition results based on the text included in the reference information includes: Performing text recognition on the reference image to obtain a reference result; Calculating the similarity between each recognition result and the reference result, and determining a preset number of recognition results with the largest corresponding similarities as first candidate results; Calculating the number of hits between the characters included in each first candidate result and the characters included in the preset character library; The first candidate result with the largest number of hit words is determined as the second candidate result; The step of determining the text recognition result corresponding to the image to be recognized from the candidate results according to the confidence level includes: If the number of the second candidate result is one, determining the text recognition result of the image to be recognized by using the second candidate result; and / or If there are multiple second candidate results, for each second candidate result, determine the confidence level corresponding to the second candidate result based on the confidence level corresponding to each character in the second candidate result; According to the confidence level corresponding to each second candidate result, a text recognition result of the image to be recognized is determined from the second candidate results.

10. A device for parsing a font protocol message, characterized in that: The device comprises: Message acquisition module, used to obtain the font protocol message; An image generation module, configured to generate a protocol image according to the font protocol based on the order of bytes included in the font protocol message and the pixel points of the text character image indicated by each byte; an image interception module, configured to intercept, from an unrecognized portion of the protocol image, a first image corresponding to a first byte to a second byte and a second image corresponding to a second byte to a third byte, respectively, wherein the number of bytes corresponding to the first image is the same as the number of bytes corresponding to the second image; the first byte and the second byte are two bytes included in different positions in the font protocol message, and the first byte and the second byte are the same; an image stitching module, configured to stitch the first image and the second image together according to the arrangement of the split text character images indicated by the font protocol, to obtain an image to be recognized; The text recognition module is used to perform text recognition on the image to be recognized and obtain a text recognition result as the analysis result of the unrecognized part.

11. The device according to claim 10, characterized in that The device further comprises: a value determination module, configured to determine a first value and a second value according to a preset determination rule, wherein the first value and the second value are used to identify a byte order of bytes in the font protocol message; a byte determination module, configured to determine whether the byte corresponding to the first value and the byte corresponding to the second value in the font protocol message are the same; a byte determination module, configured to, if the byte corresponding to the first value is the same as the byte corresponding to the second value, determine the byte corresponding to the first value as the first byte, determine the byte corresponding to the second value as the second byte, execute the steps of respectively intercepting the first image corresponding to the first byte to the second byte and the second image corresponding to the second byte to the third byte from the unrecognized portion of the protocol image, and return to the step of determining the first value and the second value according to the preset determination rule until the determined first value and second value meet a preset condition, wherein the first value and second value corresponding to the preset condition indicate that all bytes of the font protocol message have been traversed; a returning module, configured to return to the step of determining the first and second values ​​according to a preset determination rule if the byte corresponding to the first value is different from the byte corresponding to the second value, until the determined first and second values ​​satisfy the preset condition; The numerical value determination module includes: a quantity determination submodule, configured to determine whether a byte sequence corresponding to a third byte indicated by the first and second values ​​is less than the number of bytes included in the font protocol message, and to determine whether the second value is greater than the number of bytes included in the font protocol message; a second value updating submodule, configured to update the second value to a current value plus 1, and keep the first value unchanged, if the byte order corresponding to the third byte indicated by the first value and the second value is less than the number of bytes included in the font protocol message, wherein the first value and the second value have preset initial values; a first value updating submodule, configured to update the first value to a current value plus 1, and update the second value to a sum of the updated first value and a preset value if the second value is greater than the number of bytes included in the font protocol message, wherein the preset value is determined based on the number of pixels corresponding to the text character image indicated by the font protocol; The number of updates of the first value is no greater than a preset number, wherein the preset number is determined based on the number of pixels corresponding to the text character image indicated by the font protocol; The device further comprises: a target value updating module, configured to update the first value to a target value as an initial value, and return to the step of determining the first value and the second value according to a preset determination rule, when the byte order of the third byte is less than the number of bytes included in the font protocol message, wherein the target value is determined based on the byte order of the third byte and a preset error tolerance; There are multiple images to be identified; The text recognition module includes: A confidence determination submodule, configured to perform text recognition on each image to be recognized using a text recognition model, and obtain a recognition result corresponding to each image to be recognized and a confidence level of each text included in the recognition result; an alternative result determination submodule, configured to determine an alternative result from the recognition result based on text included in reference information, wherein the reference information is a reference image and / or a preset font library, the reference image is an image of a display interface corresponding to the font protocol message, and the preset font library is a font library for the scene corresponding to the font protocol message; a recognition result determination submodule, configured to determine, from the candidate results, a text recognition result corresponding to the image to be recognized based on the confidence level; The reference information is a reference image; The alternative result determination submodule includes: A reference result acquisition unit, configured to perform text recognition on the reference image to obtain a reference result; A first candidate result determination unit is configured to calculate the similarity between each recognition result and the reference result, and determine a preset number of recognition results with the largest corresponding similarities as candidate results; The recognition result determination submodule includes: a first confidence determination unit, configured to determine, for each candidate result, a confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result; a first recognition result determination unit, configured to determine a text recognition result of the image to be recognized from the candidate results according to the confidence level corresponding to each candidate result; The reference information is a preset font library; The alternative result determination submodule includes: A first hit word count calculation unit, configured to calculate the hit word count between the characters included in each recognition result and the characters included in the preset character library; A second candidate result determination unit is used to determine the corresponding recognition result with the largest number of hit words as the candidate result; The recognition result determination submodule includes: A first result determination unit is configured to determine, if the number of the candidate results is one, the text recognition result of the image to be recognized using the candidate result; and / or a confidence determination unit, configured to determine, for each candidate result, a confidence level corresponding to the candidate result based on the confidence level corresponding to each character in the candidate result, if there are multiple candidate results; a second result determination unit, configured to determine a text recognition result of the image to be recognized from the candidate results according to the confidence level corresponding to each candidate result; The reference information is a reference image and a preset font library; The alternative result determination submodule includes: A reference result acquisition unit, configured to perform text recognition on the reference image to obtain a reference result; A third candidate result determination unit is configured to calculate the similarity between each recognition result and the reference result, and determine a preset number of recognition results with the largest corresponding similarities as first candidate results; A second hit word number calculation unit, used for calculating the hit word number between the characters included in each first candidate result and the characters included in the preset character library; A fourth candidate result determination unit, configured to determine the first candidate result with the largest number of hit words as the second candidate result; The recognition result determination submodule includes: A second recognition result determining unit is configured to determine the text recognition result of the image to be recognized using the second candidate result if the number of the second candidate result is one; and / or an alternative result confidence determination unit, configured to determine, for each second alternative result, a confidence corresponding to the second alternative result based on the confidence corresponding to each character in the second alternative result, if there are multiple second alternative results; The third recognition result determination unit is configured to determine the text recognition result of the image to be recognized from the second candidate results according to the confidence level corresponding to each second candidate result.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 9 when executing a program stored in a memory.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images