A method for determining text line direction and related device

Abstract

The present application discloses a method for determining the direction of a text line and related equipment. After acquiring an image to be processed, the global area of ​​the text line to be processed and the local area of ​​the text line to be processed in the image to be processed are first determined, so that the global area of ​​the text line to be processed represents the area occupied by the text line to be processed in the image to be processed, and the local area of ​​the text line to be processed represents the area occupied by a preset part of the text line to be processed in the image to be processed; then the global area of ​​the text line to be processed is compared with the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed. In this way, the text line direction of the text line in the image can be accurately determined, which is beneficial to improving the character recognition accuracy of the text line in the image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for determining the direction of a text line and related equipment. Background Art

[0002] Character recognition (eg, optical character recognition) refers to the process of recognizing characters in an image to obtain text information carried in the image.

[0003] In fact, a text line in an image can be presented in any direction (for example, rotated 90° clockwise, etc.), so in order to improve the accuracy of character recognition, the text line direction of the text line can be referred to when performing character recognition on a text line in an image (especially a text line with a certain rotation angle). The text line direction refers to the rotation information of a text line presented in an image.

[0004] However, how to determine the direction of a text line becomes a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the above technical problems existing in the prior art, the present application provides a text line direction determination method and related equipment, which can accurately determine the text line direction of the text line in the image, which is conducive to improving the character recognition accuracy of the text line in the image.

[0006] In order to achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:

[0007] An embodiment of the present application provides a method for determining a text line direction, the method comprising: acquiring an image to be processed; determining a global area of ​​the text line to be processed and a local area of ​​the text line to be processed in the image to be processed; wherein the global area of ​​the text line to be processed represents an area occupied by the text line to be processed in the image to be processed; the local area of ​​the text line to be processed represents an area occupied by a preset part of the text line to be processed in the image to be processed; and performing a position comparison between the global area of ​​the text line to be processed and the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed.

[0008] The embodiment of the present application also provides a device for determining a text line direction, the device comprising:

[0009] An image acquisition unit, used for acquiring an image to be processed;

[0010] A region determination unit, used to determine a global region of a text line to be processed in the image to be processed and a local region of the text line to be processed; wherein the global region of the text line to be processed represents the region occupied by the text line to be processed in the image to be processed; and the local region of the text line to be processed represents the region occupied by a preset portion of the text line to be processed in the image to be processed;

[0011] The direction determination unit is used to compare the positions of the global area of ​​the text line to be processed and the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed.

[0012] The present application also provides a device, the device comprising a processor and a memory:

[0013] The memory is used to store computer programs;

[0014] The processor is used to execute any implementation of the text line direction determination method provided in the embodiments of the present application according to the computer program.

[0015] The embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute any implementation of the text line direction determination method provided in the embodiment of the present application.

[0016] The embodiment of the present application also provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any implementation of the text line direction determination method provided in the embodiment of the present application.

[0017] Compared with the prior art, the embodiments of the present application have at least the following advantages:

[0018] In the text line direction determination method and related devices provided in the embodiments of the present application, after acquiring the image to be processed, the global area of ​​the text line to be processed and the local area of ​​the text line to be processed in the image to be processed are first determined, so that the global area of ​​the text line to be processed represents the area occupied by the text line to be processed in the image to be processed, and the local area of ​​the text line to be processed represents the area occupied by a preset part of the text line to be processed in the image to be processed; then the global area of ​​the text line to be processed and the local area of ​​the text line to be processed are compared in position to obtain the text line direction of the text line to be processed.

[0019] Among them, since the global area and local area of ​​the text line to be processed can respectively represent the area occupied by the text line to be processed in the image to be processed and the area occupied by the preset part of the text line to be processed in the image to be processed, the global area and local area of ​​the text line to be processed can comprehensively represent the presentation method of the text line to be processed in the image to be processed, so that the text line direction of the text line to be processed determined based on the global area and local area of ​​the text line to be processed can accurately describe the presentation method of the text line to be processed in the image to be processed, so that the text line direction of the text line in the image can be accurately determined, which is beneficial to improve the character recognition accuracy of the text line in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of an image, its corresponding global area, and its corresponding local area provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of an original image, its corresponding global mask image, and a local mask image provided in an embodiment of the present application;

[0023] Figure 3 A flowchart of a method for determining a text line direction provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a directed connection provided in an embodiment of the present application;

[0025] Figure 5 A characteristic schematic diagram of a rotation method of a horizontal text line provided in an embodiment of the present application;

[0026] Figure 6 A characteristic schematic diagram of a rotation method of a vertical text line provided in an embodiment of the present application;

[0027] Figure 7 A characteristic schematic diagram of another method of rotating a horizontal text line provided in an embodiment of the present application;

[0028] Figure 8 A characteristic schematic diagram of another vertical text line rotation method provided in an embodiment of the present application;

[0029] Fig. 9A schematic diagram of the structure of a device for determining a text line direction provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The inventors found in their research on text line directions that for a text line in an image, when the text line is presented in the image according to different text line directions, the area occupied by the text line in the image (such as Figure 1 The global area shown in FIG. 1 ) and the preset part of the text line (eg, the first character or the preset local area of ​​each character) occupy an area in the image (eg, Figure 1 The first character area or string local area shown in the figure will be presented according to different relative relationships (such as Figure 2 shown).

[0031] It should be noted that in Figure 2 In the text, "text line-1" is a horizontal text line presented in the positive direction (i.e., the rotation angle is 0°); "text line-2" is a horizontal text line presented in a counterclockwise rotation of a first angle, and the first angle is between 0° and 90°; "text line-3" is a horizontal text line presented in a clockwise rotation of 180° or a counterclockwise rotation of 180°; "text line-4" is a horizontal text line presented in a clockwise rotation of 90°; "text line-5" is a horizontal text line presented in a counterclockwise rotation of 90°; "text line-6" is a vertical text line presented in the positive direction (i.e., the rotation angle is 0°); "text line-7" is a vertical text line presented in a clockwise rotation of 180° or a counterclockwise rotation of 180°. Among them, the text arrangement of the horizontal text line is horizontal; and the text arrangement of the vertical text line is vertical.

[0032] Based on the above findings, an embodiment of the present application provides a method for determining the direction of a text line, which may include: obtaining an image to be processed; determining the global area of ​​the text line to be processed and the local area of ​​the text line to be processed in the image to be processed; comparing the positions of the global area of ​​the text line to be processed and the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed. Wherein, since the global area and the local area of ​​the text line to be processed can respectively represent the area occupied by the text line to be processed in the image to be processed and the area occupied by the preset part of the text line to be processed in the image to be processed, the global area and the local area of ​​the text line to be processed can comprehensively represent the presentation mode of the text line to be processed in the image to be processed, so that the text line direction of the text line to be processed determined based on the global area and the local area of ​​the text line to be processed can accurately describe the presentation mode of the text line to be processed in the image to be processed, so that the text line direction of the text line in the image can be accurately determined, which is beneficial to improve the character recognition accuracy of the text line in the image.

[0033] In addition, the embodiments of the present application do not limit the execution subject of the text line direction determination method. For example, the text line direction determination method provided in the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. The terminal device can be a smart phone, a computer, a personal digital assistant (PDA) or a tablet computer. The server can be an independent server, a cluster server or a cloud server.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] Method Embodiment

[0036] See also Figure 3 , which is a flowchart of a method for determining a text line direction provided in an embodiment of the present application.

[0037] The text line direction determination method provided in the embodiment of the present application includes S1-S3:

[0038] S1: Obtain the image to be processed.

[0039] The image to be processed refers to an image that needs to be processed for determining the text line direction. The text line direction is used to describe the presentation method of a text line in an image; moreover, the embodiment of the present application does not limit the text line direction. For example, the text line direction may include the text arrangement method and / or the text line rotation method.

[0040] The text arrangement is used to describe the arrangement of all characters in a text line; and the embodiment of the present application does not limit the text arrangement. For example, the text arrangement can be horizontal (such as Figure 2 ), or vertical (such as Figure 2 Horizontal arrangement refers to the arrangement of all characters in a text line in the horizontal direction. Vertical arrangement refers to the arrangement of all characters in a text line in the vertical direction.

[0041] The text line rotation mode is used to describe the rotation information of a text line in an image; moreover, the embodiments of the present application do not limit the text line rotation mode. For example, the text line rotation mode may include a rotation direction and / or a rotation angle.

[0042] In addition, the embodiments of the present application do not limit the text arrangement of the text lines in the image to be processed. For example, for the nth text line in the image to be processed, the text arrangement of the nth text line can be horizontal, so that the nth text line belongs to a horizontal text line; or, the text arrangement of the nth text line can be vertical, so that the nth text line belongs to a vertical text line. In addition, the embodiments of the present application do not limit the text line rotation mode of the text line in the image to be processed. For example, for the nth text line in the image to be processed, the text line rotation mode of the nth text line may be the positive direction (that is, rotation of 0°); or, the text line rotation mode of the nth text line may be a second angle of clockwise rotation, and the second angle ∈(0°, 180°); or, the text line rotation mode of the nth text line may be a third angle of counterclockwise rotation, and the third angle ∈(0°, 180°); or, the text line rotation mode of the nth text line may be a rotation of 180° (that is, rotation of 180° counterclockwise or clockwise). Wherein, n is a positive integer, n≤N, and N represents the number of text lines in the image to be processed.

[0043] S2: Determine a global region of a text line to be processed and a local region of the text line to be processed in the image to be processed.

[0044] The to-be-processed text line is used to represent any text line in the to-be-processed image. For example, if there are N text lines in the to-be-processed image, the first text line, the second text line, ..., the Nth text line in the to-be-processed image can be respectively determined as the to-be-processed text line.

[0045] The global area of ​​the text line to be processed represents the area occupied by the text line to be processed in the image to be processed.

[0046] The local area of ​​the text line to be processed represents the area occupied by a preset portion of the text line to be processed in the image to be processed.

[0047] The preset part can be preset according to the application scenario. For example, the preset part can include the first character part (such as Figure 2 For example, the preset portion of the horizontal text line may include the bottom boundary portion of the horizontal text line (eg, Figure 2 The preset portion of the vertical text line may include the central portion of the vertical text line (eg, Figure 2 The part corresponding to the mask area of ​​"text line -6" in the second mask image in ).

[0048] Based on the above-mentioned relevant contents of the "preset part", it can be known that the local area may include the first character area and / or the string local area. Among them, the first character area represents the area occupied by the first character of a text line in the image. The string local area is used to represent the area occupied by the preset local areas of each character in a text line in the image.

[0049] In addition, the preset local area can be pre-set, and in particular, can be set according to the text arrangement of a text line. For example, if the text arrangement of a text line is horizontal, the preset local area of ​​each character in the text line can be the bottom boundary area of ​​each character; if the text arrangement of a text line is vertical, the preset local area of ​​each character in the text line can be the central area of ​​each character (especially the central area that can vertically penetrate the character).

[0050] Based on the above-mentioned relevant contents of "local area", it can be known that the local area of ​​the text line to be processed may include the first character area of ​​the text line to be processed and / or the local area of ​​the character string of the text line to be processed. Among them, the first character area of ​​the text line to be processed refers to the area occupied by the first character in the text line to be processed in the image to be processed. The local area of ​​the character string of the text line to be processed is obtained by connecting the preset local areas of each character in the text line to be processed, and it may specifically include: if the text arrangement of the text line to be processed is horizontal, then the local area of ​​the character string of the text line to be processed is obtained by connecting the bottom boundary areas of each character in the text line to be processed; if the text arrangement of the text line to be processed is vertical, then the local area of ​​the character string of the text line to be processed is obtained by connecting the central areas of each character in the text line to be processed.

[0051] In addition, the embodiment of the present application does not limit the implementation of S2. For example, in a possible implementation, S2 may specifically include S21-S23:

[0052] S21: Input the image to be processed into the pre-built mask map generation model, and obtain the predicted text line global mask map corresponding to the image to be processed and the predicted text line local mask map corresponding to the image to be processed output by the mask map generation model. The predicted text line global mask map corresponding to the image to be processed includes the global mask area corresponding to the text line to be processed; the predicted text line local mask map corresponding to the image to be processed includes the local mask area corresponding to the text line to be processed.

[0053] The mask map generation model is used to predict the global mask map and local mask map of the model input image. Here, the "model input image" refers to the image input to the mask map generation model (such as Figure 2 original image shown).

[0054] The global mask map is obtained after masking the area occupied by the text line in the model input image, so that the global mask map can represent the area occupied by the text line in the model input image.

[0055] The local mask image is obtained by performing mask processing on the area occupied by the preset part of the text line in the model input image, so that the local mask image can represent the area occupied by the preset part of the text line in the model input image. It should be noted that the relevant content of the "preset part" can be found above.

[0056] In addition, the embodiment of the present application does not limit the local mask map. For example, the local mask map may include a first character mask sub-map (such as Figure 2 The third mask map shown) and / or the string local mask submap (such as Figure 2The first character mask sub-graph is obtained by masking the area occupied by the first character of the text line in the model input image, so that the first character mask sub-graph can represent the area occupied by the first character of the text line in the model input image. The string local mask sub-graph is obtained by masking the area occupied by the local part of the string of the text line in the model input image, so that the string local mask sub-graph represents the area occupied by the local part of the string of the text line in the model input image.

[0057] It should be noted that the local part of the string can be preset, and in particular, can be set according to the text arrangement of a text line. For example, the local part of the string of a horizontal text line can be the bottom boundary of the horizontal text line. For another example, the local part of the string of a vertical text line can be the central area of ​​the vertical text line (especially the central area that vertically runs through the vertical text line).

[0058] In addition, the embodiments of the present application do not limit the model structure of the mask map generation model. For example, the mask map generation model can be implemented using the model structure of any deep learning model (such as a semantic segmentation model based on deep learning).

[0059] In addition, the mask map generation model can be constructed based on the sample image, the actual text line global mask map corresponding to the sample image, and the actual text line local mask map corresponding to the sample image. The sample image refers to the image required for constructing the mask map generation model; the actual text line global mask map corresponding to the sample image is used to represent the actual area occupied by each text line in the sample image in the sample image; the actual text line local mask map corresponding to the sample image is used to represent the actual area occupied by the preset part of each text line in the sample image in the sample image.

[0060] In addition, the embodiment of the present application does not limit the construction process of the mask map generation model. For example, in a possible implementation manner, the construction process of the mask map generation model may specifically include steps 11 to 15:

[0061] Step 11: Obtain a sample image, a global mask map of the actual text line corresponding to the sample image, and a local mask map of the actual text line corresponding to the sample image.

[0062] Step 12: Input the sample image into the model to be trained, and obtain the predicted text line global mask map corresponding to the sample image and the predicted text line local mask map corresponding to the sample image output by the model to be trained.

[0063] Among them, the model to be trained refers to the model that needs to be trained when constructing the mask image generation model; moreover, the embodiment of the present application does not limit the model to be trained. For example, the model to be trained can be a deep learning model.

[0064] The predicted text line global mask map corresponding to the sample image is used to represent the predicted area occupied by at least one text line in the sample image within the sample image.

[0065] The predicted text line local mask map corresponding to the sample image is used to represent the area predicted to be occupied by a preset portion of at least one text line in the sample image within the sample image.

[0066] Step 13: Determine whether the preset stop condition is reached, if so, execute step 15; if not, execute step 14.

[0067] The preset stop condition can be preset; moreover, the embodiment of the present application does not limit the preset stop condition, for example, the preset stop condition can be that the loss value of the model to be trained is lower than the first threshold, or the change rate of the loss value of the model to be trained is lower than the second threshold, or the number of updates of the model to be trained reaches the third threshold. The first threshold, the second threshold and the third threshold are all preset.

[0068] The loss value of the model to be trained is used to measure the prediction performance of the model to be trained; and the loss value of the model to be trained can be determined based on the difference between the predicted data (that is, the predicted text line global mask map and the predicted text line local mask map corresponding to the sample image) and the label data (that is, the actual text line global mask map and the actual text line local mask map corresponding to the sample image). In addition, the embodiment of the present application does not limit the calculation method of the loss value of the model to be trained. For example, it can be implemented using the Dice coefficient, Dice Loss, or Laplace smoothing.

[0069] Based on the relevant content of the above step 13, it can be known that for the current round of the model to be trained, it can be judged whether the current round of the model to be trained reaches the preset stop condition. If the preset stop condition is reached, it means that the current round of the model to be trained has good prediction performance, so the mask map generation model can be directly constructed based on the current round of the model to be trained, so that the mask map generation model also has good prediction performance; if the preset stop condition is not reached, it means that the prediction performance of the current round of the model to be trained is still relatively poor, so the current round of the model to be trained can be updated according to the predicted text line global mask map, the actual text line global mask map, the predicted text line local mask map and the actual text line local mask map corresponding to the sample image, so that the updated model to be trained has better prediction performance.

[0070] Step 14: Update the model to be trained according to the predicted text line global mask map corresponding to the sample image, the actual text line global mask map corresponding to the sample image, the predicted text line local mask map corresponding to the sample image, and the actual text line local mask map corresponding to the sample image, and return to execute step 12.

[0071] It should be noted that the embodiments of the present application do not limit the updating method of the model to be trained, and can be implemented by any existing or future model updating method (for example, the model can be updated according to the loss value of the model to be trained).

[0072] Step 15: Construct a mask image generation model based on the model to be trained.

[0073] In an embodiment of the present application, after determining that the model to be trained in the current round reaches a preset stop condition, a mask map generation model can be constructed according to the model to be trained in the current round (for example, the model to be trained in the current round is directly determined as the mask map generation model; or, according to the model structure and model parameters of the model to be trained in the current round, the model structure and model parameters of the mask map generation model are determined, so that the mask map generation model has the same model structure and model parameters as the model to be trained in the current round), so that the mask map generation model has the same prediction performance as the model to be trained in the current round, and thus the mask map generation model also has better prediction performance.

[0074] Based on the relevant contents of steps 11 to 15 above, it can be known that after obtaining the sample image, the actual text line global mask map corresponding to the sample image, and the actual text line local mask map, the sample image can be used as the model input data required to construct the mask map generation model, and the actual text line global mask map and the actual text line local mask map corresponding to the sample image can be used as the model label data required to construct the mask map generation model, and the mask map generation model is constructed based on the model input data and the model label data, so that the constructed mask map generation model has better prediction performance, so that the constructed mask map can be used to predict the global mask map and the local mask map in the future.

[0075] The predicted text line global mask map corresponding to the image to be processed is obtained by predicting the global mask map of the image to be processed using the constructed mask map generation model, so that the predicted text line global mask map is used to represent the area occupied by at least one text line in the image to be processed. It can be seen that the predicted text line global mask map corresponding to the image to be processed may include the global mask area corresponding to all or part of the text lines in the image to be processed. Among them, the global mask area is used to represent the area occupied by a text line in the image to be processed.

[0076] The predicted text line local mask map corresponding to the image to be processed is obtained by predicting the local mask map of the image to be processed using the constructed mask map generation model, so that the predicted text line local mask map is used to represent the area occupied by the preset part of at least one text line in the image to be processed in the image to be processed. It can be seen that the predicted text line local mask map corresponding to the image to be processed may include the local mask area corresponding to all or part of the text lines in the image to be processed. Among them, the local mask area is used to represent the area occupied by the preset part of a text line in the image to be processed.

[0077] In addition, the embodiments of the present application do not limit the predicted text line local mask map corresponding to the image to be processed. For example, it may include a first character mask submap corresponding to the image to be processed and / or a string local mask submap corresponding to the image to be processed. Among them, the first character mask submap corresponding to the image to be processed is used to represent the area occupied by the first character of at least one text line in the image to be processed within the model input image. The string local mask submap corresponding to the image to be processed is used to represent the area occupied by the local part of the string of at least one text line in the image to be processed within the model input image.

[0078] Based on the relevant content of S21 above, it can be known that after obtaining the image to be processed, the image to be processed can be input into a pre-constructed mask map generation model, so that the mask map generation model can predict the global mask map and the local mask map for the image to be processed, and obtain and output the predicted text line global mask map and the predicted text line local mask map corresponding to the image to be processed, so that the predicted text line global mask map and the predicted text line local mask map can accurately represent the area occupied by at least one text line and its preset part in the image to be processed.

[0079] S22: Determine the global mask area corresponding to the text line to be processed in the predicted text line global mask map corresponding to the image to be processed as the global area of ​​the text line to be processed.

[0080] In an embodiment of the present application, after obtaining the global mask map of the predicted text line corresponding to the image to be processed, the global mask area corresponding to the text line to be processed recorded in the global mask map of the predicted text line can be determined as the global area of ​​the text line to be processed, so that the direction of the text line can be subsequently determined based on the global area of ​​the text line to be processed.

[0081] S23: Determine the local mask area corresponding to the text line to be processed in the predicted text line local mask map corresponding to the image to be processed as the local area of ​​the text line to be processed.

[0082] The embodiment of the present application does not limit the process of determining the local area of ​​the text line to be processed (that is, the implementation method of S23). For ease of understanding, it is explained below with reference to three examples.

[0083] Example 1: If the predicted text line local mask image corresponding to the image to be processed includes the first character mask sub-image corresponding to the image to be processed, S23 may specifically include steps 21 and 22:

[0084] Step 21: Determine the first character region of the text line to be processed from the first character mask sub-image corresponding to the image to be processed.

[0085] The first character mask sub-image corresponding to the image to be processed is used to represent the area occupied by the first character of at least one text line in the image to be processed in the image to be processed.

[0086] The first character area of ​​the text line to be processed is used to represent the area occupied by the first character of the text line to be processed in the image to be processed.

[0087] Step 22: Determine the local area of ​​the text line to be processed according to the first character area of ​​the text line to be processed, so that the local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed.

[0088] Based on the relevant contents of the above steps 21 to 22, it can be known that for the text line to be processed, after obtaining the first character mask sub-image corresponding to the image to be processed, the first character area of ​​the text line to be processed can be determined from the first character mask sub-image, so that the first character area can accurately represent the area occupied by the first character of the text line to be processed in the image to be processed; and then, based on the first character area, the local area of ​​the text line to be processed is determined, so that the local area can also accurately represent the area occupied by the first character of the text line to be processed in the image to be processed.

[0089] Example 2: If the predicted text line local mask image corresponding to the image to be processed includes the character string local mask sub-image corresponding to the image to be processed, S23 may specifically include steps 31 and 32:

[0090] Step 31: Determine the local region of the string of the text line to be processed from the local mask sub-image of the string corresponding to the image to be processed.

[0091] The character string local mask sub-image corresponding to the image to be processed is used to represent the area occupied by the local part of the character string of at least one text line in the image to be processed in the image to be processed.

[0092] The character string local area of ​​the text line to be processed is used to represent the area occupied by the local part of the character string of the text line to be processed in the image to be processed.

[0093] Step 32: Determine the local area of ​​the text line to be processed according to the local area of ​​the character string of the text line to be processed, so that the local area of ​​the text line to be processed includes the local area of ​​the text line to be processed.

[0094] Based on the relevant contents of the above steps 31 to 32, it can be known that for the text line to be processed, after obtaining the string local mask sub-map corresponding to the image to be processed, the string local area of ​​the text line to be processed can be first determined from the string local mask sub-map, so that the string local area can accurately represent the area occupied by the local part of the string of the text line to be processed in the image to be processed; and then, based on the string local area, the local area of ​​the text line to be processed is determined, so that the local area can also accurately represent the area occupied by the local part of the string of the text line to be processed in the image to be processed.

[0095] Example 3: If the predicted text line local mask image corresponding to the image to be processed includes the first character mask sub-image corresponding to the image to be processed and the string local mask sub-image corresponding to the image to be processed, S23 may specifically include steps 41 to 43:

[0096] Step 41: Determine the first character region of the text line to be processed from the first character mask sub-image corresponding to the image to be processed.

[0097] Step 42: Determine the local region of the string of the text line to be processed from the local mask sub-image of the string corresponding to the image to be processed.

[0098] It should be noted that for the relevant contents of step 41 and step 42, please refer to the above step 21 and step 31 respectively.

[0099] Step 43: Determine the local area of ​​the text line to be processed based on the first character area of ​​the text line to be processed and the local area of ​​the string of the text line to be processed, so that the local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed and the local area of ​​the string of the text line to be processed.

[0100] Based on the relevant contents of the above steps 41 to 43, it can be known that for the text line to be processed, after obtaining the first character mask sub-image and the string local mask sub-image corresponding to the image to be processed, first determine the first character area and the string local area of ​​the text line to be processed from the first character mask sub-image and the string local mask sub-image respectively, so that the first character area and the string local area can accurately represent the areas occupied by the text line to be processed and the local part of the string in the image to be processed respectively; then determine the local area of ​​the text line to be processed based on the first character area and the string local area, so that the local area can also accurately represent the areas occupied by the text line to be processed and the local part of the string in the image to be processed respectively.

[0101] Based on the relevant content of S2 above, it can be known that after obtaining the image to be processed, the global area and the local area of ​​each text line to be processed can be determined from the image to be processed, so that the text line direction of each text line to be processed can be determined separately based on the global area and the local area of ​​each text line to be processed.

[0102] S3: Compare the positions of the global area of ​​the text line to be processed and the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed.

[0103] The text line direction of the text line to be processed is used to indicate the presentation mode of the text line to be processed in the image to be processed.

[0104] In addition, the embodiments of the present application do not limit the text line direction of the text line to be processed. For example, the text line direction of the text line to be processed may include the text line rotation mode of the text line to be processed and / or the text arrangement mode of the text line to be processed; and the text line rotation mode of the text line to be processed may include the text line rotation direction of the text line to be processed and / or the text line rotation angle of the text line to be processed.

[0105] The embodiments of the present application also do not limit the implementation methods of the above-mentioned "position comparison". For example, the relative information between the global area of ​​the text line to be processed and the local area of ​​the text line to be processed can be matched with at least one candidate standard relative information, and the text line direction corresponding to the successfully matched candidate standard relative information is determined as the text line direction of the text line to be processed. Among them, the above "relative information" may include relative parameters between different areas (for example, relative relationships and relative differences in positions, relative relationships and relative differences in area ranges, etc.). In addition, each candidate standard relative information can be pre-set, and the embodiments of the present application do not limit the number of candidate standard relative information.

[0106] In addition, in order to further improve the accuracy of the text line direction, the embodiment of the present application further provides a possible specific implementation of S3, which may specifically include S31-S33:

[0107] S31: Determine the center of the global region of the text line to be processed, so that the center of the global region of the text line to be processed is used to represent the center position of the global region of the text line to be processed.

[0108] In an embodiment of the present application, after the global area of ​​the text line to be processed is obtained, the global area center of the text line to be processed can be determined based on the center position of the global area, so that the global area center of the text line to be processed can accurately represent the center position of the area occupied by the text line to be processed in the image to be processed.

[0109] S32: Determine the center of the local area of ​​the text line to be processed, so that the center of the local area of ​​the text line to be processed is used to represent the central position of the local area of ​​the text line to be processed.

[0110] The embodiment of the present application does not limit the implementation method of S32. For ease of understanding, it is explained below with reference to three examples.

[0111] Example 1: If the local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed, S32 may specifically include steps 51 and 52:

[0112] Step 51: Determine the center of the first character area of ​​the text line to be processed from the first character area of ​​the text line to be processed, so that the center of the first character area of ​​the text line to be processed is used to represent the center position of the first character area of ​​the text line to be processed.

[0113] In an embodiment of the present application, after obtaining the first character area of ​​the text line to be processed, the center position of the first character area can be determined as the center of the first character area of ​​the text line to be processed, so that the center of the first character area can accurately represent the center position of the area occupied by the first character of the text line to be processed in the image to be processed.

[0114] Step 52: Determine the local area center of the text line to be processed according to the first character area center of the text line to be processed, so that the local area center of the text line to be processed includes the first character area center of the text line to be processed.

[0115] Based on the relevant contents of the above steps 51 to 52, it can be known that after obtaining the first character area of ​​the text line to be processed, the center of the first character area of ​​the text line to be processed can be first determined from the first character area, so that the center of the first character area can accurately represent the center position of the area occupied by the first character of the text line to be processed in the image to be processed; and then, based on the center of the first character area, the center of the local area of ​​the text line to be processed is determined, so that the center of the local area can also accurately represent the center position of the area occupied by the first character of the text line to be processed in the image to be processed.

[0116] Example 2: If the local area of ​​the text line to be processed includes the local area of ​​the character string of the text line to be processed, S32 may specifically include steps 61 and 62:

[0117] Step 61: Determine the center of the local string area of ​​the text line to be processed from the local string area of ​​the text line to be processed, so that the center of the local string area of ​​the text line to be processed is used to represent the central position of the local string area of ​​the text line to be processed.

[0118] In an embodiment of the present application, after obtaining the local string area of ​​the text line to be processed, the center position of the local string area can be determined as the center of the local string area of ​​the text line to be processed, so that the center of the local string area can accurately represent the center position of the area occupied by the local part of the string of the text line to be processed in the image to be processed.

[0119] Step 62: Determine the local area center of the text line to be processed according to the local area center of the character string of the text line to be processed, so that the local area center of the text line to be processed includes the local area center of the character string of the text line to be processed.

[0120] Based on the relevant contents of the above steps 61 to 62, it can be known that after obtaining the local area of ​​the string of the text line to be processed, the center of the local area of ​​the string of the text line to be processed can be first determined from the local area of ​​the string, so that the center of the local area of ​​the string can accurately represent the center position of the area occupied by the local part of the string of the text line to be processed in the image to be processed; and then, based on the center of the local area of ​​the string, the center of the local area of ​​the text line to be processed is determined, so that the center of the local area can also accurately represent the center position of the area occupied by the local part of the string of the text line to be processed in the image to be processed.

[0121] Example 3: If the local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed and the character string local area of ​​the text line to be processed, S32 may specifically include steps 71 to 73:

[0122] Step 71: Determine the center of the first character area of ​​the text line to be processed from the first character area of ​​the text line to be processed, so that the center of the first character area of ​​the text line to be processed is used to represent the center position of the first character area of ​​the text line to be processed.

[0123] Step 72: Determine the center of the local string area of ​​the text line to be processed from the local string area of ​​the text line to be processed, so that the center of the local string area of ​​the text line to be processed is used to represent the central position of the local string area of ​​the text line to be processed.

[0124] It should be noted that for the relevant contents of step 71 and step 72, please refer to the above steps 51 and 61 respectively.

[0125] Step 73: Determine the local area center of the text line to be processed based on the first character area center of the text line to be processed and the local area center of the string of the text line to be processed, so that the local area center of the text line to be processed includes the first character area center of the text line to be processed and the local area center of the string of the text line to be processed.

[0126] Based on the relevant contents of the above steps 71 to 73, it can be known that after obtaining the first character area and the local area of ​​the string of the text line to be processed, the center of the first character area and the center of the local area of ​​the string of the text line to be processed can be determined from the first character area and the local area of ​​the string respectively; then the set of the center of the first character area and the center of the local area of ​​the string can be determined as the center of the local area of ​​the text line to be processed, so that the local area center can accurately represent the center position of the area occupied by the first character of the text line to be processed and the local part of the string in the image to be processed.

[0127] Based on the relevant content of S32 above, it can be known that after obtaining the local area of ​​the text line to be processed, the local area center of the text line to be processed can be determined from the local area, so that the local area center can accurately represent the central position of the area occupied by the preset part of the text line to be processed (such as the first character and / or the local part of the string) in the image to be processed.

[0128] S33: Determine the text line direction of the text line to be processed according to the relative position information between the global area center of the text line to be processed and the local area center of the text line to be processed.

[0129] The relative position information is used to describe the relative relationship and relative difference between different objects (eg, the center of a global region and the center of a local region) in terms of position (eg, horizontal direction, vertical direction, etc.).

[0130] In addition, the embodiments of the present application do not limit the implementation methods of S33. To facilitate understanding of S33, two examples are provided below for illustration.

[0131] In the first example of S33, if the text line direction of the text line to be processed includes the text line rotation mode of the text line to be processed, and the local area center of the text line to be processed includes the first character area center of the text line to be processed, then S331 may specifically include: determining the text line direction of the text line to be processed based on the relative position information between the global area center of the text line to be processed and the first character area center of the text line to be processed.

[0132] In addition, the embodiment of the present application does not limit the implementation of the above determination process. For example, in a possible implementation, if the text line direction includes the text line rotation direction and / or the text line rotation angle, the determination process of the text line direction of the text line to be processed may include all or part of the steps in step 81 to step 84:

[0133] Step 81: Directedly connect the global region center of the text line to be processed and the first character region center of the text line to be processed to obtain a first vector.

[0134] It should be noted that the embodiment of the present application does not limit the connection method of the "directed connection" in step 81. For example, the global area center of the text line to be processed can be used as the starting point, and the first character area center of the text line to be processed can be used as the end point for directed connection, so that the first vector becomes a directed line segment with the global area center of the text line to be processed as the starting point and the first character area center of the text line to be processed as the end point. For another example, the first character area center of the text line to be processed can be used as the starting point, and the global area center of the text line to be processed can be used as the end point for directed connection, so that the first vector becomes a directed line segment with the first character area center of the text line to be processed as the starting point and the global area center of the text line to be processed as the end point.

[0135] Step 82: Determine the angle between the first vector and the first preset vector as the first rotation angle of the text line to be processed.

[0136] The first preset vector refers to a preset reference vector used to represent a directed connection between the center of the global region and the center of the first character region.

[0137] In fact, text lines with different text arrangements correspond to different reference vectors. Based on this, the embodiment of the present application also provides a possible implementation method for determining the first preset vector, which may specifically include: determining the first preset vector according to the text arrangement of the text line to be processed and the first mapping relationship. The first mapping relationship includes the correspondence between the text arrangement of the text line to be processed and the first preset vector.

[0138] Among them, the first mapping relationship is used to record the reference vector corresponding to different text arrangements for representing the directed connection between the global area center and the first character area center; and the embodiment of the present application does not limit the first mapping relationship. For example, the first mapping relationship may include the correspondence between the horizontal row and the first standard vector, and the correspondence between the vertical row and the second standard vector.

[0139] The first standard vector refers to a standard vector used to represent a directed connection between the global area center and the first character area center of a horizontal text line; and the embodiment of the present application does not limit the first standard vector. For example, the first standard vector can be determined based on the directed connection between the global area center of a standard horizontal text line and the first character area center of the standard horizontal text line (for example, Figure 4As shown, the directed connection between the global area center of the standard horizontal text line and the first character area center of the standard horizontal text line can be determined as the first standard vector. For another example, any parallel vector of the directed connection between the global area center of the standard horizontal text line and the first character area center of the standard horizontal text line can be determined as the first standard vector). Wherein, the text direction of the standard horizontal text line is the horizontal direction, and the text line direction of the standard horizontal text line is the positive direction (that is, rotated 0°).

[0140] The second standard vector refers to a standard vector used to represent the directed connection between the global area center and the first character area center of the vertical text line; and the embodiment of the present application does not limit the second standard vector. For example, the second standard vector can be determined based on the directed connection between the global area center of the standard vertical text line and the first character area center of the standard vertical text line (for example, Figure 4 As shown, the directed connection between the global area center of the standard vertical text line and the first character area center of the standard vertical text line can be determined as the second standard vector. For another example, any parallel vector of the directed connection between the global area center of the standard vertical text line and the first character area center of the standard vertical text line can be determined as the second standard vector). Wherein, the text direction of the standard vertical text line is the vertical direction, and the text line direction of the standard vertical text line is the positive direction (that is, rotated 0°).

[0141] Based on the above-mentioned relevant content of the "first preset vector", it can be known that for the text to be processed, if it is determined that the text arrangement of the text line to be processed is horizontal, the first standard vector can be determined as the first preset vector; if it is determined that the text arrangement of the text line to be processed is vertical, the second standard vector can be determined as the first preset vector.

[0142] In addition, the first rotation angle may be used to represent a text line rotation angle of the text line to be processed.

[0143] Based on the relevant content of the above step 82, it can be known that for the text to be processed, after obtaining the first vector, the angle between the first vector and the first preset vector can be determined as the first rotation angle of the text line to be processed, so that the first rotation angle can accurately represent the text line rotation angle of the text line to be processed.

[0144] Step 83: Determine a first rotation direction of the text line to be processed based on a relative relationship between the center of the global region of the text line to be processed and the center of the first character region of the text line to be processed in a first preset direction.

[0145] The first preset direction refers to a preset reference direction.

[0146] In fact, text lines with different text arrangements correspond to different reference directions. Based on this, the embodiment of the present application also provides a possible implementation method for determining the first preset direction, which may specifically include: after obtaining the text arrangement of the text line to be processed, determining the first preset direction according to the text arrangement of the text line to be processed and the second mapping relationship. The second mapping relationship includes the correspondence between the text arrangement of the text line to be processed and the first preset direction.

[0147] The second mapping relationship is used to record the reference directions related to the global area center and the first character area center corresponding to different text arrangement methods; and the embodiment of the present application does not limit the second mapping relationship. For example, the second mapping relationship may include the correspondence between the horizontal and vertical directions, and the correspondence between the vertical and horizontal directions.

[0148] Based on the above-mentioned relevant content of the "first preset direction", it can be known that for the text line to be processed, if it is determined that the text arrangement method of the text line to be processed is horizontal, the vertical direction can be determined as the first preset direction; if it is determined that the text arrangement method of the text line to be processed is vertical, the horizontal direction can be determined as the first preset direction.

[0149] In addition, the first rotation direction may be used to describe the text line rotation direction of the text line to be processed.

[0150] In addition, the embodiment of the present application does not limit the implementation method of step 83. For ease of understanding, two scenarios are combined for explanation below.

[0151] Scenario 1: When the text arrangement of the text line to be processed is horizontal and the first preset direction is vertical, step 83 may specifically include: if the projection position of the global area center of the text line to be processed in the first preset direction is higher than the projection position of the first character area center of the text line to be processed in the first preset direction, then the first rotation direction of the text line to be processed is determined to be counterclockwise; if the projection position of the global area center of the text line to be processed in the first preset direction is lower than the projection position of the first character area center of the text line to be processed in the first preset direction, then the first rotation direction of the text line to be processed is determined to be clockwise.

[0152] It can be seen that Figure 5 As shown, if the text arrangement of the text line to be processed is horizontal (that is, the text line to be processed belongs to a horizontal text line), then after obtaining the global area center of the text line to be processed (such as Figure 5 "O1" in the text line to be processed) and the center of the first character area of ​​the text line to be processed (such as Figure 5After determining the relative relationship in the vertical direction between the "O3" in the figure, if the correlation indicates that the projection position of the global area center of the text line to be processed in the vertical direction is higher than the projection position of the first character area center of the text line to be processed in the vertical direction, it can be determined that the first rotation direction of the text line to be processed is counterclockwise; if the correlation indicates that the projection position of the global area center of the text line to be processed in the vertical direction is lower than the projection position of the first character area center of the text line to be processed in the vertical direction, it can be determined that the first rotation direction of the text line to be processed is clockwise.

[0153] It should be noted that if the text arrangement of the text line to be processed is horizontal, and the projection position of the global area center of the text line to be processed in the vertical direction coincides with the projection position of the first character area center of the text line to be processed in the vertical direction, it can be determined that the text line to be processed is in the positive direction or the text line to be processed is rotated 180°.

[0154] Scenario 2: When the text arrangement of the text line to be processed is vertical and the first preset direction is horizontal, step 83 may specifically include: if the projection position of the global area center of the text line to be processed in the first preset direction is to the right of the projection position of the first character area center of the text line to be processed in the first preset direction, then the first rotation direction of the text line to be processed is determined to be counterclockwise; if the projection position of the global area center of the text line to be processed in the first preset direction is to the left of the projection position of the first character area center of the text line to be processed in the first preset direction, then the first rotation direction of the text line to be processed is determined to be clockwise.

[0155] It can be seen that Figure 6 As shown, if the text arrangement of the text line to be processed is vertical (that is, the text line to be processed belongs to a vertical text line), then after obtaining the global area center of the text line to be processed (such as Figure 6 "O1" in the text line to be processed) and the center of the first character area of ​​the text line to be processed (such as Figure 6 After determining the relative relationship between the two characters "O3" in the horizontal direction, if the correlation indicates that the projection position of the global area center of the text line to be processed in the horizontal direction is to the right of the projection position of the first character area center of the text line to be processed in the horizontal direction, then it can be determined that the first rotation direction of the text line to be processed is counterclockwise; if the correlation indicates that the projection position of the global area center of the text line to be processed in the horizontal direction is to the left of the projection position of the first character area center of the text line to be processed in the horizontal direction, then it can be determined that the first rotation direction of the text line to be processed is clockwise.

[0156] It should be noted that if the text arrangement of the text line to be processed is vertical, and the projection position of the global area center of the text line to be processed in the horizontal direction coincides with the projection position of the first character area center of the text line to be processed in the horizontal direction, it can be determined that the text line to be processed is in the positive direction or the text line to be processed is rotated 180°.

[0157] Based on the relevant content of the above-mentioned step 83, it can be known that after obtaining the global area center of the text line to be processed and the first character area center of the text line to be processed, the relative relationship between the global area center of the text line to be processed and the first character area center of the text line to be processed in a first preset direction can be obtained first; and then, based on the relevant relationship, the first rotation direction of the text line to be processed is determined, so that the first rotation direction can accurately represent the rotation direction of the text line of the text to be processed.

[0158] Step 84: Generate a text line direction of the text line to be processed according to the first rotation angle of the text line to be processed and / or the first rotation direction of the text line to be processed.

[0159] As an example, if the text line direction includes the text line rotation direction and / or the text line rotation angle, step 84 may include step 841 and / or step 842:

[0160] Step 841: Determine the text line rotation direction of the text line to be processed according to the first rotation direction of the text line to be processed (eg, the first rotation direction of the text line to be processed may be directly determined as the text line rotation direction of the text line to be processed).

[0161] Step 842: Determine the text line rotation angle of the text line to be processed according to the first rotation angle of the text line to be processed (eg, the first rotation angle of the text line to be processed may be directly determined as the text line rotation angle of the text line to be processed).

[0162] Based on the relevant content of the above-mentioned step 84, it can be known that after obtaining the first rotation angle and / or the first rotation direction of the text line to be processed, the text line direction (especially, the text line rotation mode) of the text line to be processed can be generated according to the first rotation angle and / or the first rotation direction, so that the text line direction can accurately represent the text line rotation mode of the text line to be processed, thereby making the text line direction accurately represent the presentation mode of the text line to be processed in the image to be processed.

[0163] Based on the relevant content of the first example of S33 above, it can be known that for the text line to be processed, after obtaining the first character area center of the text line to be processed, the text line direction of the text line to be processed can be determined based on the relative position information between the global area center and the first character area center of the text line to be processed, so that the text line direction can accurately represent the presentation method of the text line to be processed in the image to be processed.

[0164] In the second example of S33, if the local area center of the text line to be processed includes the first character area center of the text line to be processed and the string local area center of the text line to be processed, then S33 may specifically include: determining the text line direction of the text line to be processed based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed and the string local area center of the text line to be processed.

[0165] The embodiment of the present application does not limit the above determination process. For example, in a possible implementation manner, if the text line direction includes a text line rotation mode and / or a text arrangement mode, S33 may specifically include S331 and / or S332:

[0166] S331: Determine the text arrangement of the text line to be processed based on the relative position information between the global area center of the text line to be processed and the character string local area center of the text line to be processed.

[0167] In an embodiment of the present application, after obtaining the global area center and the local area center of the string of the text line to be processed, the text arrangement of the text line to be processed can be determined based on the relative relationship (especially the relative relationship in position) between the global area center and the local area center of the string of the text line to be processed; and the determination process can specifically include: if the global area center of the text line to be processed coincides with the local area center of the string of the text line to be processed, then the text arrangement of the text line to be processed is determined to be vertical; if the global area center of the text line to be processed does not coincide with the local area center of the string of the text line to be processed, then the text arrangement of the text line to be processed is determined to be horizontal.

[0168] S332: Determine a text line rotation mode of the text line to be processed based on relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the character string local area center of the text line to be processed.

[0169] The embodiment of the present application does not limit S332. For example, S332 can be implemented in the following three ways.

[0170] Method 1: Determine the text line rotation method of the text line to be processed based on the relative position information between the global area center of the text line to be processed and the first character area center of the text line to be processed.

[0171] It should be noted that for the relevant content of method one, reference may be made to the relevant content of “the first example of S33” above.

[0172] Method 2: Determine the text line rotation method of the text line to be processed based on the relative position information between the center of the first character area of ​​the text line to be processed and the center of the character string local area of ​​the text line to be processed.

[0173] The embodiment of the present application does not limit the above determination process. For example, in a possible implementation manner, the determination process of the text line rotation mode of the text line to be processed may include steps 91 to 94:

[0174] Step 91: Directedly connect the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed to obtain a third vector.

[0175] It should be noted that the embodiment of the present application does not limit the connection method of the "directed connection" in step 91. For example, the center of the first character area of ​​the text line to be processed can be used as the starting point, and the center of the string local area of ​​the text line to be processed can be used as the end point for directed connection, so that the third vector becomes a directed line segment with the center of the first character area of ​​the text line to be processed as the starting point and the center of the string local area of ​​the text line to be processed as the end point. For another example, the center of the string local area of ​​the text line to be processed can be used as the starting point, and the center of the first character area of ​​the text line to be processed can be used as the end point for directed connection, so that the third vector becomes a directed line segment with the center of the string local area of ​​the text line to be processed as the starting point and the center of the first character area of ​​the text line to be processed as the end point.

[0176] Step 92: Determine the angle between the third vector and the third preset vector as the third rotation angle of the text line to be processed.

[0177] The third preset vector refers to a preset reference vector used to represent a directed connection between the center of the first character area and the center of the local area of ​​the character string.

[0178] In fact, text lines with different text arrangements correspond to different reference vectors. Based on this, the embodiment of the present application also provides a possible implementation method for determining the third preset vector, which may specifically include: determining the third preset vector according to the text arrangement of the text line to be processed and the third mapping relationship. The third mapping relationship includes the correspondence between the text arrangement of the text line to be processed and the third preset vector.

[0179] The third mapping relationship is used to record the reference vector corresponding to different text arrangements for representing the directed connection between the center of the first character area and the center of the local area of ​​the string; and the embodiment of the present application does not limit the third mapping relationship. For example, the third mapping relationship may include the correspondence between the horizontal row and the fourth standard vector, and the correspondence between the vertical row and the fifth standard vector.

[0180] The fourth standard vector refers to a standard vector used to represent a directed connection between the center of the first character area of ​​a horizontal text line and the center of the string local area; and the embodiment of the present application does not limit the fourth standard vector. For example, the fourth standard vector can be determined according to the directed connection between the center of the first character area of ​​a standard horizontal text line and the center of the string local area of ​​the standard horizontal text line (for example, Figure 4 As shown, the directed connection between the center of the first character area of ​​a standard horizontal text line and the center of the string local area of ​​the standard horizontal text line can be determined as the fourth standard vector. For another example, any parallel vector of the directed connection between the center of the first character area of ​​a standard horizontal text line and the center of the string local area of ​​the standard horizontal text line can be determined as the fourth standard vector).

[0181] The fifth standard vector refers to a standard vector used to represent the directed connection between the center of the first character area of ​​the vertical text line and the center of the local area of ​​the string; and the embodiment of the present application does not limit the fifth standard vector. For example, the fifth standard vector can be determined according to the directed connection between the center of the first character area of ​​the standard vertical text line and the center of the local area of ​​the string of the standard vertical text line (for example, Figure 4 As shown, the directed connection between the center of the first character area of ​​a standard vertical text line and the center of the string local area of ​​the standard vertical text line can be determined as the fifth standard vector. For another example, any parallel vector of the directed connection between the center of the first character area of ​​a standard vertical text line and the center of the string local area of ​​the standard vertical text line can be determined as the fifth standard vector).

[0182] Based on the above-mentioned relevant content of the "third preset vector", it can be known that for the text to be processed, if it is determined that the text arrangement mode of the text line to be processed is horizontal, the fourth standard vector can be determined as the third preset vector; if it is determined that the text arrangement mode of the text line to be processed is vertical, the fifth standard vector can be determined as the third preset vector.

[0183] In addition, the third rotation angle may be used to represent the text line rotation angle of the text line to be processed.

[0184] Based on the relevant content of step 92 above, it can be known that for the text to be processed, after obtaining the third vector, the angle between the third vector and the third preset vector can be determined as the third rotation angle of the text line to be processed, so that the third rotation angle can accurately represent the text line rotation angle of the text line to be processed.

[0185] Step 93: Determine a third rotation direction of the text line to be processed based on the relative relationship and position difference between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed in a third preset direction.

[0186] The third preset direction refers to a preset reference direction.

[0187] In fact, text lines with different text arrangements correspond to different reference directions. Based on this, the embodiment of the present application also provides a possible implementation method for determining the third preset direction, which may specifically include: determining the third preset direction according to the text arrangement of the text line to be processed and the fourth mapping relationship; wherein the fourth mapping relationship includes the correspondence between the text arrangement of the text line to be processed and the third preset direction.

[0188] The fourth mapping relationship is used to record the reference directions related to the global area center and the first character area center corresponding to different text arrangement methods; and the embodiment of the present application does not limit the fourth mapping relationship. For example, the fourth mapping relationship may include the correspondence between the horizontal row and the vertical direction, and the correspondence between the vertical row and the horizontal direction.

[0189] Based on the above-mentioned "third preset direction" related content, it can be known that for the text line to be processed, if it is determined that the text arrangement mode of the text line to be processed is horizontal, the vertical direction can be determined as the third preset direction; if it is determined that the text arrangement mode of the text line to be processed is vertical, the horizontal direction can be determined as the third preset direction.

[0190] In addition, the third rotation direction may be used to describe the text line rotation direction of the text line to be processed.

[0191] In addition, the embodiment of the present application does not limit the implementation method of step 93. For ease of understanding, two scenarios are combined for explanation below.

[0192] Scenario 1: When the text line to be processed is arranged in horizontal format and the third preset direction is vertical, step 93 may specifically include:

[0193] If the projection position of the center of the first character area of ​​the text line to be processed in the third preset direction is higher than the projection position of the center of the string local area of ​​the text line to be processed in the third preset direction, and the projection position difference between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed in the third preset direction is higher than the preset position difference, then it is determined that the third rotation direction of the text line to be processed is clockwise;

[0194] If the projection position of the center of the first character area of ​​the text line to be processed in the third preset direction is higher than the projection position of the center of the string local area of ​​the text line to be processed in the third preset direction, and the projection position difference between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed in the third preset direction is lower than the preset position difference, then it is determined that the third rotation direction of the text line to be processed is counterclockwise;

[0195] If the projection position of the center of the first character area of ​​the text line to be processed in the third preset direction is not higher than the projection position of the center of the string local area of ​​the text line to be processed in the third preset direction, the third rotation direction of the text line to be processed is determined to be counterclockwise.

[0196] Among them, the preset position difference value can be pre-set, and can also be determined based on the height difference in the vertical direction between the center of the first character area of ​​the positive direction text line corresponding to the text line to be processed and the center of the local area of ​​the character string of the positive direction text line (for example, the height difference in the vertical direction between the center of the first character area of ​​the positive direction text line corresponding to the text line to be processed and the center of the local area of ​​the character string of the positive direction text line is determined as the preset position difference value).

[0197] It can be seen that Figure 7 As shown, when the text arrangement of the text line to be processed is horizontal (that is, the text line to be processed belongs to a horizontal text line), after obtaining the center of the first character area of ​​the text line to be processed (such as Figure 7 "O3" in the text line to be processed) and the center of the local area of ​​the string of the text line to be processed (such as Figure 7 After determining the relative relationship and position difference between the two characters in the vertical direction, if the relative relationship and position difference indicate that the projection position of the center of the first character area of ​​the text line to be processed in the vertical direction is higher than the projection position of the center of the string local area of ​​the text line to be processed in the vertical direction by a distance greater than the preset position difference (such as Figure 7), then the third rotation direction of the text line to be processed is determined to be clockwise; if the relative relationship and the position difference indicate that the projection position of the center of the first character area of ​​the text line to be processed in the vertical direction is higher than the projection position of the center of the string local area of ​​the text line to be processed in the vertical direction by a distance less than the preset position difference, or the relative relationship and the position difference indicate that the projection position of the center of the first character area of ​​the text line to be processed in the vertical direction is not higher than (for example, lower than or equal to) the projection position of the center of the string local area of ​​the text line to be processed in the vertical direction, then the third rotation direction of the text line to be processed is determined to be counterclockwise.

[0198] It should be noted that if the text arrangement of the text line to be processed is horizontal, the projection position of the center of the first character area of ​​the text line to be processed in the vertical direction is higher than the projection position of the center of the local area of ​​the character string of the text line to be processed in the vertical direction, and the position difference between the projection position of the center of the first character area of ​​the text line to be processed in the vertical direction and the projection position of the center of the local area of ​​the character string of the text line to be processed in the vertical direction is equal to the preset position difference, then it can be determined that the text line to be processed is in the positive direction or the text line to be processed is rotated 180°.

[0199] Scenario 2: When the text arrangement of the text line to be processed is vertical and the third preset direction is horizontal, step 93 may specifically include: if the projection position of the center of the first character area of ​​the text line to be processed in the third preset direction is to the left of the projection position of the center of the local area of ​​the string of the text line to be processed in the third preset direction, then the third rotation direction of the text line to be processed is determined to be counterclockwise; if the projection position of the center of the first character area of ​​the text line to be processed in the third preset direction is to the right of the center of the local area of ​​the string of the text line to be processed, then the third rotation direction of the text line to be processed is determined to be clockwise.

[0200] Among them, because the local area center of the character string of the vertical text line coincides with the global area center of the vertical text line, the "process of determining the text line rotation method of the vertical text line based on the first character area center of the vertical text line and the local area center of the character string" is similar to the above "process of determining the text line rotation method of the vertical text line based on the global area center of the vertical text line and the first character area center".

[0201] It can be seen that Figure 8 As shown, if the text arrangement of the text line to be processed is vertical (that is, the text line to be processed belongs to a vertical text line), then after obtaining the center of the first character area of ​​the text line to be processed (such as Figure 8 "O3") and the center of the local area of ​​the string (such as Figure 8If the correlation relationship indicates that the projection position of the center of the first character area of ​​the text line to be processed in the horizontal direction is to the left of the projection position of the center of the string local area of ​​the text line to be processed in the horizontal direction, then the third rotation direction of the text line to be processed is determined to be counterclockwise; if the correlation relationship indicates that the projection position of the center of the first character area of ​​the text line to be processed in the horizontal direction is to the right of the projection position of the center of the string local area of ​​the text line to be processed in the horizontal direction, then the third rotation direction of the text line to be processed is determined to be clockwise.

[0202] It should be noted that if the text arrangement of the text line to be processed is vertical, and the projection position of the center of the first character area of ​​the text line to be processed in the horizontal direction coincides with the projection position of the center of the string local area of ​​the text line to be processed in the horizontal direction, then it can be determined that the text line to be processed is in the positive direction or the text line to be processed is rotated 180°.

[0203] Based on the relevant content of the above-mentioned step 93, it can be known that after obtaining the center of the first character area of ​​the text line to be processed and the center of the local area of ​​the string of the text line to be processed, the relative relationship between the center of the first character area of ​​the text line to be processed and the center of the local area of ​​the string of the text line to be processed in the first preset direction can be obtained first; and then, based on the relevant relationship, the third rotation direction of the text line to be processed is determined, so that the third rotation direction can accurately represent the rotation direction of the text line of the text to be processed.

[0204] Step 94: Generate a text line rotation mode for the text line to be processed according to the third rotation angle of the text line to be processed and / or the third rotation direction of the text line to be processed.

[0205] As an example, if the text line rotation mode includes the text line rotation direction and / or the text line rotation angle, step 94 may include step 941 and / or step 942:

[0206] Step 941: determine the text line rotation direction of the text line to be processed according to the third rotation direction of the text line to be processed (eg, the third rotation direction of the text line to be processed can be directly determined as the text line rotation direction of the text line to be processed).

[0207] Step 942: Determine the text line rotation angle of the text line to be processed according to the third rotation angle of the text line to be processed (eg, the third rotation angle of the text line to be processed may be directly determined as the text line rotation angle of the text line to be processed).

[0208] Based on the relevant content of the above-mentioned step 94, it can be known that in an embodiment of the present application, after obtaining the third rotation angle and the third rotation direction of the text line to be processed, the text line rotation direction and the text line rotation angle of the text line to be processed can be determined respectively based on the third rotation angle and the third rotation direction; and the text line rotation mode of the text line to be processed is determined based on the text line rotation direction of the text line to be processed and / or the text line rotation angle of the text line to be processed, so that the text line rotation mode can accurately represent the text line rotation direction and / or rotation angle of the text line to be processed, so that the text line rotation mode can accurately represent the presentation mode of the text line to be processed in the image to be processed.

[0209] Based on the relevant content of the above-mentioned method 2, it can be known that after obtaining the center of the first character area and the center of the local area of ​​the string of the text to be processed, the text line rotation method of the text line to be processed can be determined according to the relative position information between the center of the first character area and the center of the local area of ​​the string of the text to be processed, so that the text line rotation method can accurately represent the text line rotation information of the text line to be processed in the image to be processed, thereby making it possible to accurately represent the presentation method of the text line to be processed in the image to be processed based on the text line rotation method.

[0210] Method three: Determine the text line rotation method of the text line to be processed based on the relative position information between the global area center of the text line to be processed, the first character area center of the text line to be processed and the character string local area center of the text line to be processed.

[0211] The embodiment of the present application does not limit the above determination process. For example, in a possible implementation manner, the process of determining the text line rotation mode of the text line to be processed may include steps 101 to 103:

[0212] Step 101: Determine the text line rotation angle of the text line to be processed based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the character string local area center of the text line to be processed.

[0213] That is, for a text line to be processed, after obtaining the global area center, the first character area center and the string local area center of the text line to be processed, the text line rotation angle of the text line to be processed can be determined in the following three ways; and the three ways can be specifically:

[0214] The first method is to determine the text line rotation angle of the text line to be processed based on the relative position relationship between the global area center of the text line to be processed and the first character area center of the text line to be processed.

[0215] It should be noted that the first method can be implemented using the above steps 81 to 82, and it is only necessary to replace the "first rotation angle of the text line to be processed" in the above steps 81 to 82 with the "text line rotation angle of the text line to be processed".

[0216] The second method is to determine the text line rotation angle of the text line to be processed based on the relative position information between the center of the first character area of ​​the text line to be processed and the center of the character string local area of ​​the text line to be processed.

[0217] It should be noted that the second method can be implemented using the above steps 91 to 92, and it is only necessary to replace the "third rotation angle of the text line to be processed" in the above steps 91 to 92 with the "text line rotation angle of the text line to be processed".

[0218] The third method: determine the text line rotation angle of the text line to be processed based on the relative position information between the global area center of the text line to be processed, the first character area center of the text line to be processed and the character string local area center of the text line to be processed.

[0219] The embodiment of the present application does not limit the implementation of the third method. For example, it may specifically include steps 111 to 113:

[0220] Step 111: Determine a first rotation angle of the text line to be processed based on the relative positional relationship between the center of the global region of the text line to be processed and the center of the first character region of the text line to be processed.

[0221] It should be noted that step 111 can be implemented by using the above steps 81 to 82.

[0222] Step 112: Determine a third rotation angle of the text line to be processed based on the relative position information between the center of the first character area of ​​the text line to be processed and the center of the character string local area of ​​the text line to be processed.

[0223] It should be noted that step 113 can be implemented by using the above steps 91 to 92.

[0224] Step 113: Determine the text line rotation angle of the text line to be processed according to the first rotation angle of the text line to be processed and the third rotation angle of the text line to be processed.

[0225] The embodiment of the present application does not limit the implementation method of step 113. For example, step 113 may specifically be: determine the maximum value (or median value, or minimum value, or mode) of the first rotation angle of the text line to be processed and the third rotation angle of the text line to be processed as the text line rotation angle of the text line to be processed. For another example, step 113 may specifically be: determine the average value (or weighted average value) between the first rotation angle of the text line to be processed and the third rotation angle of the text line to be processed as the text line rotation angle of the text line to be processed.

[0226] Based on the relevant content of the above step 101, it can be known that after obtaining the global area center, the first character area center and the string local area center of the text line to be processed, the text line rotation angle of the text line to be processed can be determined according to the relative position information between at least two of the global area center, the first character area center and the string local area center of the text line to be processed, so that the text line rotation method of the text line to be processed can be determined based on the text line rotation angle.

[0227] Step 102: Determine the text line rotation direction of the text line to be processed based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the character string local area center of the text line to be processed.

[0228] That is, for the text line to be processed, after obtaining the global area center, the first character area center and the string local area center of the text line to be processed, the following four implementation methods can be used to determine the text line rotation direction of the text line to be processed; and the three implementation methods can be specifically:

[0229] A first implementation method: determining the text line rotation direction of the text line to be processed based on the relative position relationship between the global area center of the text line to be processed and the first character area center of the text line to be processed.

[0230] It should be noted that the first implementation mode can be implemented using the above step 83, and it is only necessary to replace the "first rotation direction of the text line to be processed" in the above step 83 with the "text line rotation direction of the text line to be processed".

[0231] A second implementation manner: determining the text line rotation direction of the text line to be processed based on the relative position information between the center of the first character area of ​​the text line to be processed and the center of the character string local area of ​​the text line to be processed.

[0232] It should be noted that the second implementation mode can be implemented using the above step 93, and it is only necessary to replace the "third rotation direction of the text line to be processed" in the above step 93 with the "text line rotation direction of the text line to be processed".

[0233] A third implementation method: determining the text line rotation direction of the text line to be processed based on the relative position information between the global area center of the text line to be processed, the first character area center of the text line to be processed and the character string local area center of the text line to be processed.

[0234] The embodiment of the present application does not limit the implementation of the fourth implementation. For example, it may specifically include steps 121 to 123:

[0235] Step 121: Determine a first rotation direction of the text line to be processed based on the relative position relationship between the global area center of the text line to be processed and the first character area center of the text line to be processed.

[0236] It should be noted that step 121 can be implemented by using the above step 83.

[0237] Step 122: Determine a third rotation direction of the text line to be processed based on the relative position information between the center of the first character area of ​​the text line to be processed and the center of the character string local area of ​​the text line to be processed.

[0238] It should be noted that step 123 can be implemented by using the above step 93.

[0239] Step 123: Determine the text line rotation direction of the text line to be processed according to the first rotation direction of the text line to be processed and the third rotation direction of the text line to be processed.

[0240] The embodiment of the present application does not limit the implementation method of step 123. For example, step 123 may specifically be: determining the maximum value (or median value, or minimum value, or mode) of the first rotation direction of the text line to be processed and the third rotation direction of the text line to be processed as the text line rotation direction of the text line to be processed. For another example, step 123 may specifically be: determining the average value (or weighted average value) between the first rotation direction of the text line to be processed and the third rotation direction of the text line to be processed as the text line rotation direction of the text line to be processed.

[0241] Based on the relevant content of the above step 102, it can be known that after obtaining the global area center, the first character area center and the string local area center of the text line to be processed, the text line rotation direction of the text line to be processed can be determined according to the relative position information between at least two of the global area center, the first character area center and the string local area center of the text line to be processed, so that the text line rotation method of the text line to be processed can be determined based on the text line rotation direction.

[0242] Step 103: Generate a text line rotation mode for the text line to be processed according to the text line rotation angle and the text line rotation direction of the text line to be processed.

[0243] In an embodiment of the present application, after obtaining the text line rotation angle and rotation direction of the text line to be processed, a text line rotation mode of the text line to be processed can be generated based on the text line rotation angle and rotation direction of the text line to be processed, so that the text line rotation mode can accurately represent the text line rotation direction and text line rotation angle of the text line to be processed, so that the text line rotation mode can accurately represent the text line rotation information of the text line to be processed in the image to be processed, and then based on the text line rotation mode, it is possible to accurately represent the presentation method of the text line to be processed in the image to be processed.

[0244] Based on the relevant content of the above-mentioned method three, it can be known that after obtaining the global area center, the first character area center and the string local area center of the text to be processed, the text line rotation method of the text line to be processed can be determined according to the relative position information between these three centers, so that the text line rotation method can accurately represent the text line rotation information of the text line to be processed in the image to be processed, thereby making it possible to accurately represent the presentation method of the text line to be processed in the image to be processed based on the text line rotation method.

[0245] Based on the relevant contents of S1 to S3 above, it can be known that after acquiring the image to be processed, the global area of ​​the text line to be processed and the local area of ​​the text line to be processed in the image to be processed are first determined, so that the global area of ​​the text line to be processed represents the area occupied by the text line to be processed in the image to be processed, and the local area of ​​the text line to be processed represents the area occupied by a preset part of the text line to be processed in the image to be processed; then the global area of ​​the text line to be processed is compared with the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed. It can be seen that since the global area and local area of ​​the text line to be processed can respectively represent the area occupied by the text line to be processed in the image to be processed and the area occupied by the preset part of the text line to be processed in the image to be processed, the global area and local area of ​​the text line to be processed can comprehensively represent the presentation method of the text line to be processed in the image to be processed, so that the text line direction of the text line to be processed determined based on the global area and local area of ​​the text line to be processed can accurately describe the presentation method of the text line to be processed in the image to be processed, so that the text line direction of the text line in the image can be accurately determined, which is beneficial to improve the character recognition accuracy of the text line in the image.

[0246] Based on the text line direction determination method provided by the above method embodiment, the embodiment of the present application also provides a text line direction determination device, which is explained and illustrated in conjunction with the accompanying drawings below.

[0247] Device Embodiment

[0248] For technical details of the text line direction determination device provided by the device embodiment, please refer to the above method embodiment.

[0249] See also Fig. 9 , which is a structural schematic diagram of a text line direction determination device provided in an embodiment of the present application.

[0250] The text line direction determination device 900 provided in the embodiment of the present application includes:

[0251] An image acquisition unit 901 is used to acquire an image to be processed;

[0252] The region determination unit 902 is used to determine the global region of the text line to be processed in the image to be processed and the local region of the text line to be processed; wherein the global region of the text line to be processed represents the region occupied by the text line to be processed in the image to be processed; and the local region of the text line to be processed represents the region occupied by a preset portion of the text line to be processed in the image to be processed;

[0253] The direction determining unit 903 is used to compare the positions of the global area of ​​the text line to be processed and the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed.

[0254] In a possible implementation, the area determination unit 902 is specifically used to: input the image to be processed into a pre-constructed mask map generation model to obtain a global mask map of the predicted text line corresponding to the image to be processed and a local mask map of the predicted text line corresponding to the image to be processed output by the mask map generation model; wherein the mask map generation model is constructed based on a sample image, a global mask map of the actual text line corresponding to the sample image, and a local mask map of the actual text line corresponding to the sample image; the global mask map of the predicted text line corresponding to the image to be processed includes a global mask area corresponding to the text line to be processed; the local mask map of the predicted text line corresponding to the image to be processed includes a local mask area corresponding to the text line to be processed; the global mask area corresponding to the text line to be processed in the global mask map of the predicted text line corresponding to the image to be processed is determined as the global area of ​​the text line to be processed; the local mask area corresponding to the text line to be processed in the local mask map of the predicted text line corresponding to the image to be processed is determined as the local area of ​​the text line to be processed.

[0255] In a possible implementation manner, the local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed.

[0256] In a possible implementation manner, the direction determining unit 903 is specifically configured to:

[0257] The text line direction of the text line to be processed is determined based on the relative position information between the global area center of the text line to be processed and the first character area center of the text line to be processed; wherein the global area center of the text line to be processed represents the center position of the global area of ​​the text line to be processed; and the first character area center of the text line to be processed represents the center position of the first character area of ​​the text line to be processed.

[0258] In a possible implementation manner, the text line direction includes a text line rotation direction and / or a text line rotation angle;

[0259] The process of determining the text line rotation direction of the text line to be processed includes: determining a first rotation direction of the text line to be processed according to a relative relationship between a global area center of the text line to be processed and a first character area center of the text line to be processed in a first preset direction; determining a text line rotation direction of the text line to be processed according to the first rotation direction of the text line to be processed;

[0260] The process of determining the text line rotation angle of the text line to be processed includes: making a directed connection between the global area center of the text line to be processed and the first character area center of the text line to be processed to obtain a first vector; determining the angle between the first vector and a first preset vector as the first rotation angle of the text line to be processed; and determining the text line rotation angle of the text line to be processed based on the first rotation angle of the text line to be processed.

[0261] In a possible implementation, the local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed and the string local area of ​​the text line to be processed; wherein, the string local area of ​​the text line to be processed is obtained by connecting the preset local areas of each character in the text line to be processed.

[0262] In a possible implementation, the direction determination unit 903 is specifically used to determine the text line direction of the text line to be processed based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the string local area center of the text line to be processed; wherein the global area center of the text line to be processed represents the center position of the global area of ​​the text line to be processed; the first character area center of the text line to be processed represents the center position of the first character area of ​​the text line to be processed; and the string local area center of the text line to be processed represents the center position of the string local area of ​​the text line to be processed.

[0263] In a possible implementation, the text line direction includes a text line rotation mode and / or a text arrangement mode;

[0264] The process of determining the text line rotation mode of the text line to be processed includes: determining the text line rotation mode of the text line to be processed according to the relative position information between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed;

[0265] The process of determining the character arrangement of the text line to be processed includes: determining the character arrangement of the text line to be processed based on the relative position information between the global area center of the text line to be processed and the character string local area center of the text line to be processed.

[0266] In a possible implementation manner, the text line rotation mode includes a text line rotation direction and / or a text line rotation angle;

[0267] The process of determining the text line rotation direction of the text line to be processed includes: determining the third rotation direction of the text line to be processed according to the relative relationship between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed in a third preset direction and the position difference; determining the text line rotation direction of the text line to be processed according to the third rotation direction of the text line to be processed;

[0268] The process of determining the text line rotation angle of the text line to be processed includes: making a directed connection between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed to obtain a third vector; determining the angle between the third vector and a third preset vector as the third rotation angle of the text line to be processed; and determining the text line rotation angle of the text line to be processed based on the third rotation angle of the text line to be processed.

[0269] In a possible implementation, if the text line to be processed is arranged horizontally, the local area of ​​the character string of the text line to be processed is obtained by connecting the bottom boundary areas of each character in the text line to be processed; if the text line to be processed is arranged vertically, the local area of ​​the character string of the text line to be processed is obtained by connecting the center areas of each character in the text line to be processed.

[0270] In a possible implementation, the process of determining the text arrangement of the text line to be processed includes: if the global area center of the text line to be processed coincides with the local area center of the character string of the text line to be processed, then determining that the text arrangement of the text line to be processed is vertical; if the global area center of the text line to be processed does not coincide with the local area center of the character string of the text line to be processed, then determining that the text arrangement of the text line to be processed is horizontal.

[0271] In a possible implementation, the text line direction includes at least one of a text line rotation direction, a text line rotation angle, or a text arrangement mode;

[0272] The process of determining the text line direction of the text line to be processed includes: determining the text line rotation angle of the text line to be processed based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the string local area center of the text line to be processed; determining the text line rotation direction of the text line to be processed based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the string local area center of the text line to be processed; determining the text arrangement of the text line to be processed based on the relative position information between the global area center of the text line to be processed and the string local area center of the text line to be processed.

[0273] Furthermore, an embodiment of the present application also provides a device, the device comprising a processor and a memory:

[0274] The memory is used to store computer programs;

[0275] The processor is used to execute any implementation of the text line direction determination method provided in the embodiments of the present application according to the computer program.

[0276] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute any implementation of the text line direction determination method provided in the embodiment of the present application.

[0277] Furthermore, an embodiment of the present application also provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any implementation of the text line direction determination method provided in the embodiment of the present application.

[0278] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0279] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as above in the preferred embodiment, it is not used to limit the present invention. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A method for determining the direction of a text line, characterized in that: The method comprises: Get the image to be processed; Determine a global area of ​​a text line to be processed in the image to be processed and a local area of ​​the text line to be processed; wherein the global area of ​​the text line to be processed represents an area occupied by the text line to be processed in the image to be processed; and the local area of ​​the text line to be processed represents an area occupied by a preset portion of the text line to be processed in the image to be processed; The global area of ​​the text line to be processed is compared with the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed; the process of determining the text line direction includes: matching the relative information between the global area of ​​the text line to be processed and the local area of ​​the text line to be processed, and at least one candidate standard relative information, and determining the text line direction corresponding to the successfully matched candidate standard relative information as the text line direction of the text line to be processed, wherein the relative information includes a relative relationship and a relative difference.

2. The method according to claim 1, characterized in that The process of determining the global area of ​​the text line to be processed and the local area of ​​the text line to be processed includes: Input the image to be processed into a pre-constructed mask map generation model, and obtain the predicted text line global mask map corresponding to the image to be processed and the predicted text line local mask map corresponding to the image to be processed output by the mask map generation model; wherein the mask map generation model is constructed according to a sample image, a global mask map of an actual text line corresponding to the sample image, and a local mask map of an actual text line corresponding to the sample image; the predicted text line global mask map corresponding to the image to be processed includes a global mask area corresponding to the text line to be processed; the predicted text line local mask map corresponding to the image to be processed includes a local mask area corresponding to the text line to be processed; Determine the global mask area corresponding to the text line to be processed in the predicted text line global mask map corresponding to the image to be processed as the global area of ​​the text line to be processed; The local mask area corresponding to the text line to be processed in the predicted text line local mask map corresponding to the image to be processed is determined as the local area of ​​the text line to be processed.

3. The method according to claim 1, characterized in that The local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed.

4. The method according to claim 3, characterized in that: The step of comparing the positions of the global area of ​​the text line to be processed with the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed includes: The text line direction of the text line to be processed is determined based on the relative position information between the global area center of the text line to be processed and the first character area center of the text line to be processed; wherein the global area center of the text line to be processed represents the center position of the global area of ​​the text line to be processed; and the first character area center of the text line to be processed represents the center position of the first character area of ​​the text line to be processed.

5. The method according to claim 4, characterized in that The text line direction includes the text line rotation direction and / or the text line rotation angle; The process of determining the text line rotation direction of the text line to be processed includes: Determine a first rotation direction of the text line to be processed according to a relative relationship between a global area center of the text line to be processed and a first character area center of the text line to be processed in a first preset direction; determine a text line rotation direction of the text line to be processed according to the first rotation direction of the text line to be processed; The process of determining the text line rotation angle of the text line to be processed includes: Perform a directed connection between the global area center of the text line to be processed and the first character area center of the text line to be processed to obtain a first vector; determine the angle between the first vector and the first preset vector as the first rotation angle of the text line to be processed; and determine the text line rotation angle of the text line to be processed based on the first rotation angle of the text line to be processed.

6. The method according to claim 1, characterized in that The local area of ​​the text line to be processed includes the first character area of ​​the text line to be processed and the string local area of ​​the text line to be processed; wherein the string local area of ​​the text line to be processed is obtained by connecting the preset local areas of each character in the text line to be processed.

7. The method according to claim 6, characterized in that The step of comparing the positions of the global area of ​​the text line to be processed with the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed includes: The text line direction of the text line to be processed is determined based on the relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed and the string local area center of the text line to be processed; wherein the global area center of the text line to be processed represents the center position of the global area of ​​the text line to be processed; the first character area center of the text line to be processed represents the center position of the first character area of ​​the text line to be processed; and the string local area center of the text line to be processed represents the center position of the string local area of ​​the text line to be processed.

8. The method according to claim 7, characterized in that The text line direction includes a text line rotation mode and / or a text arrangement mode; The process of determining the text line rotation mode of the text line to be processed includes: Determining a text line rotation mode of the text line to be processed according to relative position information between the center of the first character area of ​​the text line to be processed and the center of the character string local area of ​​the text line to be processed; The process of determining the character arrangement of the text line to be processed includes: The character arrangement of the text line to be processed is determined according to the relative position information between the global area center of the text line to be processed and the character string local area center of the text line to be processed.

9. The method according to claim 8, characterized in that The text line rotation mode includes a text line rotation direction and / or a text line rotation angle; The process of determining the text line rotation direction of the text line to be processed includes: Determine the third rotation direction of the text line to be processed according to the relative relationship between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed in a third preset direction and the position difference; determine the text line rotation direction of the text line to be processed according to the third rotation direction of the text line to be processed; The process of determining the text line rotation angle of the text line to be processed includes: Perform a directed connection between the center of the first character area of ​​the text line to be processed and the center of the string local area of ​​the text line to be processed to obtain a third vector; determine the angle between the third vector and the third preset vector as the third rotation angle of the text line to be processed; and determine the text line rotation angle of the text line to be processed based on the third rotation angle of the text line to be processed.

10. The method according to claim 7, characterized in that If the text line to be processed is arranged in horizontal arrangement, the character string local area of ​​the text line to be processed is obtained by connecting the bottom boundary areas of each character in the text line to be processed; If the characters of the text line to be processed are arranged in vertical arrangement, the character string local area of ​​the text line to be processed is obtained by connecting the central areas of the characters in the text line to be processed.

11. The method according to claim 10, characterized in that The process of determining the character arrangement of the text line to be processed includes: If the global area center of the text line to be processed coincides with the local area center of the character string of the text line to be processed, the text arrangement mode of the text line to be processed is determined to be vertical; if the global area center of the text line to be processed does not coincide with the local area center of the character string of the text line to be processed, the text arrangement mode of the text line to be processed is determined to be horizontal.

12. The method according to claim 7, characterized in that The text line direction includes at least one of a text line rotation direction, a text line rotation angle, or a text arrangement mode; The process of determining the text line direction of the text line to be processed includes: Determine the text line rotation angle of the text line to be processed according to relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the character string local area center of the text line to be processed; Determine the text line rotation direction of the text line to be processed according to relative position information between at least two of the global area center of the text line to be processed, the first character area center of the text line to be processed, and the character string local area center of the text line to be processed; The character arrangement of the text line to be processed is determined according to the relative position information between the global area center of the text line to be processed and the character string local area center of the text line to be processed.

13. A device for determining the direction of a text line, characterized in that: The device comprises: An image acquisition unit, used for acquiring an image to be processed; A region determination unit, used to determine a global region of a text line to be processed in the image to be processed and a local region of the text line to be processed; wherein the global region of the text line to be processed represents the region occupied by the text line to be processed in the image to be processed; and the local region of the text line to be processed represents the region occupied by a preset portion of the text line to be processed in the image to be processed; A direction determination unit, used for comparing the positions of the global area of ​​the text line to be processed with the local area of ​​the text line to be processed to obtain the text line direction of the text line to be processed; The direction determination unit is specifically used to match the relative information between the global area of ​​the text line to be processed and the local area of ​​the text line to be processed, as well as at least one candidate standard relative information, and determine the text line direction corresponding to the successfully matched candidate standard relative information as the text line direction of the text line to be processed, wherein the relative information includes relative relationship and relative difference.

14. A device, characterized in that The device comprises a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method of any one of claims 1 to 12 according to the computer program.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that When the computer program product is executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 12.

Images