Image orientation identification method, image orientation identification model, and storage medium

Abstract

To provide a method and model for identifying an image orientation, and a storage medium.SOLUTION: An image orientation identification method includes: detecting character regions from a target image to be identified, and obtaining position coordinates and region features for each character region; identifying image features in the target image of the character region, based on the position coordinates, to be merged with the region features, to generate first merge features of the character regions, and generating text row classification results of the character regions based on the first merge features; acquiring position features and form features for each character region, performing merge for the text row classification results based on the position features and the form features, and generating an orientation identification result of the target image. The invention improves accuracy of identifying image orientation.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to image processing technology, and more particularly to a method and model for identifying image orientation (also called an image orientation identifying device) and a storage medium. [Background technology]

[0002] In intelligent office work, character recognition, such as OCR (Optical Character Recognition), is often performed on document images scanned and uploaded by users. To ensure accurate recognition results, it is necessary to ensure that the image orientation is correct before recognition. However, the orientation of the image generated by a user's scan can be 0°, 90°, 180°, or 270° (where 0° is considered correct), depending on various factors. Therefore, the image must be rotated manually or by an algorithm to be correct before recognition can begin. Here, we assume that the character orientation is parallel to the long or short edge of the image, and slight tilts in the character orientation relative to the length or width of the image are not taken into account.

[0003] The goal of image orientation identification is to rotate the image in the positive direction. Currently, the main method for image orientation identification is to directly classify the image using a depth (deep) convolutional network. However, conventional depth (deep) convolutional networks extract features from the entire image, and are unable to represent the important role of the character portion within it. This introduces a lot of noise, which reduces the classification accuracy and results in low accuracy in image orientation identification. Therefore, a method to improve the accuracy of image orientation identification is needed. Summary of the Invention [Problem to be solved by the invention]

[0004] At least one embodiment of the present invention provides an image orientation identification method, model, and storage medium that can solve the problem of low accuracy in image orientation identification in the prior art. [Means for solving the problem]

[0005] In order to solve the above problems, first, a first aspect of the present invention is Detecting character regions from the target image to be identified, and acquiring position coordinates and region features for each of the character regions; Identifying image features in the target image of the character region based on the position coordinates, and fusing the image features with the region features to generate a first fusion feature of the character region, and generating a text line classification result of the character region based on the first fusion feature; The present invention provides an image orientation identification method, which includes obtaining positional features and morphological features for each character region, and fusing the text line classification results based on the positional features and morphological features to generate an orientation identification result for the target image.

[0006] Optionally, acquiring position features and shape features for each of the character regions, and fusing the text line classification results based on the position features and the shape features to generate an orientation identification result for the target image, inputting the position coordinates of each character region and the text line classification result into an image orientation identification module, and obtaining an orientation identification result of the target image output by the image orientation identification module; The image orientation identification module includes: For each character region, identify the position feature and the form feature based on the position coordinates, fuse the position feature and the form feature to generate a second fusion feature, and generate a weight for the character region based on the second fusion feature; Based on the weight of each character region, the text line classification results of the character region are fused to generate an orientation identification result for the target image.

[0007] Optionally, the position characteristics include a relative polar angle, which is a difference between a polar angle and a reference polar angle, and a polar diameter; The morphological characteristics include at least one of the height, width, area, contour length, and center of mass of the character region.

[0008] As an option, character regions are detected from the target image to be identified, and the position coordinates and region features of each character region are obtained. The method includes inputting the target image into a character region detection module, and obtaining position coordinates and region features of each of the character regions output by the character region detection module.

[0009] Optionally, identifying image features in the target image of the character region based on the position coordinates and fusing the image features with the region features to generate a first fusion feature of the character region, and generating a text line classification result of the character region based on the first fusion feature, The target image and the position coordinates and region features of the character region are input to a feature fusion and text line classification module, and a text line classification result of the character region is generated and output by the feature fusion and text line classification module.

[0010] Optionally, the image orientation identification method further comprises, before detecting a character region from the target image, pre-training the text region detection module using first training data including a plurality of first images in which text regions are labeled; inputting second training data, including a plurality of second images labeled with text line classification results, into the trained character region detection module; and pre-training the feature fusion and text line classification module using the position coordinates and region features of the character regions output by the character region detection module and the second training data; acquiring third training data including a plurality of third images with image orientations marked in the target region; inputting the third training data into the trained character region detection module; inputting the position coordinates and region features of the character region output by the character region detection module and the third training data into a trained feature fusion and text line classification module; and training the image orientation identification module using the text line classification result output by the feature fusion and text line classification module and the third training data.

[0011] In addition, the second aspect of the present invention is a character region detection module that detects character regions from a target image to be identified and acquires position coordinates and region features for each of the character regions; a feature fusion and text line classification module that identifies image features in the target image of the character region based on the position coordinates, fuses the image features with the region features to generate a first fusion feature for the character region, and generates a text line classification result for the character region based on the first fusion feature; and an image orientation identification module that acquires positional features and morphological features for each character region, and performs fusion on the text line classification results based on the positional features and morphological features to generate an orientation identification result for the target image.

[0012] Optionally, the image orientation identification module further comprises: Identifying the position feature and the form feature for each character region based on the position coordinates, fusing the position feature and the form feature of the character region to generate a second fusion feature, and generating a weight for the character region based on the second fusion feature; Based on the weight of each character region, the text line classification results of the character region are fused to generate an orientation identification result for the target image.

[0013] Optionally, the position characteristics include a relative polar angle, which is a difference between a polar angle and a reference polar angle, and a polar diameter; The morphological characteristics include at least one of the height, width, area, contour length, and center of mass of the character region.

[0014] Optionally, the character region detection module further generates an image feature map corresponding to the target image, detects the character region based on the image feature map, obtains the position coordinates of the character region, and extracts features corresponding to the character region from the image feature map to use them as region features of the character region.

[0015] Optionally, the image orientation discrimination model comprises: pre-training the text region detection module using first training data including a plurality of first images in which text regions are labeled; inputting second training data, including a plurality of second images labeled with text line classification results, into the trained character region detection module; and pre-training the feature fusion and text line classification module using the position coordinates and region features of the character regions output by the character region detection module and the second training data; acquiring third training data including a plurality of third images with image orientations marked in the target region; The method further includes a training module that inputs the third training data into the trained character region detection module, inputs the position coordinates and region features of the character region output by the character region detection module and the third training data into a trained feature fusion and text line classification module, and trains the image orientation identification module using the text line classification result output by the feature fusion and text line classification module and the third training data.

[0016] Furthermore, a third aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, the computer program being executed by a processor to realize the image orientation identification method described above.

[0017] A fourth aspect of the present invention provides a computer program product including computer instructions, the computer instructions being executed by a processor to implement the image orientation identification method described above. [Effects of the Invention]

[0018] The image orientation identification method, image orientation identification method model, and storage medium provided by embodiments of the present invention perform fusion on text line classification results for each character region in a target image to be identified based on the positional and morphological features of the character region. Compared to the prior art, embodiments of the present invention fully consider the positional and morphological features of the character region when fusion is performed on the text line classification results for each character region, thereby reducing or filtering noise text in the image and improving the accuracy of the image orientation identification results. Furthermore, embodiments of the present invention pre-train the character region detection module and the feature fusion and text line classification module using previously published training data for multiple regions, thereby reducing the requirements for model training data. Furthermore, by training the image orientation identification module using training data for the target region, the image orientation identification performance for the target region is improved. [Brief explanation of the drawings]

[0019] Other advantages and merits will become apparent from the detailed description of the best mode of implementation below. The drawings show preferred embodiments, but should not be construed as limiting the present invention. In the drawings, like parts are designated by like reference numerals. [Figure 1] FIG. 1 is a schematic diagram showing the structure of an image orientation identification model according to an embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart illustrating a method for identifying image orientation according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram illustrating an example of an image direction identification module according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of different character regions according to an embodiment of the present invention. [Figure 5] FIG. 5 is a schematic diagram showing another structure of an image direction identification model according to an embodiment of the present invention. [Figure 6] FIG. 6 is a schematic diagram showing another structure of an image orientation identification model according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] In order to clarify the technical problems, technical solutions, and advantages of the present invention, the present invention will be described in detail below with reference to the drawings and specific embodiments. In the following description, specific details of the specific arrangement and components are provided only to assist in a comprehensive understanding of the embodiments of the invention. Therefore, various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. For the sake of clarity, descriptions of known functions and structures will be omitted.

[0021] References throughout the specification to "one embodiment" or "one embodiment" mean that a particular feature, structure, or characteristic associated with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of "in one embodiment" or "in one embodiment" in various places in the specification do not necessarily refer to the same embodiment. However, the particular feature, structure, or characteristic may be combined in any manner in one or more embodiments, as appropriate.

[0022] In each embodiment of the present invention, the magnitude of the numbers of the following processes does not mean the order of execution. The execution order of each process is determined by its function and inherent logic, and does not imply any limitation on the implementation process of the embodiment of the present invention.

[0023] An embodiment of the present invention provides an image orientation identification method for identifying the orientation of an image to be identified. Here, the image orientation is typically one of 0°, 90°, 180°, and 270°. For example, 0° indicates that the image is upright, while 90°, 180°, and 270° indicate that the image has been rotated 90°, 180°, and 270° from 0°, respectively. In this embodiment of the present invention, an upright image indicates that the orientation of the characters in the image is consistent with the reading habits of characters. In some cases, an image may have multiple character orientations. In this case, the dominant character orientation in the image is used as the character orientation in the image. The dominant character is typically the text character with the largest area and the same orientation in the image.

[0024] Similarly, the orientation of a line of text in an image is typically either 0°, 90°, 180°, or 270°, where 0° represents the normal orientation, and 90°, 180°, and 270° represent lines of text rotated 90°, 180°, and 270° from 0°, respectively. In an embodiment of the present invention, a line of text being normal means that the character direction in the line of text conforms to reading habits.

[0025] In an embodiment of the present invention, an image orientation identification model is pre-trained. As shown in Figure 1, the image orientation identification model mainly includes a pre-trained character region detection module 11, a feature fusion and text line classification module 12, and an image orientation identification module 13. When identifying the orientation of a target image containing text (target image to be identified), the target image is input to the image orientation identification model, and the image orientation identification model outputs an image orientation identification result, for example, one of 0°, 90°, 180°, or 270°. The image orientation identification result may also be a probability value of the angle.

[0026] Specifically, the text region detection module 11 detects text regions in the target image to be identified and acquires the position coordinates and region features of the text regions. The position coordinates of the text region are expressed by the coordinates of a rectangle circumscribing the text region, for example, the coordinates of two diagonal corners (e.g., the upper left corner and the lower right corner) of the circumscribing rectangle.

[0027] The feature fusion and text line classification module 12 identifies image features of the target image of the character region based on the position coordinates of the character region, fuses them with the region features of the character region to generate a first fusion feature, and generates a text line classification result for the character region based on the first fusion feature. The method for fusing the image features and the region features may be, for example, feature vector splicing or weighted summation, and the embodiment of the present invention is not limited thereto. The classification result of the text line indicates the orientation of the text line, for example, 0°, 90°, 180°, or 270°.

[0028] The image orientation identification module 13 acquires the positional and morphological features of each character region, and performs fusion on the text line classification results for each character region based on the positional and morphological features to obtain the orientation identification result of the target image. The orientation identification result of the target image can be any of 0°, 90°, 180°, and 270°, or a probability value corresponding to the angle.

[0029] The image orientation identification method includes the following steps, as shown in FIG.

[0030] In step 21, a character region is detected from the target image to be identified, and the position coordinates and region features of the character region are obtained.

[0031] Here, the target image to be identified is input to the character region detection module 11, and the position coordinates and region features of each character region are obtained from the character region detection module 11. Specifically, the character region detection module generates an image feature map corresponding to the target image, detects character regions based on the image feature map, identifies the position coordinates of the character regions, and further extracts features corresponding to the character regions from the image feature map to use them as region features.

[0032] In step 22, based on the position coordinates of the character region, image features in the target image of the character region are identified and fused with the region features to generate a first fusion feature; and based on the first fusion feature, a text line classification result is generated.

[0033] Here, the target image, the position coordinates of the character region and the region features are input to the feature fusion and text line classification module 12, and the text line classification result of the character region is output by the feature fusion and text line classification module 12.

[0034] In step 23, the position feature and the shape feature are obtained for each character region, and the text line classification result is fused based on the position feature and the shape feature to form the orientation identification result of the target image.

[0035] Here, the position coordinates and text line classification results for each character region are input to the image orientation identification module 13, and the orientation identification result of the target image is generated and output from the image orientation identification module 13. The image orientation identification result is an angle corresponding to the direction, for example, 0°, 90°, 180°, or 270°. The image orientation identification result may also be a probability value of the above-mentioned angle.

[0036] Wherein, the image direction identification module is specifically used as follows: For each character region, Obtaining position features and shape features of the character region based on the position coordinates; fusing the position features and shape features to generate second fusion features; and generating weights of the character region based on the second fusion features; Based on the weights, the text line classification results are fused to generate a target image orientation identification result.

[0037] In an embodiment of the present invention, the position features include a relative polar angle, which is the difference between the polar angle and a predetermined reference polar angle, and a polar diameter. Specifically, the polar angle and polar diameter are those of a specific corner (e.g., the upper left corner or the lower right corner) of a circumscribing rectangle of the character region, or those of a specific point (e.g., the center, center of mass, or other point) in the character region. Note that in an embodiment of the present invention, the polar angle and polar diameter of the character region may be those of a polar angle range and a polar diameter range spanned by the character region. For example, the minimum or maximum polar angle of the character region, or the minimum and maximum polar diameter of the character region. The shape features of the character region include at least one of the height, width, area, contour length, and center of mass (e.g., the position coordinates of the center of mass) of the character region.

[0038] In an embodiment of the present invention, an attention mechanism is used to direct the model to focus on the positional and shape features of character regions in an image, thereby improving image orientation identification performance. Figure 3 shows an example of an image orientation identification module. In this example, a Transformer model is used for the image orientation identification module. Shape features include the height, width, and area of ​​character regions. Considering that conventional XY coordinate systems cannot accommodate image features that are as consistent as possible even when the image is rotated at various angles, an embodiment of the present invention uses polar coordinate-based position coding to represent the positional features of character regions. The positional characteristics of character regions include the polar radius and polar angle of the character region. The polar radius is coded using a normalized coding method, and the polar angle can be expressed as the difference (relative polar angle) between the polar angle of each character region (token) and the polar angle (reference polar angle) of the CLS flag.

[0039] Through the above steps, the embodiment of the present invention performs fusion on the text line classification results of the text regions in the target image, which are identified based on the positional and morphological features of the text regions. By fully considering the positional and morphological features of the text regions during the fusion of the text line classification results, noise text in the image can be reduced or filtered, and the accuracy of the image orientation identification result can be improved.

[0040] For example, in Figure 4, there are multiple character regions surrounded by red dotted lines. The text lines in these character regions are oriented in different directions. For example, the text lines in character regions 41-43 are oriented at 0°, while the text lines in character regions 44-45 are oriented at 270°. In this embodiment of the present invention, in step 23, a pre-trained image orientation identification module merges the text line classification results for each character region based on the positional and morphological features of each character region. During training, when learning the weights for the character regions, the image orientation identification module assigns a relatively low weight to character regions located near the image edges. Therefore, because the positions of dashed-line frames 44 and 45 are biased toward the edge of the image, the text line classification results within the dashed-line frames are assigned a relatively low weight during merging, thereby reducing the influence of noise text in this area. In contrast, dashed-line frame 41 has a large area and is therefore assigned a relatively high weight. Furthermore, dashed-line frame 42 is assigned a relatively high weight because it is closer to the image center. In this way, by appropriately weighting the text line classification results of the character regions according to the positional and morphological features of the character regions, the accuracy of the image orientation identification results is improved.

[0041] The learning of the image orientation identification model according to the embodiment of the present invention will be described below.

[0042] In an embodiment of the present invention, the image orientation identification model is pre-trained before step 21. Here, a large number of publicly labeled datasets already exist for text region detection and text line classification tasks in various fields. Therefore, to reduce the need for data representation, the text region detection module and the feature fusion and text line classification module are pre-trained based on existing datasets from many fields. Furthermore, the image orientation identification module can be precisely trained (fine-tuned) using training data from a specific field to improve its image orientation identification performance. This training method in an embodiment of the present invention reduces the need for training data from a specific field, and pre-training with a large number of general-purpose datasets allows the model to acquire better a priori knowledge. Specifically, the training includes the following steps:

[0043] (1) Pre-training a text region detection module using first training data including a plurality of first images of marked text regions (a plurality of first images with marked text regions), where the first training data is obtained from public data collections in multiple fields.

[0044] (2) Second training data including a plurality of second images of the marked text line classification results (a plurality of second images marked with the text line classification results) is input to a pre-trained character region detection module, and the position coordinates and region features of the character regions output by the trained character region detection module and the second training data are used to pre-train a feature fusion and text line classification module, where the second training data may be obtained from public data collections in multiple fields.

[0045] (3) Obtain third training data for the target region. The third training data includes a plurality of third images with image orientations marked (a plurality of third images with image orientations marked). Here, the third training data is usually training data for a specific field. Then, the third training data is input to a pre-trained text region detection module. The position coordinates of the text region, region features, and the third training data output by the text region detection module are input to a pre-trained feature fusion and text line classification module. The text line classification results output by the feature fusion and text line classification module and the third training data are used to train the image orientation identification module.

[0046] As can be seen from the above, the embodiment of the present invention uses previously published training data from multiple fields to pre-train the character region detection module and the feature fusion and text line classification module, thereby reducing the requirement for model training data, and by training the image orientation identification module using training data from the target region, the image orientation identification performance of the target region is improved.

[0047] In view of the above, an embodiment of the present invention further provides an image orientation identification model for implementing the above method. Figure 1 shows an example of the configuration of the image orientation identification model.

[0048] The image orientation identification module 13 further comprises: For each character region, identify and fuse position features and shape features based on the position coordinates to generate a second fusion feature, and generate a weight for the character region based on the second fusion feature; The text line classification results are fused based on the weights of the character regions to obtain the orientation identification results of the target image.

[0049] Here, the position features of the character region include a relative polar angle, which is the difference between the polar angle and the reference polar angle, and a polar diameter.

[0050] The shape characteristics of the character region include at least one of the height, width, area, contour length, and center of mass of the character region.

[0051] Furthermore, the character area detection module 11 generates an image feature map corresponding to the target image, detects character areas based on the image feature map, identifies the position coordinates of the character areas, and extracts features corresponding to the character areas from the image feature map to use them as area features of the character areas.

[0052] Another image orientation identification model according to an embodiment of the present invention is also provided. As shown in Figure 5, the image orientation identification model further includes a training module 14. The training module 14 includes: pre-training a character region detection module using first training data including a plurality of first images marked with character regions; inputting second training data including a plurality of second images marked in the text line classification results (a plurality of second images marked with the text line classification results) into a pre-trained character region detection module, and pre-training a feature fusion and text line classification module using the position coordinates and region features of the character regions output by the character region detection module and the second training data; acquiring third training data including a plurality of third images with image orientations marked in the target region; The third training data is input to a pre-trained character region detection module, and the position coordinates and region features of the character region output by the character region detection module and the third training data are input to a pre-trained feature fusion and text line classification module, and the text line classification result output by the feature fusion and text line classification module and the third training data are used to train an image orientation identification module.

[0053] 6 shows a hardware configuration of an image orientation identification model according to an embodiment of the present invention. As shown in FIG. 6, the image orientation identification model 600 includes a processor 602 and a memory 604 storing computer program instructions. By executing the computer program instructions on the processor 602, the image orientation identification model 600 can: Detecting character regions from the target image to be identified, and acquiring position coordinates and region features for each of the character regions; Identifying image features in the target image of the character region based on the position coordinates, and fusing the image features with the region features to generate a first fusion feature of the character region, and generating a text line classification result of the character region based on the first fusion feature; A step of obtaining position features and shape features for each character region, and fusing the text line classification results based on the position features and shape features to generate an orientation identification result for the target image is realized.

[0054] Furthermore, as shown in FIG. 6, the model training device 600 further includes a network interface 601 , an input device 603 , a hard disk 605 , and a display device 606 .

[0055] These interfaces and devices are interconnected via a bus architecture. The bus architecture can be a bus and bridge that can include any number of interconnections. Specifically, various circuits, such as one or more central processing units (CPUs) and / or graphics processing units (GPUs), represented by processor 602, and one or more memories, represented by memory 604, are connected. The bus architecture can also connect various other circuits, such as peripherals, voltage regulators, and power management circuits. It should be understood that a bus architecture is used to enable communication between these components. The bus architecture can include a data bus, as well as a power bus, a control bus, and a status signal bus, all of which are known in the art and will not be described in detail herein.

[0056] The network interface 601 can be connected to a network (for example, the Internet, a local area network, etc.), can receive data such as training data from the network, and can store the received data in the hard disk 605 .

[0057] The input device 603 can receive various commands input by an operator and send them to the processor 602 for execution. The input device 603 can include a keyboard or a click device (e.g., a mouse, a trackball, a touch panel, or a touch screen).

[0058] The display device 606 can display the results obtained by the processor 602 executing the instructions, such as displaying the progress of training a model.

[0059] Memory 604 stores programs and data necessary to run the operating system, as well as data such as intermediate results of calculations by processor 602 .

[0060] In embodiments of the invention, memory 604 may be volatile or non-volatile, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), such as an external cache. Memory 604 of the apparatus and methods described herein may include, but is not limited to, these and any other suitable types of memory.

[0061] In some implementations, memory 604 stores an operating system (OS) 6041 and application programs (APP) 6042 as executable modules or data structures, or a subset or extension thereof.

[0062] Among them, the operating system 6041 includes various system programs, such as a framework layer, a core library layer, and a driver layer, and is used to realize various core business and hardware-based tasks. The application program 6042 includes various application programs, such as a web browser, and is used to realize various application tasks. The program that executes the method according to this embodiment is included in the application program 6042.

[0063] The method according to the embodiment of the present invention is applied to or implemented by the processor 602. The processor 602 is a type of integrated circuit chip capable of processing signals. During implementation, each step of the method is implemented by an integrated logic circuit in hardware or instructions in software form in the processor 602. The processor 602 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing each method, step, and logic box disclosed in the embodiment of the present invention. The general-purpose processor may be a microprocessor or any other general processor. Each step of the method according to the embodiment of the present invention may be implemented by a decoder, which is hardware, or may be implemented by a combination of hardware and software that can be implemented in the decoder. The software modules are stored in a storage medium well-known in the art, such as a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The processor 602 reads information from the memory 604, which comprises a storage medium on which the software is stored, and configures the hardware to implement the steps of the above-described method.

[0064] The above-described embodiments may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof, wherein, for a hardware implementation, the processing unit may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units that perform the functions of the present invention, or a combination thereof.

[0065] Regarding software implementation, the above technology is realized by modules (e.g., processes, functions, etc.) that implement the functions described above. The software code is stored in a memory and executed by a processor. The memory may be implemented inside or outside the processor.

[0066] Specifically, when the computer program is executed by the processor 602, the step of inputting the position coordinates of each character region and the text line classification result into an image orientation identification module and obtaining the orientation identification result of the target image output by the image orientation identification module is further realized.

[0067] The image orientation identification module includes: For each character region, identify the position feature and the form feature based on the position coordinates, fuse the position feature and the form feature to generate a second fusion feature, and generate a weight for the character region based on the second fusion feature; Based on the weight of each character region, the text line classification results of the character region are fused to generate an orientation identification result for the target image.

[0068] Specifically, when the computer program instructions are executed by the processor 602, the position characteristics include a relative polar angle, which is a difference between a polar angle and a reference polar angle, and a polar diameter; The shape feature is at least one of the height, width, area, contour length, and center of mass of the character region.

[0069] In particular, by having the processor 602 execute the computer program instructions, the steps of inputting the target image into a character region detection module and obtaining the position coordinates and region features of each of the character regions output by the character region detection module are further realized.

[0070] In particular, by causing the processor 602 to execute the computer program instructions, the target image and the position coordinates and region features of the character region are input to a feature fusion and text line classification module, and a text line classification result of the character region is generated and output by the feature fusion and text line classification module.

[0071] Specifically, the computer program instructions are executed by the processor 602 to: pre-training the text region detection module using first training data including a plurality of first images in which text regions are labeled; inputting second training data, including a plurality of second images labeled with text line classification results, into the trained character region detection module; and pre-training the feature fusion and text line classification module using the position coordinates and region features of the character regions output by the character region detection module and the second training data; acquiring third training data including a plurality of third images with image orientations marked in the target region; The method further implements a step of inputting the third training data into the trained character region detection module, inputting the position coordinates and region features of the character region output by the character region detection module and the third training data into a trained feature fusion and text line classification module, and training the image orientation identification module using the text line classification result output by the feature fusion and text line classification module and the third training data.

[0072] The present invention also provides a computer-readable storage medium, which stores computer program instructions, and which can be executed by a processor to realize the image orientation identification method and achieve the same effects. To avoid repetition, detailed descriptions are omitted. The computer-readable storage medium can be, for example, a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk.

[0073] An embodiment of the present invention further provides a computer program, which includes computer program instructions, and when the computer program instructions are executed by a processor, the image orientation identification method is realized and the same effects can be achieved. To avoid repetition, the description will be omitted here.

[0074] As used herein, the words "comprises," "comprises in," or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a set of elements includes only those elements and also other elements not expressly stated or inherent in such process, method, article, or apparatus. Unless further limited, an element qualified by the word "comprises" does not exclude the presence of further identical elements in the process, method, article, or apparatus that includes that element.

[0075] From the above-described embodiments, it is clear that the methods of the embodiments can be realized through software and a necessary general-purpose hardware platform. Of course, they can also be realized through hardware, but in many cases the former is a superior embodiment. Based on this understanding, the present invention, in its essential form or in its contribution to the prior art, is expressed in the form of a software product. The computer program product is stored in a storage medium (e.g., ROM / RAM, disk, optical disk) and includes computer program instructions that cause a terminal (which may be a mobile phone, computer, server, air conditioner, network device, etc.) to execute the methods described in each embodiment of the present invention.

[0076] Although the present invention has been described above with reference to the drawings, the present invention is not limited to the specific embodiments. The specific embodiments described above are merely illustrative and are not limiting. Changes and modifications may be made without departing from the spirit and scope of the present invention, and all such changes and modifications are within the scope of the present invention.

Claims

1. 1. A computer-implemented method for identifying image orientation, comprising: Detecting character regions from the target image to be identified, and acquiring position coordinates and region features for each of the character regions; Identifying image features in the target image of the character region based on the position coordinates, and fusing the image features with the region features to generate a first fusion feature of the character region, and generating a text line classification result of the character region based on the first fusion feature; obtaining position features and shape features for each character region, and fusing the text line classification results based on the position features and shape features to generate an orientation identification result for the target image; 1. A method for identifying image orientation.

2. obtaining positional features and morphological features for each character region, and fusing the text line classification results based on the positional features and morphological features to generate an orientation identification result for the target image; inputting the position coordinates of each character region and the text line classification result into an image orientation identification module, and obtaining an orientation identification result of the target image output by the image orientation identification module; The image orientation identification module includes: For each character region, identify the position feature and the form feature based on the position coordinates, fuse the position feature and the form feature to generate a second fusion feature, and generate a weight for the character region based on the second fusion feature; Performing a fusion process on the text line classification results of each character region based on the weight of the character region to generate an orientation identification result for the target image.

2. The image orientation identification method according to claim 1.

3. the position characteristics include a relative polar angle, which is a difference between a polar angle and a reference polar angle, and a polar diameter; The morphological features include at least one of the height, width, area, contour length, and center of mass of the character region.

3. The method of claim 2, wherein the image orientation is identified by the image orientation detecting means.

4. Detecting character regions from the target image to be identified and acquiring position coordinates and region features for each character region is inputting the target image into a character region detection module, and obtaining position coordinates and region features of each of the character regions output by the character region detection module; 3. The method of claim 2, wherein the image orientation is identified by the image orientation detecting means.

5. Identifying image features in the target image of the character region based on the position coordinates, and fusing the image features with the region features to generate a first fusion feature of the character region, and generating a text line classification result of the character region based on the first fusion feature, inputting the target image, and the position coordinates and region features of the character region into a feature fusion and text line classification module, and generating a text line classification result for the character region output by the feature fusion and text line classification module; 5. The method of claim 4, wherein the image orientation is identified by the image orientation identifying method.

6. Before detecting a character region from the target image, pre-training the text region detection module using first training data including a plurality of first images in which text regions are labeled; inputting second training data, including a plurality of second images labeled with text line classification results, into the trained character region detection module; and pre-training the feature fusion and text line classification module using the position coordinates and region features of the character regions output by the character region detection module and the second training data; acquiring third training data including a plurality of third images with image orientations marked in the target region; inputting the third training data into the trained character region detection module, and inputting the position coordinates and region features of the character region output by the character region detection module and the third training data into a trained feature fusion and text line classification module, and training the image orientation identification module using the text line classification result output by the feature fusion and text line classification module and the third training data; 6. The method of claim 5, wherein the image orientation is identified by the image orientation identifying method.

7. a character region detection module that detects character regions from a target image to be identified and acquires position coordinates and region features for each of the character regions; a feature fusion and text line classification module that identifies image features in the target image of the character region based on the position coordinates, fuses the image features with the region features to generate a first fusion feature of the character region, and generates a text line classification result of the character region based on the first fusion feature; an image orientation identification module for acquiring position features and morphological features for each of the character regions, and fusing the text line classification results based on the position features and the morphological features to generate an orientation identification result for the target image; An image orientation identification device characterized by:

8. The image orientation identification module further comprises: Identifying the position feature and the form feature for each character region based on the position coordinates, fusing the position feature and the form feature of the character region to generate a second fusion feature, and generating a weight for the character region based on the second fusion feature; Performing a fusion process on the text line classification results of each character region based on the weight of the character region to generate an orientation identification result for the target image.

8. The image orientation identification device according to claim 7.

9. the position characteristics include a relative polar angle, which is a difference between a polar angle and a reference polar angle, and a polar diameter; The morphological features include at least one of the height, width, area, contour length, and center of mass of the character region.

9. The image orientation identification device according to claim 8.

10. The character region detection module further generating an image feature map corresponding to the target image, detecting the character region based on the image feature map, acquiring position coordinates of the character region, and extracting features corresponding to the character region from the image feature map to set them as region features of the character region; 9. The image orientation identification device according to claim 8.

11. pre-training the text region detection module using first training data including a plurality of first images in which text regions are labeled; inputting second training data, including a plurality of second images labeled with text line classification results, into the trained character region detection module; and pre-training the feature fusion and text line classification module using the position coordinates and region features of the character regions output by the character region detection module and the second training data; acquiring third training data including a plurality of third images with image orientations marked in the target region; a training module for inputting the third training data into the trained character region detection module, and inputting the position coordinates and region features of the character region output by the character region detection module and the third training data into a trained feature fusion and text line classification module, and training the image orientation identification module using the text line classification result output by the feature fusion and text line classification module and the third training data; 11. The image orientation identification device according to claim 10.

12. A program for causing a computer to execute the image orientation identification method according to any one of claims 1 to 6.

13. A computer-readable storage medium storing the program according to claim 12.

Images