Image recognition method, device, equipment and readable storage medium

Abstract

The application discloses an image recognition method, device and equipment and a readable storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first table image, wherein the first table image comprises at least one first text region; determining first position information of each first text region; determining table attributes of each first text region according to the first position information of each first text region, wherein the table attributes comprise at least one of row attributes or column attributes; performing text recognition on each first text region to obtain a text recognition result; and acquiring a table recognition result corresponding to the first table image according to the text recognition result of each first text region and the table attributes. According to the table structure restored according to the position information of each text region in the table image, the application has high accuracy, can accurately put the text recognition result of the text region into the corresponding cell, and improves the accuracy of the table recognition result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image recognition method, apparatus, device, and readable storage medium. Background Technology

[0002] With the advancement of digitalization, much data enters digital devices in the form of image data. When image data contains text, Optical Character Recognition (OCR) technology is typically used to recognize the text in the image data.

[0003] Taking a table image containing a table as an example, in related technologies, OCR technology is used to first identify the row and column lines in the table image to restore the table structure, that is, to obtain each cell in the table image. Then, the plain text data in each cell of the table is identified and placed in the corresponding cell to obtain a table recognition result that contains both table structure and text.

[0004] The above method only applies to table images with row and column borders. By recognizing these borders, the table structure can be reconstructed. For table images without row and column borders, the reconstructed table structure using this method is inaccurate, resulting in plain text data in each cell not being accurately placed within its corresponding cell, thus leading to low accuracy in the table recognition results. Summary of the Invention

[0005] This application provides an image recognition method, apparatus, device, and readable storage medium, which can improve the accuracy of table recognition results. The technical solution is as follows:

[0006] On one hand, embodiments of this application provide an image recognition method, the method comprising:

[0007] Obtain a first table image, wherein the first table image contains at least one first text area;

[0008] Determine the first position information of each first text region;

[0009] Based on the first position information of each first text region, the table attributes of each first text region are determined, and the table attributes include at least one of row attributes or column attributes;

[0010] Text recognition is performed on each of the first text regions to obtain text recognition results;

[0011] Based on the text recognition results of each first text region and the table attributes, obtain the table recognition result corresponding to the first table image.

[0012] In one possible implementation, the first table image includes at least one of a shopping list image, a ticket image, an examination report image, and a laboratory report image.

[0013] On the other hand, embodiments of this application provide an image recognition device, the device comprising:

[0014] A first acquisition module is used to acquire a first table image, wherein the first table image contains at least one first text area;

[0015] The first determining module is used to determine the first position information of each first text region;

[0016] The second determining module is used to determine the table attributes of each of the first text regions based on the first position information of each of the first text regions. The table attributes include at least one of row attributes or column attributes.

[0017] The recognition module is used to perform text recognition on each of the first text regions and obtain text recognition results;

[0018] The second acquisition module is used to acquire the table recognition result corresponding to the first table image based on the text recognition result of each first text region and the table attributes.

[0019] In one possible implementation, the first acquisition module is configured to acquire a second table image, the second table image containing a tilted table image region; identify at least one second text region contained in the table image region; determine the rotation angle of the table image region based on the second position information of each second text region; perform rotation correction on the table image region according to the rotation angle of the table image region, and take the corrected table image region as the target table image region; and acquire the first table image based on the target table image region.

[0020] In one possible implementation, the first acquisition module is configured to determine the rotation angle of each second text region based on the second position information of each second text region; and to take the rotation angle of the middle position in the sorted rotation angles of each second text region as the rotation angle of the table image region, or to take the average value of the rotation angles of each second text region as the rotation angle of the table image region.

[0021] In one possible implementation, the first acquisition module is configured to acquire key content input by the user; in response to the target table image region not containing the key content, rotate and correct the target table image region according to a target rotation angle, and acquire the first table image based on the corrected target table image region; in response to the target table image region containing the key content, acquire the first table image based on the target table image region.

[0022] In one possible implementation, each of the second text regions corresponds one-to-one with each of the first text regions. The first determining module is used to determine the center position information of the table image region; determine the position rotation information of each second text region based on the center position information and the rotation angle of the table image region; and determine the first position information of the first text region corresponding to any second text region based on the position rotation information and the second position information of any second text region.

[0023] In one possible implementation, if the table attributes include row attributes or column attributes, the second determining module is configured to repeatedly perform the following operations based on each of the first text regions until the table attributes of each of the first text regions are determined:

[0024] Based on the first position information of each first text region, a third text region is determined from each first text region, and the table attributes of the third text region are determined.

[0025] Calculate the table attribute similarity between the third text region and each other text region, where any other text region is a first text region other than the third text region among the first text regions.

[0026] Based on the similarity of table attributes between the third text region and each of the other text regions, a target text region is determined from each of the other text regions, and the table attributes of the target text region are determined to be the table attributes of the third text region.

[0027] In one possible implementation, the second determining module is used for any of the following:

[0028] For any other text region, using the vertex position information of the third text region and the vertex position information of any other text region, calculate the table attribute similarity between the third text region and any other text region;

[0029] Using the center position information of the third text region and the center position information of any other text region, calculate the table attribute similarity between the third text region and any other text region.

[0030] In one possible implementation, the first table image includes at least one of a shopping list image, a ticket image, an examination report image, and a laboratory report image.

[0031] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory, the memory storing at least one instruction, the at least one instruction, when executed by the processor, causing the computer device to implement any of the image recognition methods described above.

[0032] On the other hand, a computer-readable storage medium is also provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to enable a computer to implement any of the image recognition methods described above.

[0033] On the other hand, a computer program or computer program product is also provided, wherein the computer program or computer program product stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to implement any of the above-described image recognition methods.

[0034] The technical solution provided in this application has at least the following beneficial effects:

[0035] The technical solution provided in this application is to determine the table attributes of each text region in a table image based on the position information of each text region. These table attributes include at least one of row or column attributes, thereby reconstructing the table structure. Since the text regions in a table image are arranged in a regular and orderly manner, regardless of whether the table image has row and column borders, the table structure reconstructed based on the position information of each text region is highly accurate. Further, the text in each text region is identified, and based on the text recognition results of each text region and the table attributes, the corresponding table recognition result is obtained. This achieves accurate placement of the text recognition results of the text regions into the corresponding cells, improving the accuracy of the table recognition results. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the implementation environment of an image recognition method provided in an embodiment of this application;

[0038] Figure 2 This is a flowchart of an image recognition method provided in an embodiment of this application;

[0039] Figure 3 This is a schematic diagram of a tilted table image area provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of a rotated and corrected table image area provided in an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of the location of a first text region provided in an embodiment of this application;

[0042] Figure 6 This is a schematic diagram illustrating how to determine the table attributes of a first text area according to an embodiment of this application;

[0043] Figure 7 This is a flowchart of another image recognition method provided in the embodiments of this application;

[0044] Figure 8 This is a flowchart of a two-stage row alignment operation provided in an embodiment of this application;

[0045] Figure 9 This is a structural diagram of an image recognition device provided in an embodiment of this application;

[0046] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application;

[0047] Figure 11 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0049] Figure 1 This is a schematic diagram of the implementation environment of an image recognition method provided in an embodiment of this application, such as... Figure 1 The implementation environment shown includes a computer device 11, and the image recognition method in this embodiment can be executed by the computer device 11. Exemplarily, the computer device 11 may include at least one of a terminal device or a server.

[0050] The terminal device can be at least one of a smartphone, game console, desktop computer, tablet computer, and laptop computer. The server can be a single server, a server cluster consisting of multiple servers, or any of a cloud computing platform and virtualization center; this application embodiment does not limit this. The server can communicate with the terminal device via a wired or wireless network. The server can have functions such as data processing, data storage, and data transmission and reception; this application embodiment does not limit this.

[0051] The image recognition method in this application can be implemented based on artificial intelligence (AI) technology. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have the functions of perception, reasoning, and decision-making.

[0052] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0053] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0054] The data involved in the image recognition method of this application embodiment can be realized based on cloud technology. The data processing / data calculation involved in the implementation of the solution can be realized based on cloud computing. The data involved in the implementation of the solution can be stored on the blockchain.

[0055] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0056] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0057] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0058] Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.

[0059] Cloud computing refers to the delivery and usage model of IT infrastructure, meaning obtaining necessary resources in an on-demand and easily scalable manner through a network. In a broader sense, cloud computing also refers to the delivery and usage model of services, meaning obtaining necessary services in an on-demand and easily scalable manner through a network. These services can be IT and software related, internet-related, or other services. Cloud computing is a product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0060] With the development of the internet, real-time data streams, and the diversification of connected devices, as well as the demands for search services, social networks, mobile commerce, and open collaboration, cloud computing has rapidly developed. Unlike previous parallel distributed computing, cloud computing will fundamentally revolutionize the entire internet model and enterprise management model.

[0061] Based on the above implementation environment, this application provides an image recognition method to... Figure 2 The flowchart shown in this embodiment of the application illustrates an image recognition method. This method can be implemented by... Figure 1 The computer device 11 in the middle performs the operation. For example... Figure 2 As shown, the method includes steps S21-S25.

[0062] Step S21: Obtain a first table image, the first table image containing at least one first text area.

[0063] In this embodiment of the application, the first table image can be a table image with row and column frames or a table image without row and column frames. For example, the first table image includes at least one of a shopping list image, a ticket image, a medical report image, and a laboratory test report image. The shopping list image is an image of a shopping ticket; the ticket image includes, but is not limited to, images of train tickets, bus tickets, airplane tickets, and ship tickets; the medical report image is an image of a hospital medical report; and the laboratory test report image is an image of a hospital laboratory test report.

[0064] In one possible implementation, the first table image is either the original image taken by the user or a rotated and corrected version of the original image taken by the user. Due to factors such as the shooting angle and unevenness of the original paper, the original image taken by the user may contain tilted table image areas. That is, the original image taken by the user contains at least one of tilted or non-tilted table image areas. When the original image taken by the user contains tilted table image areas, this embodiment uses the original image taken by the user as the second table image and performs rotational correction on the tilted table image areas in the second table image.

[0065] One possible implementation of this application embodiment is to obtain a first table image, including: obtaining a second table image, the second table image containing a tilted table image region; identifying at least one second text region contained in the table image region; determining the rotation angle of the table image region according to the second position information of each second text region; performing rotation correction on the table image region according to the rotation angle of the table image region, and taking the corrected table image region as the target table image region; and obtaining the first table image based on the target table image region.

[0066] The second table image includes, but is not limited to, the original image taken by the user, and includes at least one tilted table image area. In practical applications, the second table image may also include at least one untilted table image area.

[0067] For any tilted table image region, at least one dark region within that tilted table image region is identified using OCR technology. This dark region is a region containing at least one character, and is thus the second text region. In other words, at least one second text region within the tilted table image region is identified using OCR technology. In this embodiment, the shape of the second text region is not limited; for example, the second text region is rectangular.

[0068] Once any second text region is identified, its second position information is determined based on the set coordinate axes. Specifically, when the second text region is rectangular, the second position information is the coordinates of at least three vertices of the second text region; or the second position information is the lengths of two sides of the second text region and the coordinates of one vertex; or the second position information is the coordinates of two diagonal vertices of the second text region.

[0069] In this embodiment, for any tilted table image region, the rotation angle of the tilted table image region is calculated based on the second position information of each second text region within that tilted table image region. In one possible implementation, determining the rotation angle of the table image region based on the second position information of each second text region includes: determining the rotation angle of each second text region based on its second position information; using the rotation angle of the second text regions that is in the middle position in the sequence as the rotation angle of the table image region, or using the average value of the rotation angles of the second text regions as the rotation angle of the table image region.

[0070] For any given second text region, the minimum bounding rectangle of that second text region is determined based on its second position information, thereby determining the rotation angle of that second text region. For example, the second position information of any second text region consists of the coordinates of its four vertices. The minimum bounding rectangle is calculated using OpenCV (a software that provides algorithms for image processing and computer vision), and the rotation angle of the given second text region is obtained from OpenCV.

[0071] For any tilted table image region, the rotation angle of that tilted table image region is determined based on the rotation angles of each second text region within that tilted table image region. For example, the rotation angles of each second text region within the tilted table image region are sorted, and the rotation angle at the middle position in the sorting is taken as the rotation angle of that tilted table image region; alternatively, the average rotation angle of each second text region within the tilted table image region is calculated, and the calculated average value is taken as the rotation angle of that tilted table image region.

[0072] Furthermore, based on the rotation angle of any tilted table image region, the tilted table image region is rotated and corrected, and the corrected table image region is taken as the target table image region. If the second table image contains one tilted table image region, the target table image region is taken as the first table image; if the second table image contains at least two tilted table image regions, the target table image regions corresponding to each of the at least two tilted table image regions are stitched together, and the stitched image is taken as the first table image; if the second table image contains both tilted and untilted table image regions, the target table image region and the untilted table image region are stitched together, and the stitched image is taken as the first table image region.

[0073] like Figure 3 and Figure 4 As shown, Figure 3 This is a schematic diagram of a tilted table image area provided in an embodiment of this application. Figure 4 This is a schematic diagram of a rotated and corrected table image area provided in an embodiment of this application. For Figure 3 The tilted table image region is shown. Based on the coordinates of the four vertices of the second text region indicated by label 31 in the tilted table image region, the rotation angle of the second text region indicated by label 31 is obtained using OpenCV. Similarly, based on the coordinates of the four vertices of the second text region indicated by label 32 in the tilted table image region, the rotation angle of the second text region indicated by label 32 is obtained using OpenCV. The average rotation angle between the rotation angles of the second text regions indicated by label 31 and label 32 is calculated, and this average rotation angle is used as... Figure 3 The rotation angle of the tilted table image area is shown, and according to... Figure 3 The rotation angle of the tilted table image region shown is used to correct the rotation of this tilted table image region. The corrected table image region is as follows: Figure 4 As shown, the rotated and corrected table image area is the target table image area.

[0074] In one possible implementation, obtaining a first table image based on a target table image region includes: obtaining key content input by the user; in response to the target table image region not containing key content, rotating and correcting the target table image region according to a target rotation angle, and obtaining a first table image based on the corrected target table image region; and in response to the target table image region containing key content, obtaining a first table image based on the target table image region.

[0075] In this embodiment, in response to user input of key content, and the target table image area does not contain that key content, it indicates that the target table image area is not fully corrected. The target table image area is then rotated and corrected according to the target rotation angle, and a first table image is obtained based on the rotated and corrected target table image area. For example, if the target rotation angle is 180 degrees, meaning the target table image area does not contain the user-inputted key content, it indicates that the target table image area is upside down. The target table image area is then rotated 180 degrees to obtain a corrected target table image area, and the first table image is obtained based on this corrected target table image area.

[0076] In practical applications, it can be further determined whether there is key content in the target table image area after rotation correction. If there is key content in the target table image area after rotation correction, the first table image is obtained based on the target table image area after rotation correction. If there is no key content in the target table image area after rotation correction, it means that the key content entered by the user is not the content in the target table image area. Then, a prompt message is generated and displayed. For example, the prompt message is a message prompting the user to re-enter the key content, or a message prompting to change the table image.

[0077] If a user inputs key information and the target table image area contains that key information, it indicates that the target table image area is fully corrected, and the first table image can be obtained based on the target table image area.

[0078] In this embodiment, when obtaining a first table image based on a rotationally corrected target table image region or a target table image region, if the second table image contains a tilted table image region, then the rotationally corrected target table image region or the target table image region is used as the first table image; if the second table image contains at least two tilted table image regions, then the target table image regions or rotationally corrected target table image regions corresponding to each of the at least two tilted table image regions are stitched together, and the stitched image is used as the first table image; if the second table image contains a tilted table image region and a non-tilted table image region, then the target table image region and the non-tilted table image region are stitched together, or the rotationally corrected target table image region and the non-tilted table image region are stitched together, and the stitched image is used as the first table image region.

[0079] Step S22: Determine the first position information of each first text region.

[0080] In one possible implementation, OCR technology is used to identify at least one dark region in the first table image, where each dark region contains at least one character, and the dark region is the first text region. That is, at least one first text region in the first table image is identified using OCR technology. In this embodiment, the shape of the first text region is not limited; for example, the first text region is rectangular. After identifying any first text region, its first position information can be determined based on a set coordinate axis.

[0081] In another possible implementation, each second text region corresponds one-to-one with each first text region. Determining the first position information of each first text region includes: determining the center position information of the table image region; determining the position rotation information of each second text region based on the center position information and the rotation angle of the table image region; and determining the first position information of the first text region corresponding to any second text region based on the position rotation information and the second position information of any second text region.

[0082] In this embodiment, for any tilted table image region, the center position information of any tilted table image region can be determined based on the set coordinate axes. Based on the center position information and rotation angle of any tilted table image region, the position rotation information of each second text region is calculated using the affine transformation algorithm in OpenCV. The position rotation information of each second text region is the same, consisting of a rotation matrix, as shown in the following formula:

[0083]

[0084]

[0085]

[0086] in, This is the rotation matrix, which contains the position and rotation information of the second text region. α The coefficients of the first rotation matrix are... β The coefficients of the second rotation matrix, Let x be the x-coordinate of the center position information of any tilted table image region. Let y be the ordinate of the center position information of any tilted tabular image region. The coefficients of the third rotation matrix are set based on human experience; for example, The value is 1. Let be the rotation angle of any tilted table image region.

[0087] In this embodiment of the application, based on the position rotation information and second position information of any second text region, the first position information of the first text region corresponding to the any second text region is calculated according to the following formula:

[0088]

[0089]

[0090] in, Let x be the x-coordinate of the first position information of the first text region corresponding to any second text region. Let y be the ordinate of the first position information of the first text region corresponding to any second text region. x Let x be the x-coordinate of the second position information of any second text region. y Let x be the x-coordinate of the second position information of any second text region. Given a 2x3 rotation matrix, This is the value at row 0, column 0 of the rotation matrix, as mentioned earlier. α , This is the value of the first instance in the 0th row of the rotation matrix, as mentioned earlier. β , This is the value of the second instance in the 0th row of the rotation matrix, as mentioned earlier. , This is the value of the 0th instance in the 1st row of the rotation matrix, which is the value mentioned earlier. β , This is the value of the first instance in the first row of the rotation matrix, as mentioned earlier. α , This is the value of the second instance in the first row of the rotation matrix, as mentioned earlier. .

[0091] For example, the second position information of the second text region is the horizontal and vertical coordinates of the four vertices of the second text region, and the first position information of the first text region is the horizontal and vertical coordinates of the four vertices of the first text region. According to the above formula, the horizontal and vertical coordinates of the four vertices of the transformed first text region are calculated based on the horizontal and vertical coordinates of the four vertices of the second text region and the rotation matrix.

[0092] Step S23: Determine the table attributes of each first text region based on the first position information of each first text region. The table attributes include at least one of row attributes or column attributes.

[0093] In practical applications, due to factors such as rotation correction and paper folding during user photography, after determining the first position information of each first text region, the position of the first text region may not be uniform. That is, after identifying the position of each first text region, the rows and columns of the first text region are not regular. Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the location of a first text region according to an embodiment of this application. Figure 5 The document includes four first text areas, designated as first text areas 1-4. In practical applications, it is often assumed that first text areas 1, 3, and 4 are on the same line. However, due to paper folding, rotation correction, or other reasons, first text area 2 may partially overlap with first text areas 1 and 3, making them appear horizontally similar. In reality, first text area 2 is not on the same line as first text areas 1 and 3; rather, first text area 2 is typically on the line preceding first text areas 1 and 3. Based on these reasons, this application provides a method for determining the table attributes of each first text area based on its first position information, as detailed below:

[0094] In one possible implementation, if the table attributes include row attributes or column attributes, the table attributes of each first text region are determined based on the first position information of each first text region, including: repeatedly performing the table attribute determination operation based on each first text region until the table attributes of each first text region are determined.

[0095] The table attribute determination operation includes, but is not limited to: determining a third text region from each first text region based on the first position information of each first text region, and determining the table attribute of the third text region; calculating the table attribute similarity between the third text region and each other text region, wherein any other text region is a first text region other than the third text region among the first text regions; determining a target text region from each other text region based on the table attribute similarity between the third text region and each other text region, and determining the table attribute of the target text region as the table attribute of the third text region.

[0096] In this embodiment, a chained search algorithm is used to determine the table attributes of each first text region. Specifically, the first position information of each first text region is sorted to obtain the sequence number of each first text region. During each table attribute determination operation, the first text region with the same sequence number as the number of table attribute determination operations is identified from each first text region. This identified first text region is designated as the third text region. Based on the table attributes of the already determined text regions, the table attributes of the third text region are determined. The table attribute similarity between the third text region and each of the other text regions is calculated. Based on the table attribute similarity between the third text region and each of the other text regions, a target text region is identified from each of the other text regions, and the table attributes of the target text region are determined as the table attributes of the third text region.

[0097] In practical applications, the first text regions can be sorted based on the sum of the horizontal and vertical coordinates of the target vertices of each first text region. For example, the first text regions 1-9 can be sorted based on the sum of the horizontal and vertical coordinates of the top left vertices of the first text regions 1-9.

[0098] This application provides a specific example, such as... Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the determination of table attributes for a first text region according to an embodiment of this application. After sorting the first position information of each first text region, the following is obtained: Figure 6 The sequence numbers of each first text region shown indicate that the current text region set includes... Figure 6 The first text region is defined as 1-9. During the initial table attribute determination process, first text region 1 is identified as the third text region from the first text regions 1-9, and its row attribute is determined to be the first row. The table attribute similarity between first text region 1 and first text regions 2-9 is calculated. Based on the table attribute similarity between first text region 1 and first text regions 2-9, the target text region is determined from first text regions 2-9, i.e., first text region 2 is determined as the target text region, and its row attribute is determined to be the first row.

[0099] In the second table attribute determination process, first text region 2 is determined as the third text region from the first text regions 1-9. The table attribute similarity between first text region 2 and first text regions 1 and 3-9 is calculated. Based on the table attribute similarity between first text region 2 and first text regions 1 and 3-9, the target text regions are determined from first text regions 1 and 3-9. That is, first text regions 1 and 3 are determined as target text regions. The row attribute of first text regions 1 and 3 is determined as the first row. This process is repeated until the row attributes of all first text regions are determined.

[0100] In practical applications, during each table attribute determination operation, the table attribute similarity between the third text region and other text regions with undetermined table attributes can be calculated. Based on this similarity, the target text region is determined from these undetermined text regions, and its table attribute is set to the table attribute of the third text region. It is understandable that in any given table attribute determination operation, there may be instances where the target text region cannot be determined from the undetermined text regions.

[0101] For example, such as Figure 6 In the second table attribute determination operation, first text region 2 is determined as the third text region from the first text regions 1-9. Since the table attribute of first text regions 1 and 2 was already determined as the first row in the first table attribute determination operation, the other text regions whose table attributes have not been determined, besides the third text region, include first text regions 3-9. The table attribute similarity between first text region 2 and first text regions 3-9 is calculated. Based on the table attribute similarity between first text region 2 and first text regions 3-9, the target text region is determined from first text regions 3-9, that is, first text region 3 is determined as the target text region, and the row attribute of first text region 3 is determined as the first row.

[0102] In the third table attribute determination operation, text region 3 is identified as the third text region from text regions 1-9. Since the table attribute of text regions 1, 2, and 3 has been determined to be the first row in the previous two table attribute determination operations, the other text regions with undetermined table attributes, excluding the third text region, include text regions 4-9. The table attribute similarity between text region 3 and text regions 4-9 is calculated. Based on the table attribute similarity between text region 3 and text regions 4-9, the target text region cannot be determined from text regions 4-9. In other words, the table attribute of text regions 4-9 is not the first row. At this point, the result of the third table attribute determination operation leads to the fourth table attribute determination operation.

[0103] In the fourth table attribute determination operation, first text region 4 is determined as the third text region from first text regions 1-9. Since the table attribute of first text regions 1, 2, and 3 has already been determined as the first row in the first three table attribute determination operations, the other text regions whose table attributes have not been determined, excluding the third text region, include first text regions 5-9. The table attribute of first text region 4 is determined as the second row. The table attribute similarity between first text region 4 and first text regions 5-9 is calculated. Based on the table attribute similarity between first text region 4 and first text regions 5-9, the target text region is determined from first text regions 5-9, that is, first text region 5 is determined as the target text region, and the row attribute of first text region 5 is determined as the second row. This process continues until the row attributes of all first text regions are determined.

[0104] In one possible implementation, the table attribute similarity between the third text region and each of the other text regions is calculated by any of the following: for any other text region, the table attribute similarity between the third text region and any other text region is calculated using the vertex position information of the third text region and the vertex position information of any other text region; the table attribute similarity between the third text region and any other text region is calculated using the center position information of the third text region and the center position information of any other text region.

[0105] In this embodiment of the application, table attribute similarity includes at least one of row attribute similarity and column attribute similarity.

[0106] In one possible implementation, when calculating the line attribute similarity between the third text region and any other text region, the line attribute similarity between the third text region and any other text region is calculated using the top (or bottom) vertex position information of the third text region and the top (or bottom) vertex position information of any other text region. The top vertex position information includes the coordinates of the top-left and top-right vertexes, the bottom vertex position information includes the coordinates of the bottom-left and bottom-right vertexes, each coordinate includes an x-coordinate and a y-coordinate, and the line attribute similarity includes at least one of the first line attribute similarity and the second line attribute similarity.

[0107] In practice, the absolute value between the top-right vertex coordinate of the third text region and the top-left vertex coordinate of any other text region is calculated, and this absolute value is used as the first-row attribute similarity between the third text region and any other text region. Similarly, the absolute value between the bottom-right vertex coordinate of the third text region and the bottom-left vertex coordinate of any other text region is calculated, and this absolute value is used as the second-row attribute similarity between the third text region and any other text region. This process can be represented by the following formula:

[0108]

[0109]

[0110] in, This represents the similarity of the attributes in the first row. This represents the similarity of the attributes in the second row. abs The symbol for the absolute value function is... The y-coordinate of the top right vertex of the third text region. The y-coordinate of the bottom right vertex of the third text region. Let y be the ordinate of the top-left vertex of any other text region. Let be the y-coordinate of the bottom left vertex of any other text region.

[0111] Of course, in actual implementation, the absolute value between the top-left vertex coordinate of the third text region and the top-left vertex coordinate of any other text region can also be used as the first-row attribute similarity between the third text region and any other text region. Alternatively, the absolute value between the top-left vertex coordinate of the third text region and the top-right vertex coordinate of any other text region can be used as the first-row attribute similarity between the third text region and any other text region. Line attribute similarity: The absolute value between the bottom left vertices of the third text region and the bottom left vertices of any other text region is used as the second line attribute similarity between the third text region and any other text region; or, the absolute value between the bottom left vertices of the third text region and the bottom right vertices of any other text region is used as the second line attribute similarity between the third text region and any other text region; or, the absolute value between the bottom right vertices of the third text region and the bottom right vertices of any other text region is used as the second line attribute similarity between the third text region and any other text region.

[0112] In this embodiment, if the similarity of the first row attributes is less than or equal to the first row attribute similarity threshold, and / or the similarity of the second row attributes is less than or equal to the second row attribute similarity threshold, then any other text region is determined as the target text region. The first row attribute similarity threshold and the second row attribute similarity threshold can be the same value or different values, and are determined based on human experience.

[0113] In another possible implementation, the line attribute similarity between the third text region and any other text region is calculated using the center position information of the third text region and the center position information of any other text region. Here, the center position information refers to the center coordinates, which include the center x-coordinate and the center y-coordinate.

[0114] In practice, the absolute value between the center y-coordinate of the third text region and the center y-coordinate of any other text region is calculated, and this absolute value is used as the line attribute similarity between the third text region and any other text region. If the line attribute similarity is less than or equal to the third line attribute similarity threshold, then any other text region is determined as the target text region.

[0115] In another possible implementation, when calculating the column attribute similarity between the third text region and any other text region, the first column attribute similarity between the third text region and any other text region is calculated using the left vertex position information of the third text region and the left vertex position information of any other text region, and / or, the second column attribute similarity between the third text region and any other text region is calculated using the right vertex position information of the third text region and the right vertex position information of any other text region. Wherein, the left vertex position information includes the coordinates of the top-left vertex and the bottom-left vertex, the right vertex position information includes the coordinates of the top-right vertex and the bottom-right vertex, each coordinate includes an x-coordinate and a y-coordinate, and the column attribute similarity includes at least one of the first column attribute similarity and the second column attribute similarity.

[0116] In actual implementation, the absolute value between the left vertex x-coordinate of the third text region and the left vertex x-coordinate of any other text region is calculated, and the calculated absolute value is used as the first column of attribute similarity between the third text region and any other text region; the absolute value between the right vertex x-coordinate of the third text region and the right vertex x-coordinate of any other text region is calculated, and the calculated absolute value is used as the second column of attribute similarity between the third text region and any other text region.

[0117] In this embodiment, if the similarity of the first column attributes is less than or equal to the first column attribute similarity threshold, and / or the similarity of the second column attributes is less than or equal to the second column attribute similarity threshold, then any other text region is determined as the target text region. The first column attribute similarity threshold and the second column attribute similarity threshold can be the same value or different values, and are determined based on human experience.

[0118] In another possible implementation, the column attribute similarity between the third text region and any other text region is calculated using the center position information of the third text region and the center position information of any other text region. Here, the center position information refers to the center coordinates, which include the center x-coordinate and the center y-coordinate.

[0119] In practice, the absolute value between the center x-coordinate of the third text region and the center x-coordinate of any other text region is calculated. This absolute value is used as the third column attribute similarity threshold between the third text region and any other text region. If the column attribute similarity is less than or equal to the third column attribute similarity threshold, then any other text region is determined as the target text region.

[0120] Step S24: Perform text recognition on each first text region to obtain the text recognition result.

[0121] In this embodiment, text recognition is performed on any first text region to obtain the text recognition result for that first text region. The method of performing text recognition on the first text region is not limited; for example, OCR technology or a text recognition model can be used to perform text recognition on the first text region.

[0122] Step S25: Based on the text recognition results of each first text region and the table attributes, obtain the table recognition results corresponding to the first table image.

[0123] By using the method of the embodiments of this application, the text recognition result and table attribute of any first text region can be determined, and the text recognition result of any first text region can be arranged according to the table attribute of the first text region to obtain the table recognition result corresponding to the first table image.

[0124] The technical solution provided in this application embodiment determines the table attributes of each text region based on the position information of each text region contained in the table image. The table attributes include at least one of row attributes or column attributes, thereby restoring the table structure. Since the text regions in the table image are arranged in a regular and orderly manner, whether the table image has row and column line frames or not, the table structure restored based on the position information of each text region in the table image is highly accurate. Further, the text in each text region is identified. Based on the text recognition results of each text region and the table attributes, the table recognition result corresponding to the table image is obtained, which realizes the accurate placement of the text recognition results of the text regions into the corresponding cells, thereby improving the accuracy of the table recognition results.

[0125] The above describes in detail the image recognition method of this application from the perspective of method steps. The following will provide a detailed explanation using a specific scenario. In this scenario, the second table image is a lab report image taken by the user, and the lab report image contains a tilted lab report table image area, with the table attribute being row attribute. For example... Figure 7 As shown, Figure 7 This is a flowchart of another image recognition method provided in an embodiment of this application. The method includes steps S71-S78.

[0126] Step S71: Obtain the image of the test report taken by the user. The test report image contains a tilted test report table image area.

[0127] Step S72: Identify the various text regions contained in the image area of ​​the test report form.

[0128] Step S73: Determine the rotation angle of the test report image area based on the position information of each text area, rotate and correct the test report image area according to the rotation angle, and use the corrected test report image area as the target image area.

[0129] Step S74: Obtain the key information input by the user.

[0130] If the target table image area does not contain key content, then step S75 is executed, the target table image area is rotated 180 degrees, and a first table image is obtained based on the corrected target table image area, and then steps S76 and S77 are executed; if the target table image area contains key content, and a first table image is obtained based on the target table image area, then step S78 is executed, the first table image is obtained based on the target table image area, and then steps S76 and S77 are executed.

[0131] Step S76: Determine the row attributes of each text region based on the position information of each text region in the first table image.

[0132] Step S77: Perform text recognition on each text region to obtain text recognition results, and integrate the text recognition results and row attributes of each text region into a table recognition result of the test report image.

[0133] In this embodiment, the implementation of steps S71-S75 can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0134] In step S76, when determining the row attributes of each text region in the first table image, the row attribute determination operation is divided into two row alignment operations. Through these two row alignment operations, the row attributes of the text regions can be accurately determined. Figure 8 As shown, Figure 8 This is a flowchart of a two-stage row alignment operation provided in an embodiment of this application. Specifically, step S81 is executed first to sort the position information of each text region in the first table image to obtain the sequence number of each text region. Then, the first row alignment operation is performed. The first row alignment operation includes steps S82-S84.

[0135] In the first line alignment operation, step S82 is executed first to determine the text region with the same sequence number as the number of iterations from each text region. This determined text region is designated as the third text region. Based on the line attributes of the determined text regions, the line attributes of the third text region are determined. The number of iterations is the number of times the first line alignment operation is performed. Then, step S83 is executed to calculate the first line attribute similarity and the second line attribute similarity between the third text region and each of the other text regions. Any other text region is any text region other than the third text region. The calculation method for the first line attribute similarity and the second line attribute similarity is described in the previous embodiment and will not be repeated here. Next, step S84 is executed. If the first line attribute similarity of any other text region is less than or equal to the first line attribute similarity threshold, and the second line attribute similarity is less than or equal to the second line attribute similarity threshold, then any other text region is determined as the target text region, and the line attribute of the target text region is determined as the line attribute of the third text region.

[0136] If there is a text region with undetermined line attributes, the first line alignment operation is performed again, that is, steps S82-S84 are re-executed; if there is no text region with undetermined line attributes, the second line alignment operation is performed, which includes steps S85-S86.

[0137] In the second line alignment operation, step S85 is executed first. Based on the line attributes of each text region, the number of text regions corresponding to each line attribute is determined, thereby identifying the text region corresponding to the line attribute with a text region count of 1 as the fourth text region. There is at least one fourth text region. Then, step S86 is executed to calculate the first line attribute similarity and the second line attribute similarity between the fourth text region and each fifth text region. Any fifth text region is any text region other than the fourth text region. Next, step S87 is executed. If the first line attribute similarity corresponding to any fifth text region is less than or equal to the first line attribute similarity threshold, or if the second line attribute similarity is less than or equal to the second line attribute similarity threshold, then any fifth text region is determined as the sixth text region, and the line attribute of the fourth text region is modified to the line attribute of the sixth text region; otherwise, the line attribute of the fourth text region remains unchanged. If the fourth text region meets the modification conditions, then the line attribute of the fourth text region is modified to the line attribute of the sixth text region; if the fourth text region does not meet the modification conditions, then the line attribute of the fourth text region remains unchanged. The modification conditions are: the similarity of the first line attribute corresponding to the fourth text region and any fifth text region is less than or equal to the first line attribute similarity threshold, or the similarity of the second line attribute is less than or equal to the second line attribute similarity threshold.

[0138] If there is a fourth text region whose line attributes have not been redefined, then the second line alignment operation is performed again, that is, steps S85-S87 are repeated; if there is no fourth text region whose line attributes have not been redefined, then the second line alignment operation ends and step S77 is executed.

[0139] In this embodiment, the implementation principles of the first row alignment operation and the second row alignment operation are similar to those of the aforementioned table attribute determination operations, and will not be repeated here.

[0140] In actual implementation, the modification conditions can also be as follows: the similarity of the first row attribute corresponding to any fifth text region is less than or equal to the similarity threshold of the fourth row attribute, and the similarity of the second row attribute is less than or equal to the similarity threshold of the fifth row attribute, wherein the similarity threshold of the fourth row attribute is the same as or different from the similarity threshold of the fifth row attribute, the similarity threshold of the fourth row attribute is greater than the similarity threshold of the first row attribute and / or greater than the similarity threshold of the second row attribute, and the similarity threshold of the fifth row attribute is greater than the similarity threshold of the first row attribute and / or greater than the similarity threshold of the second row attribute.

[0141] In this embodiment, the row attributes of each text region are accurately determined through the first row alignment operation to accurately restore the table structure. Through the second row alignment operation, the text region corresponding to the row attribute with a text region quantity of 1 is re-determined, that is, the row attribute of the text region that is on a separate line is re-determined to correct the row attribute of the text region that is on a separate line, thereby improving the accuracy of the table structure and accurately placing the text recognition result of the text region into the corresponding cell, thereby improving the accuracy of the table recognition result.

[0142] like Figure 9 As shown, Figure 9 This is a structural diagram of an image recognition device 90 provided in an embodiment of this application. The image recognition device 90 includes:

[0143] The first acquisition module 91 is used to acquire a first table image, the first table image containing at least one first text area.

[0144] The first determining module 92 is used to determine the first position information of each first text region.

[0145] The second determining module 93 is used to determine the table attributes of each first text region based on the first position information of each first text region. The table attributes include at least one of row attributes or column attributes.

[0146] The recognition module 94 is used to perform text recognition on each first text region and obtain the text recognition result.

[0147] The second acquisition module 95 is used to acquire the table recognition result corresponding to the first table image based on the text recognition result of each first text region and the table attributes.

[0148] In one possible implementation, the first acquisition module 91 is used to acquire a second table image, the second table image containing a tilted table image region; identify at least one second text region contained in the table image region; determine the rotation angle of the table image region according to the second position information of each second text region; perform rotation correction on the table image region according to the rotation angle of the table image region, and take the corrected table image region as the target table image region; and acquire a first table image based on the target table image region.

[0149] In one possible implementation, the first acquisition module 91 is used to determine the rotation angle of each second text region based on the second position information of each second text region; and to take the rotation angle of the middle position in the sorted rotation angle of each second text region as the rotation angle of the table image region, or to take the average value of the rotation angles of each second text region as the rotation angle of the table image region.

[0150] In one possible implementation, the first acquisition module 91 is used to acquire key content input by the user; in response to the target table image area not containing key content, the target table image area is rotated and corrected according to the target rotation angle, and a first table image is acquired based on the corrected target table image area; in response to the target table image area containing key content, the first table image is acquired based on the target table image area.

[0151] In one possible implementation, each second text region corresponds one-to-one with each first text region. The first determining module 92 is used to determine the center position information of the table image region; determine the position rotation information of each second text region based on the center position information and the rotation angle of the table image region; and determine the first position information of the first text region corresponding to any second text region based on the position rotation information and the second position information of any second text region.

[0152] In one possible implementation, if the table attributes include row attributes or column attributes, the second determining module 93 is used to repeatedly perform the following operations based on each first text region until the table attributes of each first text region are determined:

[0153] Based on the first position information of each first text region, determine the third text region from each first text region, and determine the table attributes of the third text region;

[0154] Calculate the table attribute similarity between the third text region and each of the other text regions, where any other text region is a first text region other than the third text region among the first text regions.

[0155] Based on the similarity of table attributes between the third text region and each other text region, the target text region is determined from each other text region, and the table attributes of the target text region are determined to be the table attributes of the third text region.

[0156] In one possible implementation, the second determining module 93 is used for any of the following:

[0157] For any other text region, using the vertex position information of the third text region and the vertex position information of any other text region, calculate the table attribute similarity between the third text region and any other text region;

[0158] Using the center position information of the third text region and the center position information of any other text region, calculate the table attribute similarity between the third text region and any other text region.

[0159] In one possible implementation, the first table image includes at least one of a shopping list image, a ticket image, an examination report image, and a laboratory report image.

[0160] It should be understood that the above Figure 9 The provided device, in implementing its functions, is only illustrated by the division of the above-described functional modules. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0161] Figure 10 This illustration shows a structural block diagram of a computer device 1000 provided in an exemplary embodiment of this application. The computer device 1000 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The computer device 1000 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0162] Typically, computer device 1000 includes a processor 1001 and a memory 1002.

[0163] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0164] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the image recognition method provided in the method embodiments of this application.

[0165] In some embodiments, the computer device 1000 may optionally include a peripheral device interface 1003 and at least one peripheral device. The processor 1001, memory 1002, and peripheral device interface 1003 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, and a power supply 1009.

[0166] Peripheral device interface 1003 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002 and peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1001, memory 1002 and peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0167] The radio frequency (RF) circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1004 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0168] Display screen 1005 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1005 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1001 for processing. In this case, display screen 1005 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1005 may be a single screen, disposed on the front panel of computer device 1000; in other embodiments, display screen 1005 may be at least two screens, disposed on different surfaces of computer device 1000 or in a folded design; in still other embodiments, display screen 1005 may be a flexible display screen, disposed on a curved or folded surface of computer device 1000. Furthermore, display screen 1005 may also be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0169] The camera assembly 1006 is used to acquire images or videos. Optionally, the camera assembly 1006 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0170] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1001 for processing, or input to the radio frequency circuit 1004 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located in a different part of the computer device 1000. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0171] Power supply 1009 is used to supply power to the various components in computer device 1000. Power supply 1009 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1009 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0172] In some embodiments, the computer device 1000 further includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to, an accelerometer 1011, a gyroscope 1012, a pressure sensor 1013, an optical sensor 1015, and a proximity sensor 1016.

[0173] Accelerometer 1011 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by computer device 1000. For example, accelerometer 1011 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1001 can control display screen 1005 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1011. Accelerometer 1011 can also be used for games or for acquiring user motion data.

[0174] The gyroscope sensor 1012 can detect the orientation and rotation angle of the computer device 1000. The gyroscope sensor 1012 can work in conjunction with the accelerometer sensor 1011 to acquire 3D motion data from the user on the computer device 1000. Based on the data acquired by the gyroscope sensor 1012, the processor 1001 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0175] The pressure sensor 1013 can be disposed on the side bezel of the computer device 1000 and / or on the lower layer of the display screen 1005. When the pressure sensor 1013 is disposed on the side bezel of the computer device 1000, it can detect the user's grip signal on the computer device 1000, and the processor 1001 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1013. When the pressure sensor 1013 is disposed on the lower layer of the display screen 1005, the processor 1001 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0176] An optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 based on the ambient light intensity collected by the optical sensor 1015. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is decreased. In another embodiment, the processor 1001 can also dynamically adjust the shooting parameters of the camera assembly 1006 based on the ambient light intensity collected by the optical sensor 1015.

[0177] The proximity sensor 1016, also known as a distance sensor, is typically installed on the front panel of the computer device 1000. The proximity sensor 1016 is used to detect the distance between the user and the front of the computer device 1000. In one embodiment, when the proximity sensor 1016 detects that the distance between the user and the front of the computer device 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from a screen-on state to a screen-off state; when the proximity sensor 1016 detects that the distance between the user and the front of the computer device 1000 is gradually increasing, the processor 1001 controls the display screen 1005 to switch from a screen-off state to a screen-on state.

[0178] Those skilled in the art will understand that Figure 10 The structure shown does not constitute a limitation on the computer device 1000, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0179] Figure 11This is a schematic diagram of the server structure provided in the embodiments of this application. The server 1100 can vary considerably due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1101 and one or more memories 1102. The one or more memories 1102 store at least one line of program code, which is loaded and executed by the one or more processors 1101 to implement the image recognition methods provided in the various method embodiments described above. Of course, the server 1100 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 1100 may also include other components for implementing device functions, which will not be elaborated here.

[0180] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction that is loaded and executed by a processor to enable a computer to implement any of the above-described image recognition methods.

[0181] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0182] In an exemplary embodiment, a computer program or computer program product is also provided, which stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to implement any of the above-described image recognition methods.

[0183] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0184] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0185] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image recognition method characterized by, The method comprises: obtaining a first table image, the first table image comprising at least one first text region; determining first position information of each first text region; performing a first line alignment operation until determining respective line properties of each first text region, the first line alignment operation comprising: determining a third text region from each first text region according to the first position information of each first text region, determining a line property of the third text region; determining a first line property similarity and a second line property similarity between the third text region and any other text region according to an absolute value of a difference between a vertical coordinate of the third text region and a vertical coordinate of the any other text region, the any other text region being a first text region other than the third text region among the each first text region; if the first line property similarity corresponding to the any other text region is less than or equal to a first line property similarity threshold value, and the second line property similarity corresponding to the any other text region is less than or equal to a second line property similarity threshold value, determining the any other text region as a target text region, and determining the line property of the target text region as the line property of the third text region; performing a second line alignment operation, the second line alignment operation comprising: determining a number of text regions corresponding to each line property based on the line properties of the each first text region, and taking a first text region corresponding to a line property with a number of text regions of 1 as a fourth text region; calculating a first line property similarity and a second line property similarity between the fourth text region and each fifth text region, the fifth text region being a first text region other than the fourth text region among the each first text region; if a first line property similarity corresponding to any fifth text region is less than or equal to a fourth line property similarity threshold value, and a second line property similarity corresponding to the any fifth text region is less than or equal to a fifth line property similarity threshold value, modifying the line property of the fourth text region to be the line property of the any fifth text region, the fourth line property similarity threshold value and the fifth line property similarity threshold value being greater than at least one of the first line property similarity threshold value or the second line property similarity threshold value; performing text recognition on the each first text region to obtain a text recognition result; obtaining a table recognition result corresponding to the first table image according to the text recognition result of the each first text region and the line property.

2. The method of claim 1, wherein, The method comprises: obtaining a second table image, the second table image comprising a table image region with an inclination; recognizing at least one second text region included in the table image region; determining a rotation angle of the table image region according to second position information of each second text region; rotating the table image region according to the rotation angle of the table image region to correct the table image region, and taking the corrected table image region as a target table image region; obtaining the first table image based on the target table image region.

3. The method of claim 2, wherein, The determining the rotation angle of the table image region according to the second position information of each second text region comprises: determining the rotation angle of each second text region according to the second position information of each second text region; taking the rotation angle that is arranged in the middle position in the rotation angles of each second text region as the rotation angle of the table image region, or taking the average value of the rotation angles of each second text region as the rotation angle of the table image region.

4. The method of claim 2, wherein, The obtaining the first table image based on the target table image region comprises: obtaining the key content input by a user; in response to the target table image region not containing the key content, rotating and correcting the target table image region according to a target rotation angle, and obtaining the first table image based on the corrected target table image region; in response to the target table image region containing the key content, obtaining the first table image based on the target table image region.

5. The method according to any one of claims 2 to 4, characterized in that, The each second text region corresponds to the each first text region one by one, and the determining the first position information of each first text region comprises: determining the center position information of the table image region; determining the position rotation information of each second text region according to the center position information and the rotation angle of the table image region; determining the first position information of the first text region corresponding to any second text region according to the position rotation information and the second position information of the any second text region.

6. An image recognition apparatus characterized by comprising: The device comprises: a first obtaining module configured to obtain a first table image, the first table image containing at least one first text region; a first determining module configured to determine first position information of each first text region; a second determining module configured to perform a first time line alignment operation until the line attribute of each first text region is determined, the first time line alignment operation comprising: determining a third text region from each first text region according to the first position information of each first text region, and determining the line attribute of the third text region; determining the first line attribute similarity and the second line attribute similarity between the third text region and any other text region according to the absolute value of the difference between the ordinate of the third text region and the ordinate of the any other text region, the any other text region being a first text region other than the third text region in each first text region; if the first line attribute similarity corresponding to the any other text region is less than or equal to a first line attribute similarity threshold value, and the second line attribute similarity corresponding to the any other text region is less than or equal to a second line attribute similarity threshold value, determining the any other text region as a target text region, and determining the line attribute of the target text region as the line attribute of the third text region. The second determining module is further configured to perform a second line alignment operation, which includes: determining a number of text regions corresponding to each line attribute based on line attributes of the respective first text regions, and taking a first text region corresponding to a line attribute with a number of text regions of 1 as a fourth text region; calculating a first line attribute similarity and a second line attribute similarity between the fourth text region and each fifth text region, the fifth text region being a first text region other than the fourth text region among the respective first text regions; and modifying a line attribute of the fourth text region to a line attribute of any fifth text region if a first line attribute similarity corresponding to the fifth text region is less than or equal to a fourth line attribute similarity threshold value, and a second line attribute similarity corresponding to the fifth text region is less than or equal to a fifth line attribute similarity threshold value, the fourth line attribute similarity threshold value and the fifth line attribute similarity threshold value both being greater than at least one of the first line attribute similarity threshold value or the second line attribute similarity threshold value. The recognition module is configured to perform text recognition on the respective first text regions to obtain text recognition results. The second obtaining module is configured to obtain a table recognition result corresponding to the first table image based on the text recognition results and the line attributes of the respective first text regions.

7. The apparatus of claim 6, wherein, The first obtaining module is configured to obtain a second table image, the second table image including a table image region with an inclination; recognize at least one second text region included in the table image region; determine a rotation angle of the table image region based on second position information of the respective second text regions; and perform rotation correction on the table image region according to the rotation angle of the table image region, and take the corrected table image region as a target table image region. The first obtaining module is configured to obtain the first table image based on the target table image region.

8. The apparatus of claim 7, wherein, The first obtaining module is configured to determine rotation angles of the respective second text regions based on the second position information of the respective second text regions; take a rotation angle located in a middle position in the rotation angles of the respective second text regions as the rotation angle of the table image region, or take an average value of the rotation angles of the respective second text regions as the rotation angle of the table image region.

9. The apparatus of claim 7, wherein, The first obtaining module is configured to obtain key content input by a user; perform rotation correction on the target table image region according to a target rotation angle in response to the target table image region not including the key content, and obtain the first table image based on the corrected target table image region; and obtain the first table image based on the target table image region in response to the target table image region including the key content.

10. The device of any of claims 7-9, wherein, The respective second text regions correspond one-to-one to the respective first text regions. The first determining module is configured to determine center position information of the table image region; and determine position rotation information of the respective second text regions based on the center position information and the rotation angle of the table image region. According to the position rotation information and the second position information of any one of the second text regions, first position information of a first text region corresponding to the any one of the second text regions is determined.

11. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one instruction, which, when executed by the processor, causes the computer device to implement the image recognition method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, which is loaded and executed by the processor to cause the computer to implement the image recognition method according to any one of claims 1 to 5.

Images